Using elasticsearch to search across multiple fields for exact match

jerkywez · July 10, 2013, 6:07am

I am using elasticsearch to search across multiple fields using the java
elasticsearch client:

BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(text, fields.toArray(new

String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):

BaseQueryBuilder queryBuilder =

QueryBuilders.fuzzyLikeThisFieldQuery("title").likeText(request.getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have run
out of ideas..

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

spinscale · July 10, 2013, 9:35am

Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : { "multi_match" :
{ "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least find
your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a bit
strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell
wesleyarchbell@gmail.comwrote:

I am using elasticsearch to search across multiple fields using the java
elasticsearch client:
BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(text, fields.toArray(new
String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder =
QueryBuilders.fuzzyLikeThisFieldQuery("title").likeText(request.getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have run
out of ideas..

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 10, 2013, 10:44am

If i only include the title in the multimatch search it returns the
correct results:

List<String> fields = new ArrayList<String>();
fields.add("title");
return QueryBuilders.multiMatchQuery(request.getAll(),

fields.toArray(new String[fields.size()]));

Here is an example of one of the documents:

index: book-index
_type: book
_id: 4f02cac7e4b0d44e1bf2f1e3
_score: 10.14605
_source: {
- _id: 4f02cac7e4b0d44e1bf2f1e3
- agentname: null
- agentrole: null
- basicmainsubject: COM051070
- basicversion: 2011
- contributor[0]: {
  - personname: Stephen R. Davis
  - personnameinverted: Davis, Stephen R.
  - contributorrole: A01
  - keynames: Davis
    }
- date: null
- editionnumber: 6
- epubtype: 029
- imprintname: For Dummies
- isbn: 9780470524121
- languagecode: eng
- languagerole: 01
- marketcountry: null
- marketdaterole: null
- marketpublishingstatus: null
- notificationtype: 03
- numberofpages: 432
- numberwithinseries: null
- productclassificationcode: null
- productclassificationtype: null
- productform: DG
- productidentifier: {
  - 15: 9780470524121
  - : 9780470317266
    }
- publicationdate: 2009-04-19T14:00:00.000Z
- publishername: John Wiley & Sons
- publishingrole: 01
- publishingstatus: 04
- publishingstatusnote: null
- recordreference: 047052412X
- rightscountry: null
- rightsterritory: WORLD
- salesrightstype: 01
- subject[0]: {
  - subjectschemeidentifier: 23
  - subjectheadingtext: Programming / C & C++
    }
- subject[1]: {
  - subjectschemeidentifier: 20
  - subjectheadingtext: C++, C++ programming, computer programming,
    C++ compiler, object-oriented programming, oo programming, ANSI standard,
    C++ standard, beginning programming, programming
    }
- supplydetail: {
  - suppliername: John Wiley
  - price[0]: {
    - pricetypecode: 01
    - priceamount: 29.99
    - currencycode: USD
    - countrycode:
      }
  - price[1]: {
    - pricetypecode: 01
    - priceamount: 25.00
    - currencycode: EUR
    - countrycode:
      }
  - price[2]: {
    - pricetypecode: 01
    - priceamount: 19.99
    - currencycode: GBP
    - countrycode:
      }
  - price[3]: {
    - pricetypecode: 01
    - priceamount: 35.99
    - currencycode: CAD
    - countrycode:
      }
  - price[4]: {
    - pricetypecode: 42
    - priceamount: 24.99
    - currencycode: AUD
    - countrycode:
      }
  - price[5]: {
    - pricetypecode: 42
    - priceamount: 12.99
    - currencycode: GBP
    - countrycode:
      }
  - price[6]: {
    - pricetypecode: 42
    - priceamount: 27.99
    - currencycode: NZD
    - countrycode:
      }
  - price[7]: {
    - pricetypecode: 04
    - priceamount: 16.99
    - currencycode: EUR
    - countrycode:
      }
  - price[8]: {
    - pricetypecode: 42
    - priceamount: 16.99
    - currencycode: EUR
    - countrycode:
      }
  - price[9]: {
    - pricetypecode: 41
    - priceamount: 19.99
    - currencycode: USD
    - countrycode:
      }
  - price[10]: {
    - pricetypecode: 42
    - priceamount: 25.99
    - currencycode: SGD
    - countrycode:
      }
  - price[11]: {
    - pricetypecode: 41
    - priceamount: 23.99
    - currencycode: CAD
    - countrycode:
      }
      }
- text: Enter the world of computer programming with this
  step-by-step guide to the C++ language! C++ is a great introduction to
  object-oriented programming, and this friendly guide covers everything you
  need to know and nothing you don’t. You’ll write your first
  program by the end of Chapter 1.
  C++ For Dummies, 6th Edition,
  helps you understand C++ programming from the ground up. It’s full of
  examples to show you how things work, and it even explains
  “why”, so you understand how the pieces fit together. And the
  bonus CD includes a special code editor, an update GNU compiler, and all
  source code from the book to save you time.
  - Learn
    programming lingo and what terms like object-oriented, compiler, and
    executable mean, so you can write a program right away
  - See how to bundle
    sections of your code into modules that can be reused in different
    programs
  - Work with features of object-oriented programming such as
    classes, constructors, and destructors
  - Discover how the concept of
    inheritance is the key to effective C++ programming
  - Work with assignment
    operators, stream I/O, and other more advanced concepts, once you’ve
    grasped the basics
  You’ll discover ten ways to avoid adding
  bugs to your programs, what pointers are and how to use them, how to work
  with strings, and some advanced features new to C++. C++ For Dummies,
  6th Edition gets you up and running with this popular object-oriented
  language.
  Note: CD-ROM/DVD and other supplementary materials are
  not included as part of eBook file.
- textformat: 02
- texttypecode: 18
- title: C++ For Dummies
- titleofseries: null
- titletype: 01
- websitelink: http://www.wiley.com
- yearFirstPublished: null
  }

}

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:

Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : { "multi_match"
: { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least find
your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell <wesleya...@gmail.com<javascript:>

wrote:
I am using elasticsearch to search across multiple fields using the java
elasticsearch client:
BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(text, fields.toArray(new 
String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = 
QueryBuilders.fuzzyLikeThisFieldQuery("title").likeText(request.getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have run
out of ideas..

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 10, 2013, 10:48am

How do i get it to recoginze the '++' not just the 'c' in 'c++' ?

curl http://:9200/_analyze?pretty -d 'c++'
{
"tokens" : [ {
"token" : "c",
"start_offset" : 0,
"end_offset" : 1,
"type" : "",
"position" : 1
} ]

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:

Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : { "multi_match"
: { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least find
your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell <wesleya...@gmail.com<javascript:>

wrote:
I am using elasticsearch to search across multiple fields using the java
elasticsearch client:
BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(text, fields.toArray(new 
String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = 
QueryBuilders.fuzzyLikeThisFieldQuery("title").likeText(request.getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have run
out of ideas..

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

spinscale · July 10, 2013, 12:04pm

Hey,

try changing your anlyzing chain... see the analyzers/tokenizers available

curl
'localhost:9200/_analyze?pretty&tokenizer=whitespace&filters=lowercase' -d
'C++'

--Alex

On Wed, Jul 10, 2013 at 12:48 PM, Wesley Archbell
wesleyarchbell@gmail.comwrote:

How do i get it to recoginze the '++' not just the 'c' in 'c++' ?

curl http://:9200/_analyze?pretty -d 'c++'
{
"tokens" : [ {
"token" : "c",
"start_offset" : 0,
"end_offset" : 1,
"type" : "",
"position" : 1
} ]

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:
Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : { "multi_match"
: { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?**pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least
find your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell wesleya...@gmail.comwrote:
I am using elasticsearch to search across multiple fields using the java
elasticsearch client:
BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(**text,
fields.toArray(new String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = QueryBuilders.**
fuzzyLikeThisFieldQuery("**title").likeText(request.**getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have
run out of ideas..

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

spinscale · July 10, 2013, 12:11pm

Hey,

another small hint. Regarding the array driven field names of your
documents, you may want to read up about nested documents, which should
provide you a better way of doing this kind of queries. See

--Alex

On Wed, Jul 10, 2013 at 12:44 PM, Wesley Archbell
wesleyarchbell@gmail.comwrote:

If i only include the title in the multimatch search it returns the
correct results:
List<String> fields = new ArrayList<String>();
fields.add("title");
return QueryBuilders.multiMatchQuery(request.getAll(),
fields.toArray(new String[fields.size()]));

Here is an example of one of the documents:

index: book-index

_type: book

_id: 4f02cac7e4b0d44e1bf2f1e3

_score: 10.14605

_source: {

_id: 4f02cac7e4b0d44e1bf2f1e3

agentname: null

agentrole: null

basicmainsubject: COM051070

basicversion: 2011

contributor[0]: {

personname: Stephen R. Davis

personnameinverted: Davis, Stephen R.

contributorrole: A01

keynames: Davis
}

date: null

editionnumber: 6

epubtype: 029

imprintname: For Dummies

isbn: 9780470524121

languagecode: eng

languagerole: 01

marketcountry: null

marketdaterole: null

marketpublishingstatus: null

notificationtype: 03

numberofpages: 432

numberwithinseries: null

productclassificationcode: null

productclassificationtype: null

productform: DG

productidentifier: {

15: 9780470524121

: 9780470317266
}

publicationdate: 2009-04-19T14:00:00.000Z

publishername: John Wiley & Sons

publishingrole: 01

publishingstatus: 04

publishingstatusnote: null

recordreference: 047052412X

rightscountry: null

rightsterritory: WORLD

salesrightstype: 01

subject[0]: {

subjectschemeidentifier: 23

subjectheadingtext: Programming / C & C++
}

subject[1]: {

subjectschemeidentifier: 20

subjectheadingtext: C++, C++ programming, computer
programming, C++ compiler, object-oriented programming, oo programming,
ANSI standard, C++ standard, beginning programming, programming
}

supplydetail: {

suppliername: John Wiley

price[0]: {

pricetypecode: 01

priceamount: 29.99

currencycode: USD

countrycode:
}

price[1]: {

pricetypecode: 01

priceamount: 25.00

currencycode: EUR

countrycode:
}

price[2]: {

pricetypecode: 01

priceamount: 19.99

currencycode: GBP

countrycode:
}

price[3]: {

pricetypecode: 01

priceamount: 35.99

currencycode: CAD

countrycode:
}

price[4]: {

pricetypecode: 42

priceamount: 24.99

currencycode: AUD

countrycode:
}

price[5]: {

pricetypecode: 42

priceamount: 12.99

currencycode: GBP

countrycode:
}

price[6]: {

pricetypecode: 42

priceamount: 27.99

currencycode: NZD

countrycode:
}

price[7]: {

pricetypecode: 04

priceamount: 16.99

currencycode: EUR

countrycode:
}

price[8]: {

pricetypecode: 42

priceamount: 16.99

currencycode: EUR

countrycode:
}

price[9]: {

pricetypecode: 41

priceamount: 19.99

currencycode: USD

countrycode:
}

price[10]: {

pricetypecode: 42

priceamount: 25.99

currencycode: SGD

countrycode:
}

price[11]: {

pricetypecode: 41

priceamount: 23.99

currencycode: CAD

countrycode:
}
}

text: Enter the world of computer programming with this
step-by-step guide to the C++ language! C++ is a great introduction to
object-oriented programming, and this friendly guide covers everything you
need to know and nothing you don’t. You’ll write your first
program by the end of Chapter 1.
C++ For Dummies, 6th Edition,
helps you understand C++ programming from the ground up. It’s full of
examples to show you how things work, and it even explains
“why”, so you understand how the pieces fit together. And the
bonus CD includes a special code editor, an update GNU compiler, and all
source code from the book to save you time.
Learn
programming lingo and what terms like object-oriented, compiler, and
executable mean, so you can write a program right away
See how to bundle
sections of your code into modules that can be reused in different
programs
Work with features of object-oriented programming such as
classes, constructors, and destructors
Discover how the concept of
inheritance is the key to effective C++ programming
Work with assignment
operators, stream I/O, and other more advanced concepts, once you’ve
grasped the basics
You’ll discover ten ways to avoid adding
bugs to your programs, what pointers are and how to use them, how to work
with strings, and some advanced features new to C++. C++ For Dummies,
6th Edition gets you up and running with this popular object-oriented
language.
Note: CD-ROM/DVD and other supplementary materials are
not included as part of eBook file.

textformat: 02

texttypecode: 18

title: C++ For Dummies

titleofseries: null

titletype: 01

websitelink: http://www.wiley.com

yearFirstPublished: null
}

}

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:
Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : { "multi_match"
: { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?**pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least
find your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell wesleya...@gmail.comwrote:
I am using elasticsearch to search across multiple fields using the java
elasticsearch client:
BaseQueryBuilder queryBuilder;
List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(**text,
fields.toArray(new String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = QueryBuilders.**
fuzzyLikeThisFieldQuery("**title").likeText(request.**getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have
run out of ideas..

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 10, 2013, 11:49pm

Ok greart thanks, How do i create my own analyzer to be used in the
elasticsearch java api? I can see you can specify an analyzer on the
multimatch query but not sure how to create this:

QueryBuilders.multiMatchQuery(request.getAll(), fields.toArray(new
String[fields.size()])).analyzer("my-analyser");

On Wednesday, July 10, 2013 10:04:58 PM UTC+10, Alexander Reelsen wrote:

Hey,

try changing your anlyzing chain... see the analyzers/tokenizers available

curl
'localhost:9200/_analyze?pretty&tokenizer=whitespace&filters=lowercase' -d
'C++'

--Alex

On Wed, Jul 10, 2013 at 12:48 PM, Wesley Archbell <wesleya...@gmail.com<javascript:>

wrote:
How do i get it to recoginze the '++' not just the 'c' in 'c++' ?

curl http://:9200/_analyze?pretty -d 'c++'
{
"tokens" : [ {
"token" : "c",
"start_offset" : 0,
"end_offset" : 1,
"type" : "",
"position" : 1
} ]

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:
Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : {
"multi_match" : { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?**pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least
find your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell wesleya...@gmail.comwrote:
I am using elasticsearch to search across multiple fields using the
java elasticsearch client:
BaseQueryBuilder queryBuilder;
 List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(**text, 
fields.toArray(new String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = QueryBuilders.**
fuzzyLikeThisFieldQuery("**title").likeText(request.**getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have
run out of ideas..

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 10, 2013, 11:59pm

Would this work?

client.admin().indices().prepareCreate("book-index")

.setSettings(ImmutableSettings.settingsBuilder().loadFromSource(jsonBuilder()
.startObject()
.startObject("analysis")
.startObject("analyzer")
.startObject("my-analyzer")
.field("type", "custom")
.field("tokenizer", "whitespace")
.field("filter", new String{"lowercase"})
.endObject()
.endObject()
.endObject()
.endObject().string()))
.execute().actionGet();

On Thursday, July 11, 2013 9:49:58 AM UTC+10, Wesley Archbell wrote:

Ok greart thanks, How do i create my own analyzer to be used in the
elasticsearch java api? I can see you can specify an analyzer on the
multimatch query but not sure how to create this:

QueryBuilders.multiMatchQuery(request.getAll(), fields.toArray(new
String[fields.size()])).analyzer("my-analyser");

On Wednesday, July 10, 2013 10:04:58 PM UTC+10, Alexander Reelsen wrote:
Hey,

try changing your anlyzing chain... see the analyzers/tokenizers available

curl
'localhost:9200/_analyze?pretty&tokenizer=whitespace&filters=lowercase' -d
'C++'

--Alex

On Wed, Jul 10, 2013 at 12:48 PM, Wesley Archbell wesleya...@gmail.comwrote:
How do i get it to recoginze the '++' not just the 'c' in 'c++' ?

curl http://:9200/_analyze?pretty -d 'c++'
{
"tokens" : [ {
"token" : "c",
"start_offset" : 0,
"end_offset" : 1,
"type" : "",
"position" : 1
} ]

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:
Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : {
"multi_match" : { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?**pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least
find your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one
field and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell wesleya...@gmail.comwrote:
I am using elasticsearch to search across multiple fields using the
java elasticsearch client:
BaseQueryBuilder queryBuilder;
 List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(**text, 
fields.toArray(new String[fields.size()]));

Say the title for a document is "Programming in C++" and i search
'C++' with the logic above it does not bring the book up in the results,
how can i refine this query to better find the documents with the word
'C++' in it? I thought it was the special characters in the text but if i
search just on the title field using the following logic (on the single
title field) the results come up correctly (books with the word 'C++' in it
come up in top results):
BaseQueryBuilder queryBuilder = QueryBuilders.**
fuzzyLikeThisFieldQuery("**title").likeText(request.**getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have
run out of ideas..

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Dan_Everton · July 11, 2013, 1:37am

Should work. Though I'd change .field("filter", new String[] { "lowercase"
}) to .array("filter", "lowercase")

You will also have to change your index mapping so that the fields you want
to search on are analyzed with your custom analyzer. And definitely look in
to nested documents for your price array.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 11, 2013, 1:43am

LOL! i didnt expect you to respond.. good to hear from ya

I could not run the above code as the index already exists so i tried to
run this:

Ive tried updating the existing book-index with a custom analyzer:

curl -XPOST 'http://localhost:9200/book-index/_close'
curl -XPUT 'http:///localhost:9200/book-index/_settings' -d '
{
"settings": {
"analysis": {
"analyzer": {
"book-index-analyser" : {
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase"]
}
}
}
}
}'
curl -XPOST 'http://localhost:9200/book-index/_open'

I then try create a multimatch:

QueryBuilders.multiMatchQuery(request.getAll(), fields.toArray(new
String[fields.size()])).analyzer("book-index-analyzer")

But im still not getting my books with 'C++' in top results..

Am i missing something when creating the settings for the index?

On Thursday, July 11, 2013 11:37:34 AM UTC+10, Dan Everton wrote:

Should work. Though I'd change .field("filter", new String { "lowercase"
}) to .array("filter", "lowercase")

You will also have to change your index mapping so that the fields you
want to search on are analyzed with your custom analyzer. And definitely
look in to nested documents for your price array.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 11, 2013, 3:48am

Ive tried the following to try and update an existing index with a custom
analyzer and then mapping the title field to use it when searching but
still not getting the desired results... Does this look right:

curl -XPOST 'http://localhost:9200/book-index/_close'
curl -XPUT 'http://localhost:9200/book-index/_settings' -d '
{
"settings": {
"analysis": {
"analyzer": {
"title-analyser" : {
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase"]
}
}
}
},
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"title" : { "type" : "string", "search_analyzer" :
"title-analyser" }
}
}
}
}'
curl -XPOST 'http://localhost:9200/book-index/_open'

On Thursday, July 11, 2013 11:37:34 AM UTC+10, Dan Everton wrote:

Should work. Though I'd change .field("filter", new String { "lowercase"
}) to .array("filter", "lowercase")

You will also have to change your index mapping so that the fields you
want to search on are analyzed with your custom analyzer. And definitely
look in to nested documents for your price array.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 11, 2013, 4:05am

Here is the multimatch query in full which returns no results, it does
return results if you remove the '++':

{
"size" : 20, "query" : {
"multi_match" : {
"query" : "C++",
"fields" : [ "isbn", "title", "contributor[0].personname",
"contributor[1].personname", "contributor[2].personname",
"contributor[3].personname", "contributor[4].personname",
"contributor[5].personname", "contributor[6].personname",
"subject[0].subjectcode", "subject[1].subjectcode",
"subject[2].subjectcode", "subject[3].subjectcode",
"subject[4].subjectcode", "subject[5].subjectcode",
"subject[6].subjectcode", "subject[7].subjectcode",
"subject[8].subjectcode", "subject[9].subjectcode",
"subject[10].subjectcode" ],
"analyzer" : "title-analyser"
}
}

On Thursday, July 11, 2013 1:48:40 PM UTC+10, Wesley Archbell wrote:

Ive tried the following to try and update an existing index with a custom
analyzer and then mapping the title field to use it when searching but
still not getting the desired results... Does this look right:

curl -XPOST 'http://localhost:9200/book-index/_close'
curl -XPUT 'http://localhost:9200/book-index/_settings' -d '
{
"settings": {
"analysis": {
"analyzer": {
"title-analyser" : {
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase"]
}
}
}
},
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"title" : { "type" : "string", "search_analyzer" :
"title-analyser" }
}
}
}
}'
curl -XPOST 'http://localhost:9200/book-index/_open'

On Thursday, July 11, 2013 11:37:34 AM UTC+10, Dan Everton wrote:

Should work. Though I'd change .field("filter", new String {
"lowercase" }) to .array("filter", "lowercase")

You will also have to change your index mapping so that the fields you
want to search on are analyzed with your custom analyzer. And definitely
look in to nested documents for your price array.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 11, 2013, 4:40am

This is what my mapping chain looks like for index and title field:

{

state: open
settings: {
- index.analysis.analyzer.title-analyser.tokenizer: whitespace
- index.analysis.analyzer.title-analyser.filter.0: lowercase
- index.analysis.analyzer.title-analyser.type: custom
- index.number_of_shards: 5
- index.number_of_replicas: 1
- index.version.created: 900099
  }
mappings: {
- book: {
  - _source: {
    - enabled: false
      }
  - properties: {
    - title: {
      - search_analyzer: title-analyser
      - type: string
        }
        }
        }
        }
aliases:

}

But when i try see if it has applied the analyzer to the title field it
does not work:

curl -XGET 'localhost:9200/book-index/_analyze?field=book.title' -d 'c++'

{"tokens":[{"token":"c"
,"start_offset":0,"end_offset":1,"type":"","position":1}]}

On Wednesday, July 10, 2013 10:04:58 PM UTC+10, Alexander Reelsen wrote:

Hey,

try changing your anlyzing chain... see the analyzers/tokenizers available

curl
'localhost:9200/_analyze?pretty&tokenizer=whitespace&filters=lowercase' -d
'C++'

--Alex

On Wed, Jul 10, 2013 at 12:48 PM, Wesley Archbell <wesleya...@gmail.com<javascript:>

wrote:
How do i get it to recoginze the '++' not just the 'c' in 'c++' ?

curl http://:9200/_analyze?pretty -d 'c++'
{
"tokens" : [ {
"token" : "c",
"start_offset" : 0,
"end_offset" : 1,
"type" : "",
"position" : 1
} ]

On Wednesday, July 10, 2013 7:35:18 PM UTC+10, Alexander Reelsen wrote:
Hey Wesley

lets try to dissect your query a bit and see if it can be made working.
This is what I tried to reproduce (using curl) your case

curl -X PUT localhost:9200/books/book/1 -d '{ "isbn" :"5654353454353",
"title":"Programming in C" }'
curl -X PUT localhost:9200/books/book/2 -d '{ "isbn" :"5654353454351",
"title":"Programming in C++" }'
curl -X POST localhost:9200/books/_search -d '{ "query" : {
"multi_match" : { "query" : "c++", "fields": ["title","author"] } } }'

This query matches both books, because C++ gets tokenized to c, you can
easily check via
curl 'localhost:9200/_analyze?**pretty' -d 'C++'

So, even though this query returns a bit too much, it should at least
find your c++. The question now is, what is different with your setup?
Have you tried executing your multifield search with only this one field
and check if it fails as well? Maybe your mapping is different?

On another note, your queries for contributors and subjectcodes looks a
bit strange. Can you maybe show the corresponding data you are indexing in
order to make sure that the fields you are querying actually match (maybe
that's the problem, you never know).

Hope this helps, if not, just provide more data!

--Alex

On Wed, Jul 10, 2013 at 8:07 AM, Wesley Archbell wesleya...@gmail.comwrote:
I am using elasticsearch to search across multiple fields using the
java elasticsearch client:
BaseQueryBuilder queryBuilder;
 List<String> fields = new ArrayList<String>();
fields.add("isbn");
fields.add("title");
for (int i = 0; i < 7; i++) {
    fields.add("contributor[" + i + "].personname");
}
for (int i = 0; i < 11; i++) {
    fields.add("subject[" + i + "].subjectcode");
}
queryBuilder = QueryBuilders.multiMatchQuery(**text, 
fields.toArray(new String[fields.size()]));

Say the title for a document is "Programming in C++" and i search 'C++'
with the logic above it does not bring the book up in the results, how can
i refine this query to better find the documents with the word 'C++' in it?
I thought it was the special characters in the text but if i search just on
the title field using the following logic (on the single title field) the
results come up correctly (books with the word 'C++' in it come up in top
results):
BaseQueryBuilder queryBuilder = QueryBuilders.**
fuzzyLikeThisFieldQuery("**title").likeText(request.**getTitle());

Ive tried all sorts of things to try get search on multiple fields to
return books with title that contains words like 'C++' or 'C#' in it i.e.
boosting query that boosts the title field higher but then when searching
by contributor (author) it favours title over author and brings back books
who's title matches instead of author.

If anyone can help that would be great! Ive tried everything and have
run out of ideas..

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.

For more options, visit https://groups.google.com/**groups/opt_out https://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Dan_Everton · July 11, 2013, 5:35am

You've only set the search time analyzer not the index time one. So while
you're searching for C++ what's getting indexed is just C because indexing
is using the standard analyzer. Try just setting the analyzer field to
title-analyzer or set both search_analyzer and index_analyzer to
title-analyzer.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jerkywez · July 11, 2013, 5:37am

Yeah that seemed to help, i finally figured it about 15 minutes ago... what
a relief, at least i now understand a bit more of how ES hangs together..
thanks Dan

On Thursday, July 11, 2013 3:35:52 PM UTC+10, Dan Everton wrote:

You've only set the search time analyzer not the index time one. So while
you're searching for C++ what's getting indexed is just C because indexing
is using the standard analyzer. Try just setting the analyzer field to
title-analyzer or set both search_analyzer and index_analyzer to
title-analyzer.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.