termQuery - how does it work?

Hi,

I am still puzzled by the behaviour of the termQuery. The best way to
explain it is by showing you an example:

I have created a record in elasticsearch db using an example of tutorial
“ElasticSearch in 5 minutes”.

I used the following command:

"http://localhost:9200/blog/user/dilbert" with the body "{ "name" :
"Dilbert Brown" }" (I used postman instead of curl but the final result is
the same)

So my record in elasticsearch db looks like this:

{

"took": 9,

"timed_out": false,

"_shards": {

    "total": 5,

    "successful": 5,

    "failed": 0

},

"hits": {

    "total": 1,

    "max_score": 1,

    "hits": [

        {

            "_index": "blog",

            "_type": "user",

            "_id": "dilbert",

            "_score": 1,

            "_source": {

                "name": "Dilbert Brown"

            }

        }

    ]

}

}

I am trying to retrieve the “name” with value “Dilbert Brown” using
termQuery. My code looks like this:

  TransportClient client = *new* TransportClient(ImmutableSettings.*

settingsBuilder*().put(

      "cluster.name", clusterName).build());

  client.addTransportAddress(*new* InetSocketTransportAddress(

"localhost", 9300));

  SearchResponse response = client.prepareSearch("blog").setTypes("user"

).setSearchType(

      SearchType.*DEFAULT*).setQuery(QueryBuilders.*termQuery*("name", "Dilbert 

Brown")).setFrom(0)

      .setSize(60).setExplain(*true*).execute().actionGet();

  SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.

When I replace (as suggested before by Ivan) QueryBuilders.termQuery(
"name", "Dilbert Brown") by QueryBuilders.queryString("name:Dilbert
Brown") - it does work – but why?

I do have a field “name” with the precisely value “Dilbert Brown” – isn’t
it what termQuery is supposed to do – match the elasticsearch db record’s
field value with the value specified in a query?

Regards,

Janusz

--

You have to understand some basic concepts.

When you index something like "Dilbert Brown", a default analyzer is applied to
break your field into terms.
In short, "Dilbert Brown" is indexed as ["dilbert","brown"]

A QueryString analyze your query before search. So, if you search for "Dilbert
Brown", in fact it will search for "dilbert" or "brown" in the index. And your
document will match.
A TermQuery is not analyzed. If you search for "Dilbert Brown", you won't find
this "term" in the index as it has not been indexed as is.

If you don't want let ES analyze your term when indexing, you have to define a
mapping and set your field as not analyzed. That way, your field will be indexed
in "Dilbert Brown" (case sensitive).

Let me say that when you come to search engine, you have to think search and
forget all what you know about relational databases (and SQL queries).
You have to think about "how I create my index" and "how do I search in my
index".

I hope this will help you
David.

Le 26 décembre 2012 à 10:00, JD jdalecki@tycoint.com a écrit :

Hi,

I am still puzzled by the behaviour of the termQuery. The best way to explain
it is by showing you an example:

I have created a record in elasticsearch db using an example of tutorial
“ElasticSearch in 5 minutes”.

I used the following command:

"http://localhost:9200/blog/user/dilbert
http://localhost:9200/blog/user/dilbert " with the body "{ "name" : "Dilbert
Brown" }" (I used postman instead of curl but the final result is the same)

So my record in elasticsearch db looks like this:

{

 "took": 9,

 "timed_out": false,

 "_shards": {

     "total": 5,

     "successful": 5,

     "failed": 0

 },

 "hits": {

     "total": 1,

     "max_score": 1,

     "hits": [

         {

             "_index": "blog",

             "_type": "user",

             "_id": "dilbert",

             "_score": 1,

             "_source": {

                 "name": "Dilbert Brown"

             }

         }

     ]

 }

}

I am trying to retrieve the “name” with value “Dilbert Brown” using
termQuery. My code looks like this:

   TransportClient client = new

TransportClient(ImmutableSettings.settingsBuilder().put(

       "cluster.name", clusterName).build());

   client.addTransportAddress(new InetSocketTransportAddress("localhost",

9300));

   SearchResponse response =

client.prepareSearch("blog").setTypes("user").setSearchType(

       SearchType.DEFAULT).setQuery(QueryBuilders.termQuery("name",

"Dilbert Brown")).setFrom(0)

       .setSize(60).setExplain(true).execute().actionGet();

   SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.

When I replace (as suggested before by Ivan) QueryBuilders.termQuery("name",
"Dilbert Brown") by QueryBuilders.queryString("name:Dilbert Brown") - it does
work – but why?

I do have a field “name” with the precisely value “Dilbert Brown” – isn’t it
what termQuery is supposed to do – match the elasticsearch db record’s field
value with the value specified in a query?

Regards,

Janusz

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

Thanks, that really helps to understand the mechanism that works behind the
scenes.
Is there any Java example to play with that does it - I mean defining a
mapping and set your field as not analyzed.

Is there any command that I could use to see the mapping stored in the
elasticsearch database (is the _analyze the only one or there is something
else?)

Regards,

Janusz

On Wednesday, December 26, 2012 8:00:09 PM UTC+11, JD wrote:

Hi,

I am still puzzled by the behaviour of the termQuery. The best way to
explain it is by showing you an example:

I have created a record in elasticsearch db using an example of tutorial
“ElasticSearch in 5 minutes”.

I used the following command:

"http://localhost:9200/blog/user/dilbert" with the body "{ "name" :
"Dilbert Brown" }" (I used postman instead of curl but the final result
is the same)

So my record in elasticsearch db looks like this:

{

"took": 9,

"timed_out": false,

"_shards": {

    "total": 5,

    "successful": 5,

    "failed": 0

},

"hits": {

    "total": 1,

    "max_score": 1,

    "hits": [

        {

            "_index": "blog",

            "_type": "user",

            "_id": "dilbert",

            "_score": 1,

            "_source": {

                "name": "Dilbert Brown"

            }

        }

    ]

}

}

I am trying to retrieve the “name” with value “Dilbert Brown” using
termQuery. My code looks like this:

  TransportClient client = *new* TransportClient(ImmutableSettings.*

settingsBuilder*().put(

      "cluster.name", clusterName).build());

  client.addTransportAddress(*new* InetSocketTransportAddress(

"localhost", 9300));

  SearchResponse response = client.prepareSearch("blog").setTypes(

"user").setSearchType(

      SearchType.*DEFAULT*).setQuery(QueryBuilders.*termQuery*("name", 

"Dilbert Brown")).setFrom(0)

      .setSize(60).setExplain(*true*).execute().actionGet();

  SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.

When I replace (as suggested before by Ivan) QueryBuilders.termQuery(
"name", "Dilbert Brown") by QueryBuilders.queryString("name:Dilbert
Brown") - it does work – but why?

I do have a field “name” with the precisely value “Dilbert Brown” – isn’t
it what termQuery is supposed to do – match the elasticsearch db record’s
field value with the value specified in a query?

Regards,

Janusz

--

Look here: http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html

For java, do you use Spring?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 déc. 2012 à 06:59, JD jdalecki@tycoint.com a écrit :

Thanks, that really helps to understand the mechanism that works behind the scenes.
Is there any Java example to play with that does it - I mean defining a mapping and set your field as not analyzed.
Is there any command that I could use to see the mapping stored in the elasticsearch database (is the _analyze the only one or there is something else?)
Regards,
Janusz

On Wednesday, December 26, 2012 8:00:09 PM UTC+11, JD wrote:

Hi,
I am still puzzled by the behaviour of the termQuery. The best way to explain it is by showing you an example:
I have created a record in elasticsearch db using an example of tutorial “ElasticSearch in 5 minutes”.

I used the following command:
"http://localhost:9200/blog/user/dilbert" with the body "{ "name" : "Dilbert Brown" }" (I used postman instead of curl but the final result is the same)

So my record in elasticsearch db looks like this:

{
"took": 9,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "blog",
"_type": "user",
"_id": "dilbert",
"_score": 1,
"_source": {
"name": "Dilbert Brown"
}
}
]
}
}

I am trying to retrieve the “name” with value “Dilbert Brown” using termQuery. My code looks like this:
TransportClient client = new TransportClient(ImmutableSettings.settingsBuilder().put(
"cluster.name", clusterName).build());
client.addTransportAddress(new InetSocketTransportAddress("localhost", 9300));
SearchResponse response = client.prepareSearch("blog").setTypes("user").setSearchType(
SearchType.DEFAULT).setQuery(QueryBuilders.termQuery("name", "Dilbert Brown")).setFrom(0)
.setSize(60).setExplain(true).execute().actionGet();
SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.
When I replace (as suggested before by Ivan) QueryBuilders.termQuery("name", "Dilbert Brown") by QueryBuilders.queryString("name:Dilbert Brown") - it does work – but why?
I do have a field “name” with the precisely value “Dilbert Brown” – isn’t it what termQuery is supposed to do – match the elasticsearch db record’s field value with the value specified in a query?

Regards,
Janusz

--

--

Hi,
David, we are trying to use elasticsearch Java API. We don't use spring at
all?
Thanks for pointing out _mapping command - is there any command that could
display how the indexed document is actually stored?
I tried _analyze but it just shows how it would stored the indexed string,
but if I have already indexed document - can I view how elasticsearch
stored the indexed version of it?
Regards,
Janusz

On Wednesday, December 26, 2012 8:00:09 PM UTC+11, JD wrote:

Hi,

I am still puzzled by the behaviour of the termQuery. The best way to
explain it is by showing you an example:

I have created a record in elasticsearch db using an example of tutorial
“ElasticSearch in 5 minutes”.

I used the following command:

"http://localhost:9200/blog/user/dilbert" with the body "{ "name" :
"Dilbert Brown" }" (I used postman instead of curl but the final result
is the same)

So my record in elasticsearch db looks like this:

{

"took": 9,

"timed_out": false,

"_shards": {

    "total": 5,

    "successful": 5,

    "failed": 0

},

"hits": {

    "total": 1,

    "max_score": 1,

    "hits": [

        {

            "_index": "blog",

            "_type": "user",

            "_id": "dilbert",

            "_score": 1,

            "_source": {

                "name": "Dilbert Brown"

            }

        }

    ]

}

}

I am trying to retrieve the “name” with value “Dilbert Brown” using
termQuery. My code looks like this:

  TransportClient client = *new* TransportClient(ImmutableSettings.*

settingsBuilder*().put(

      "cluster.name", clusterName).build());

  client.addTransportAddress(*new* InetSocketTransportAddress(

"localhost", 9300));

  SearchResponse response = client.prepareSearch("blog").setTypes(

"user").setSearchType(

      SearchType.*DEFAULT*).setQuery(QueryBuilders.*termQuery*("name", 

"Dilbert Brown")).setFrom(0)

      .setSize(60).setExplain(*true*).execute().actionGet();

  SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.

When I replace (as suggested before by Ivan) QueryBuilders.termQuery(
"name", "Dilbert Brown") by QueryBuilders.queryString("name:Dilbert
Brown") - it does work – but why?

I do have a field “name” with the precisely value “Dilbert Brown” – isn’t
it what termQuery is supposed to do – match the elasticsearch db record’s
field value with the value specified in a query?

Regards,

Janusz

--

Ok. So, let me answer to the question: how to push mapping in Java.
I just added a TestCase here:
https://github.com/elasticsearchfr/elasticsearch-java-tests/blob/master/src/test/java/org/elasticsearchfr/tests/ES003PutMappingNotAnalyzedTest.java
https://github.com/elasticsearchfr/elasticsearch-java-tests/blob/master/src/test/java/org/elasticsearchfr/tests/ES003PutMappingNotAnalyzedTest.java

Your other question was: how to know how ES indexed our docs? Have a look at
Luke http://code.google.com/p/luke/ . It could help you.

You can also run a facet on the desired field. Something like:

curl -XPOST localhost:9200/_search -d
'{"query":{"match_all":{}},"facets":{"tag":{"terms":{"field":"yourfieldhere"}}},"size":0}'

The best option is for me to use the _analyze API as you did.

David

Le 27 décembre 2012 à 13:35, JD jdalecki@tycoint.com a écrit :

Hi,
David, we are trying to use elasticsearch Java API. We don't use spring at
all?
Thanks for pointing out _mapping command - is there any command that could
display how the indexed document is actually stored?
I tried _analyze but it just shows how it would stored the indexed string,
but if I have already indexed document - can I view how elasticsearch stored
the indexed version of it?
Regards,
Janusz

On Wednesday, December 26, 2012 8:00:09 PM UTC+11, JD wrote:

Hi,

I am still puzzled by the behaviour of the termQuery. The best way to
explain it is by showing you an example:

I have created a record in elasticsearch db using an example of tutorial
“ElasticSearch in 5 minutes”.

I used the following command:

"http://localhost:9200/blog/user/dilbert"
http://localhost:9200/blog/user/dilbert
http://localhost:9200/blog/user/dilbert with the body "{ "name" : "Dilbert
Brown" }" (I used postman instead of curl but the final result is the same)
http://localhost:9200/blog/user/dilbert

So my record in elasticsearch db looks like this:

{

   "took": 9,

   "timed_out": false,

   "_shards": {

       "total": 5,

       "successful": 5,

       "failed": 0

   },

   "hits": {

       "total": 1,

       "max_score": 1,

       "hits": [

           {

               "_index": "blog",

               "_type": "user",

               "_id": "dilbert",

               "_score": 1,

               "_source": {

                   "name": "Dilbert Brown"

               }

           }

       ]

   }

}

I am trying to retrieve the “name” with value “Dilbert Brown” using
termQuery. My code looks like this:

     TransportClient client = new

TransportClient(ImmutableSettings.settingsBuilder().put(

         " <http://localhost:9200/blog/user/dilbert> cluster.name

http://cluster.name ", clusterName).build());

     client.addTransportAddress(new

InetSocketTransportAddress("localhost", 9300));

     SearchResponse response =

client.prepareSearch("blog").setTypes("user").setSearchType(

         SearchType.DEFAULT).setQuery(QueryBuilders.termQuery("name",

"Dilbert Brown")).setFrom(0)

         .setSize(60).setExplain(true).execute().actionGet();

     SearchHit[] docs = response.getHits().getHits();

I am always getting 0 results – so docs.length is always 0.

When I replace (as suggested before by Ivan)
QueryBuilders.termQuery("name", "Dilbert Brown") by
QueryBuilders.queryString("name:Dilbert Brown") - it does work – but why?

I do have a field “name” with the precisely value “Dilbert Brown” – isn’t
it what termQuery is supposed to do – match the elasticsearch db record’s
field value with the value specified in a query?

Regards,

Janusz

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--