# One particular value in a field isn't indexed **URL:** \ **Category:** Elasticsearch\ **Created:** [February 11, 2014, 3:57pm UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734 "2014-02-11T15:57:38Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![felix\_kofink](https://avatars.discourse-cdn.com/v4/letter/f/bc8723/32.png) [@felix\_kofink](https://discuss.elastic.co/u/felix_kofink)\ **Post date:** [February 11, 2014, 3:57pm UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734/1 "2014-02-11T15:57:38Z")
Hi, this might be a rookie problem since I'm very new to elasticsearch. I'm trying to put JSON documents into elasticsearch with a field "lang". However if "lang" is set to "it" elasticsearch doesn't seem to recognize the field since it's only returned when I filter for missing fields. The problem can be recreated very simple: curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{"lang":"de"}' curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{"lang":"it"}' If I try to search for lang:de: curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d ' { "query": { "query\_string": { "query": "lang:de" } } I get a result: { "took" : 2, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max\_score" : 0.30685282, "hits" : [ { "\_index" : "test", "\_type" : "test", "\_id" : "1", "\_score" : 0.30685282, "\_source" : {"lang":"de"} } ] } } However if I search for lang:it: curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d ' { "query": { "query\_string": { "query": "lang:it" } } }' There is no result: { "took" : 1, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max\_score" : null, "hits" : [] } } If I search for missing field lang: curl -XGET [http://s445.gfsrv.net:9200/test2/test/\_search?pretty](http://s445.gfsrv.net:9200/test2/test/_search?pretty) -d '{"query":{"filtered":{"filter":{"missing":{"field":"lang"}}}}}' There it is: { "took" : 1, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max\_score" : 1.0, "hits" : [ { "\_index" : "test2", "\_type" : "test", "\_id" : "2", "\_score" : 1.0, "\_source" : {"lang":"it"} } ] } } Thanks for your time and any help would be appreciated. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 11, 2014, 5:00pm UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734/2 "2014-02-11T17:00:56Z")
Very rookie problem. 🙂 The default (aka standard) analyzer uses a stopword filter and "it" is a stopword. Try configuring your field with a custom analyzer which does not use stopwords or a custom set of stopwords. Cheers, Ivan On Tue, Feb 11, 2014 at 7:57 AM, [felix.kofink@gameforge.de](mailto:felix.kofink@gameforge.de) wrote: > Hi, > > this might be a rookie problem since I'm very new to elasticsearch. > > I'm trying to put JSON documents into elasticsearch with a field "lang". > However if "lang" is set to "it" elasticsearch doesn't seem to recognize > the field since it's only returned when I filter for missing fields. > The problem can be recreated very simple: > > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{"lang":"de"}' > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{"lang":"it"}' > > If I try to search for lang:de: > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d > ' > { > "query": { > "query\_string": { > "query": "lang:de" > } > } > > I get a result: > { > "took" : 2, > "timed\_out" : false, > "\_shards" : { > "total" : 5, > "successful" : 5, > "failed" : 0 > }, > "hits" : { > "total" : 1, > "max\_score" : 0.30685282, > "hits" : [ { > "\_index" : "test", > "\_type" : "test", > "\_id" : "1", > "\_score" : 0.30685282, "\_source" : {"lang":"de"} > } ] > } > } > > However if I search for lang:it: > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d ' > { > "query": { > "query\_string": { > "query": "lang:it" > } > } > }' > > There is no result: > { > "took" : 1, > "timed\_out" : false, > "\_shards" : { > "total" : 5, > "successful" : 5, > "failed" : 0 > }, > "hits" : { > "total" : 0, > "max\_score" : null, > "hits" : > } > } > > If I search for missing field lang: > curl -XGET [http://s445.gfsrv.net:9200/test2/test/\_search?pretty](http://s445.gfsrv.net:9200/test2/test/_search?pretty) -d > '{"query":{"filtered":{"filter":{"missing":{"field":"lang"}}}}}' > > There it is: > { > "took" : 1, > "timed\_out" : false, > "\_shards" : { > "total" : 5, > "successful" : 5, > "failed" : 0 > }, > "hits" : { > "total" : 1, > "max\_score" : 1.0, > "hits" : [ { > "\_index" : "test2", > "\_type" : "test", > "\_id" : "2", > "\_score" : 1.0, "\_source" : {"lang":"it"} > } ] > } > } > > Thanks for your time and any help would be appreciated. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZKgp6PHf\_Do9PK4nBQbtHzowk6odWon%2BBqPXtL%2BZnqw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBZKgp6PHf_Do9PK4nBQbtHzowk6odWon%2BBqPXtL%2BZnqw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 11, 2014, 5:05pm UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734/3 "2014-02-11T17:05:38Z")
Actually in your case, your search terms probably do not need to be analyzed at all since you are not executing full-text searches on that field. Try setting the field as non\_analyzed and use a term query (which does not analyze search terms). Better yet, using a term filter since filters are faster, can be cached and do not influence scoring. Cheers, Ivan On Tue, Feb 11, 2014 at 9:00 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Very rookie problem. 🙂 > > The default (aka standard) analyzer uses a stopword filter and "it" is a > stopword. Try configuring your field with a custom analyzer which does not > use stopwords or a custom set of stopwords. > > Cheers, > > Ivan > > On Tue, Feb 11, 2014 at 7:57 AM, [felix.kofink@gameforge.de](mailto:felix.kofink@gameforge.de) wrote: > > > Hi, > > > > this might be a rookie problem since I'm very new to elasticsearch. > > > > I'm trying to put JSON documents into elasticsearch with a field "lang". > > However if "lang" is set to "it" elasticsearch doesn't seem to recognize > > the field since it's only returned when I filter for missing fields. > > The problem can be recreated very simple: > > > > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{"lang":"de"}' > > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{"lang":"it"}' > > > > If I try to search for lang:de: > > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d > > ' > > { > > "query": { > > "query\_string": { > > "query": "lang:de" > > } > > } > > > > I get a result: > > { > > "took" : 2, > > "timed\_out" : false, > > "\_shards" : { > > "total" : 5, > > "successful" : 5, > > "failed" : 0 > > }, > > "hits" : { > > "total" : 1, > > "max\_score" : 0.30685282, > > "hits" : [ { > > "\_index" : "test", > > "\_type" : "test", > > "\_id" : "1", > > "\_score" : 0.30685282, "\_source" : {"lang":"de"} > > } ] > > } > > } > > > > However if I search for lang:it: > > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d ' > > { > > "query": { > > "query\_string": { > > "query": "lang:it" > > } > > } > > }' > > > > There is no result: > > { > > "took" : 1, > > "timed\_out" : false, > > "\_shards" : { > > "total" : 5, > > "successful" : 5, > > "failed" : 0 > > }, > > "hits" : { > > "total" : 0, > > "max\_score" : null, > > "hits" : > > } > > } > > > > If I search for missing field lang: > > curl -XGET [http://s445.gfsrv.net:9200/test2/test/\_search?pretty](http://s445.gfsrv.net:9200/test2/test/_search?pretty) -d > > '{"query":{"filtered":{"filter":{"missing":{"field":"lang"}}}}}' > > > > There it is: > > { > > "took" : 1, > > "timed\_out" : false, > > "\_shards" : { > > "total" : 5, > > "successful" : 5, > > "failed" : 0 > > }, > > "hits" : { > > "total" : 1, > > "max\_score" : 1.0, > > "hits" : [ { > > "\_index" : "test2", > > "\_type" : "test", > > "\_id" : "2", > > "\_score" : 1.0, "\_source" : {"lang":"it"} > > } ] > > } > > } > > > > Thanks for your time and any help would be appreciated. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBWn\_%3DXqBX8U\_ngHO23RbVx10eCtvoTSojY334DirhDpA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBWn_%3DXqBX8U_ngHO23RbVx10eCtvoTSojY334DirhDpA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![felix\_kofink](https://avatars.discourse-cdn.com/v4/letter/f/bc8723/32.png) [@felix\_kofink](https://discuss.elastic.co/u/felix_kofink)\ **Post date:** [February 13, 2014, 11:29am UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734/4 "2014-02-13T11:29:50Z")
Hi Ivan, thanks for your help. So as far as I understand I have to do some reindexing now -\> this will take a while. Thanks again and I will do some RTFMing to repent of not RTFMing enough 😉 Cheers Felix Am Dienstag, 11. Februar 2014 18:05:38 UTC+1 schrieb Ivan Brusic: > Actually in your case, your search terms probably do not need to be > analyzed at all since you are not executing full-text searches on that > field. Try setting the field as non\_analyzed and use a term query (which > does not analyze search terms). Better yet, using a term filter since > filters are faster, can be cached and do not influence scoring. > > Cheers, > > Ivan > > On Tue, Feb 11, 2014 at 9:00 AM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com)\ > > > wrote: > > > Very rookie problem. 🙂 > > > > The default (aka standard) analyzer uses a stopword filter and "it" is a > > stopword. Try configuring your field with a custom analyzer which does not > > use stopwords or a custom set of stopwords. > > > > Cheers, > > > > Ivan > > > > On Tue, Feb 11, 2014 at 7:57 AM, \<[felix....@gameforge.de](mailto:felix....@gameforge.de) \\>wrote: > > > > > Hi, > > > > > > this might be a rookie problem since I'm very new to elasticsearch. > > > > > > I'm trying to put JSON documents into elasticsearch with a field "lang". > > > However if "lang" is set to "it" elasticsearch doesn't seem to recognize > > > the field since it's only returned when I filter for missing fields. > > > The problem can be recreated very simple: > > > > > > curl -XPUT '[http://localhost:9200/test/test/1](http://localhost:9200/test/test/1)' -d '{"lang":"de"}' > > > curl -XPUT '[http://localhost:9200/test/test/2](http://localhost:9200/test/test/2)' -d '{"lang":"it"}' > > > > > > If I try to search for lang:de: > > > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d > > > ' > > > { > > > "query": { > > > "query\_string": { > > > "query": "lang:de" > > > } > > > } > > > > > > I get a result: > > > { > > > "took" : 2, > > > "timed\_out" : false, > > > "\_shards" : { > > > "total" : 5, > > > "successful" : 5, > > > "failed" : 0 > > > }, > > > "hits" : { > > > "total" : 1, > > > "max\_score" : 0.30685282, > > > "hits" : [ { > > > "\_index" : "test", > > > "\_type" : "test", > > > "\_id" : "1", > > > "\_score" : 0.30685282, "\_source" : {"lang":"de"} > > > } ] > > > } > > > } > > > > > > However if I search for lang:it: > > > curl -XGET [http://s445.gfsrv.net:9200/test/test/\_search?pretty](http://s445.gfsrv.net:9200/test/test/_search?pretty) -d ' > > > { > > > "query": { > > > "query\_string": { > > > "query": "lang:it" > > > } > > > } > > > }' > > > > > > There is no result: > > > { > > > "took" : 1, > > > "timed\_out" : false, > > > "\_shards" : { > > > "total" : 5, > > > "successful" : 5, > > > "failed" : 0 > > > }, > > > "hits" : { > > > "total" : 0, > > > "max\_score" : null, > > > "hits" : > > > } > > > } > > > > > > If I search for missing field lang: > > > curl -XGET [http://s445.gfsrv.net:9200/test2/test/\_search?pretty](http://s445.gfsrv.net:9200/test2/test/_search?pretty) -d > > > '{"query":{"filtered":{"filter":{"missing":{"field":"lang"}}}}}' > > > > > > There it is: > > > { > > > "took" : 1, > > > "timed\_out" : false, > > > "\_shards" : { > > > "total" : 5, > > > "successful" : 5, > > > "failed" : 0 > > > }, > > > "hits" : { > > > "total" : 1, > > > "max\_score" : 1.0, > > > "hits" : [ { > > > "\_index" : "test2", > > > "\_type" : "test", > > > "\_id" : "2", > > > "\_score" : 1.0, "\_source" : {"lang":"it"} > > > } ] > > > } > > > } > > > > > > Thanks for your time and any help would be appreciated. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1bfddb52-2bbb-4e24-ab71-facbe365f45c%40googlegroups.com) > > > . > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/937117d4-9c54-42e3-955b-b71757396d39%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/937117d4-9c54-42e3-955b-b71757396d39%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:50am UTC](https://discuss.elastic.co/t/one-particular-value-in-a-field-isnt-indexed/15734/5 "2017-07-06T01:50:33Z")