# Mapping modifications in index template breaks search queries **URL:** https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977 **Category:** Elasticsearch **Created:** [July 28, 2013, 8:08pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977 "2013-07-28T20:08:17Z") **Posts on this page:** 19 **Page:** 1
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 8:08pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/1 "2013-07-28T20:08:17Z")
I've got an Elasticsearch cluster of two machines both running version 0.20.2. Log data is fed into this cluster from Logstash. This has been running for months without issue. I've recently realised that when I upgraded from a single machine to this cluster I forgot to port over my index template > . So I've implemented that index template which works fine and has properties defined for the standard fields of syslog type data that Logstash would ordinarily feed in. So up until this point everything is working as expected. The problems start when I implement my new index template > which has some additional fields specific to the nginx logs that come through that same process, from Logstash and into the same Elasticsearch index. To explain my problem I'll use these two fields '@message' and the 'useragent'. Both have exactly the same mappings and both should as far as I can see be searchable in the same way but they aren't. I can perform a simple text search on '@message' and get results, if I run a similar search on the 'useragent' field I get no results, even though there seems to be data that should be a match. If I edit the index template and just remove the 'useragent' mapping and then perform a similar search I get the results back ok. Basically if I try and add mapping for any of these fields then they are no longer searchable even though I'm setting the fields up to be searchable in the same way as the '@message' field is defined. For a second yesterday I thought sod it I will just not bother adding any mapping for those new fields, those fields will get indexed automatically and everything will be searchable, no problems. But I NEED to add mapping for some of those other fields because I need things like upstrtime (which is the time the upstream server takes to generate a PHP page) to be numeric so that I can run statistics on that field. For your info, whenever I make modifications to the index template I do so on both servers by modifying the template file, then restart ES on both servers, then once the cluster status is green I delete the index starting fresh. I've double checked that the index template is taking effect by running this command curl -XGET '[http://192.168.11.3:9200/logstash-2013.07.27/\_mapping?pretty=true](http://192.168.11.3:9200/logstash-2013.07.27/_mapping?pretty=true)' and everything looks fine. Has anyone any idea what could be going on here? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 8:45pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/2 "2013-07-28T20:45:14Z")
Could you post a query which is supposée to work? And what kind of values you have your field useragent? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 28 juil. 2013 à 22:08, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : I've got an Elasticsearch cluster of two machines both running version 0.20.2. Log data is fed into this cluster from Logstash. This has been running for months without issue. I've recently realised that when I upgraded from a single machine to this cluster I forgot to port over my index template [https://gist.github.com/WPsites/4685598/7a750337e100d1474e7dc54c1406c8a1da5e4afc](https://gist.github.com/WPsites/4685598/7a750337e100d1474e7dc54c1406c8a1da5e4afc) . So I've implemented that index template which works fine and has properties defined for the standard fields of syslog type data that Logstash would ordinarily feed in. So up until this point everything is working as expected. The problems start when I implement my new index template [https://gist.github.com/WPsites/4685598/cc2b240e26f476e163d97ea92ea9c88a9d5f082a](https://gist.github.com/WPsites/4685598/cc2b240e26f476e163d97ea92ea9c88a9d5f082a) which has some additional fields specific to the nginx logs that come through that same process, from Logstash and into the same Elasticsearch index. To explain my problem I'll use these two fields '@message' and the 'useragent'. Both have exactly the same mappings and both should as far as I can see be searchable in the same way but they aren't. I can perform a simple text search on '@message' and get results, if I run a similar search on the 'useragent' field I get no results, even though there seems to be data that should be a match. If I edit the index template and just remove the 'useragent' mapping and then perform a similar search I get the results back ok. Basically if I try and add mapping for any of these fields then they are no longer searchable even though I'm setting the fields up to be searchable in the same way as the '@message' field is defined. For a second yesterday I thought sod it I will just not bother adding any mapping for those new fields, those fields will get indexed automatically and everything will be searchable, no problems. But I NEED to add mapping for some of those other fields because I need things like upstrtime (which is the time the upstream server takes to generate a PHP page) to be numeric so that I can run statistics on that field. For your info, whenever I make modifications to the index template I do so on both servers by modifying the template file, then restart ES on both servers, then once the cluster status is green I delete the index starting fresh. I've double checked that the index template is taking effect by running this command curl -XGET '[http://192.168.11.3:9200/logstash-2013.07.27/\_mapping?pretty=true](http://192.168.11.3:9200/logstash-2013.07.27/_mapping?pretty=true)' and everything looks fine. ## Has anyone any idea what could be going on here? You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 9:36pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/3 "2013-07-28T21:36:37Z")
The 'useragent' field contains a user agent string like this 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36' The same issue exists for any of the fields that I'm adding mappings for, so something like the 'syslog\_program' field which just contains a simple string such as 'upstream.log' suffers from the same issue, so that's ruling out an issue with special characters I think. Thanks for your input David. I'm using Kibana to search through the logs and you asking about my query has just made me think maybe there is an issue with the Kibana config so I'm just going to take a quick look at that. If I don't find an issue with that then I'll post some query examples on here. Thanks On Sunday, July 28, 2013 9:45:14 PM UTC+1, David Pilato wrote: > Could you post a query which is supposée to work? > And what kind of values you have your field useragent? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 28 juil. 2013 à 22:08, simon\_thepiman \<[si...@wpsites.co.uk](mailto:si...@wpsites.co.uk)\\> > a écrit : > > I've got an Elasticsearch cluster of two machines both running version > 0.20.2. Log data is fed into this cluster from Logstash. > > This has been running for months without issue. I've recently realised > that when I upgraded from a single machine to this cluster I forgot to port > over my index template > [Elasticsearch index template for logstash that contains additional NGINX fields · GitHub](https://gist.github.com/WPsites/4685598/7a750337e100d1474e7dc54c1406c8a1da5e4afc). So I've implemented that index template which works fine and has > properties defined for the standard fields of syslog type data that > Logstash would ordinarily feed in. So up until this point everything is > working as expected. > > The problems start when I implement my new index template > [https://gist.github.com/WPsites/4685598/cc2b240e26f476e163d97ea92ea9c88a9d5f082awhich](https://gist.github.com/WPsites/4685598/cc2b240e26f476e163d97ea92ea9c88a9d5f082awhich) has some additional fields specific to the nginx logs that come > through that same process, from Logstash and into the same Elasticsearch > index. > > To explain my problem I'll use these two fields '@message' and the > 'useragent'. Both have exactly the same mappings and both should as far as > I can see be searchable in the same way but they aren't. I can perform a > simple text search on '@message' and get results, if I run a similar search > on the 'useragent' field I get no results, even though there seems to be > data that should be a match. If I edit the index template and just remove > the 'useragent' mapping and then perform a similar search I get the results > back ok. Basically if I try and add mapping for any of these fields then > they are no longer searchable even though I'm setting the fields up to be > searchable in the same way as the '@message' field is defined. > > For a second yesterday I thought sod it I will just not bother adding any > mapping for those new fields, those fields will get indexed automatically > and everything will be searchable, no problems. But I NEED to add mapping > for some of those other fields because I need things like upstrtime (which > is the time the upstream server takes to generate a PHP page) to be numeric > so that I can run statistics on that field. > > For your info, whenever I make modifications to the index template I do so > on both servers by modifying the template file, then restart ES on both > servers, then once the cluster status is green I delete the index starting > fresh. I've double checked that the index template is taking effect by > running this command curl -XGET ' > [http://192.168.11.3:9200/logstash-2013.07.27/\_mapping?pretty=true](http://192.168.11.3:9200/logstash-2013.07.27/_mapping?pretty=true)' and > everything looks fine. > > Has anyone any idea what could be going on here? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 10:20pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/4 "2013-07-28T22:20:42Z")
You did not set any analyzer. Your fields are analyzed with the standard analyzer (english analyzer). I think that if you search for mozilla, you should get a result. I will start to run queries using curl and only when everything work as you desire, start to play with Kibana. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 28 juil. 2013 à 23:36, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : The 'useragent' field contains a user agent string like this 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36' The same issue exists for any of the fields that I'm adding mappings for, so something like the 'syslog\_program' field which just contains a simple string such as 'upstream.log' suffers from the same issue, so that's ruling out an issue with special characters I think. Thanks for your input David. I'm using Kibana to search through the logs and you asking about my query has just made me think maybe there is an issue with the Kibana config so I'm just going to take a quick look at that. If I don't find an issue with that then I'll post some query examples on here. Thanks On Sunday, July 28, 2013 9:45:14 PM UTC+1, David Pilato wrote: > Could you post a query which is supposée to work? > And what kind of values you have your field useragent? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 28 juil. 2013 à 22:08, simon\_thepiman [si...@wpsites.co.uk](mailto:si...@wpsites.co.uk) a écrit : > > I've got an Elasticsearch cluster of two machines both running version 0.20.2. Log data is fed into this cluster from Logstash. > > This has been running for months without issue. I've recently realised that when I upgraded from a single machine to this cluster I forgot to port over my index template [Elasticsearch index template for logstash that contains additional NGINX fields · GitHub](https://gist.github.com/WPsites/4685598/7a750337e100d1474e7dc54c1406c8a1da5e4afc) . So I've implemented that index template which works fine and has properties defined for the standard fields of syslog type data that Logstash would ordinarily feed in. So up until this point everything is working as expected. > > The problems start when I implement my new index template [Elasticsearch index template for logstash that contains additional NGINX fields · GitHub](https://gist.github.com/WPsites/4685598/cc2b240e26f476e163d97ea92ea9c88a9d5f082a) which has some additional fields specific to the nginx logs that come through that same process, from Logstash and into the same Elasticsearch index. > > To explain my problem I'll use these two fields '@message' and the 'useragent'. Both have exactly the same mappings and both should as far as I can see be searchable in the same way but they aren't. I can perform a simple text search on '@message' and get results, if I run a similar search on the 'useragent' field I get no results, even though there seems to be data that should be a match. If I edit the index template and just remove the 'useragent' mapping and then perform a similar search I get the results back ok. Basically if I try and add mapping for any of these fields then they are no longer searchable even though I'm setting the fields up to be searchable in the same way as the '@message' field is defined. > > For a second yesterday I thought sod it I will just not bother adding any mapping for those new fields, those fields will get indexed automatically and everything will be searchable, no problems. But I NEED to add mapping for some of those other fields because I need things like upstrtime (which is the time the upstream server takes to generate a PHP page) to be numeric so that I can run statistics on that field. > > For your info, whenever I make modifications to the index template I do so on both servers by modifying the template file, then restart ES on both servers, then once the cluster status is green I delete the index starting fresh. I've double checked that the index template is taking effect by running this command curl -XGET '[http://192.168.11.3:9200/logstash-2013.07.27/\_mapping?pretty=true](http://192.168.11.3:9200/logstash-2013.07.27/_mapping?pretty=true)' and everything looks fine. > > ## Has anyone any idea what could be going on here? > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 10:28pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/5 "2013-07-28T22:28:56Z")
Hi David, I've modified my index template slightly to make pretty much all of the fields analyzed. For example my 'method' field is mapped like so: "method": { "type": "string", "index": "analyzed" } And then performing a pretty straight forward looking query directly on the database: curl --globoff '[http://192.168.11.2:9200/logstash-2013.07.28/\_search?q=method:"GET"&pretty=true](http://192.168.11.2:9200/logstash-2013.07.28/_search?q=method:%22GET%22&pretty=true)' { "took" : 6, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max\_score" : null, "hits" : [] } When I run a trend on the 'method' field it comes back with 491 'GET' and 5 'HEAD' so the data is there, it's just not coming back for a search, even searching method:_GET_. I don't get it!? Thanks, Simon -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 10:39pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/6 "2013-07-28T22:39:59Z")
Try this: curl -XGET 'localhost:9200/\_analyze?analyzer=standard' -d 'GET You will see how Elasticsearch index it using default analyzer. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:28, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : Hi David, I've modified my index template slightly to make pretty much all of the fields analyzed. For example my 'method' field is mapped like so: "method": { "type": "string", "index": "analyzed" } And then performing a pretty straight forward looking query directly on the database: curl --globoff '[http://192.168.11.2:9200/logstash-2013.07.28/\_search?q=method:"GET"&pretty=true](http://192.168.11.2:9200/logstash-2013.07.28/_search?q=method:%22GET%22&pretty=true)' { "took" : 6, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max\_score" : null, "hits" : [] } When I run a trend on the 'method' field it comes back with 491 'GET' and 5 'HEAD' so the data is there, it's just not coming back for a search, even searching method:_GET_. I don't get it!? Thanks, ## Simon You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 10:43pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/7 "2013-07-28T22:43:41Z")
curl -XGET '192.168.11.2:9200/\_analyze?analyzer=standard&pretty=true' -d 'GET' { "tokens" : [ { "token" : "get", "start\_offset" : 0, "end\_offset" : 3, "type" : "", "position" : 1 } ] } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 10:47pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/8 "2013-07-28T22:47:36Z")
Get is a stop word in english analyzer. That's the reason it's not indexed. You should use another analyzer for your field. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:43, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : curl -XGET '192.168.11.2:9200/\_analyze?analyzer=standard&pretty=true' -d 'GET' ## { "tokens" : [{ "token" : "get", "start\_offset" : 0, "end\_offset" : 3, "type" : "", "position" : 1 }] } You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 10:47pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/9 "2013-07-28T22:47:57Z")
Forget what I wrote. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:47, David Pilato [david@pilato.fr](mailto:david@pilato.fr) a écrit : Get is a stop word in english analyzer. That's the reason it's not indexed. You should use another analyzer for your field. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:43, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : curl -XGET '192.168.11.2:9200/\_analyze?analyzer=standard&pretty=true' -d 'GET' ## { "tokens" : [{ "token" : "get", "start\_offset" : 0, "end\_offset" : 3, "type" : "", "position" : 1 }] } You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 10:49pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/10 "2013-07-28T22:49:37Z")
Try: curl --globoff '[http://192.168.11.2:9200/logstash-2013.07.28/\_search?q=method:get&pretty=true](http://192.168.11.2:9200/logstash-2013.07.28/_search?q=method:get&pretty=true)' -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:43, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : curl -XGET '192.168.11.2:9200/\_analyze?analyzer=standard&pretty=true' -d 'GET' ## { "tokens" : [{ "token" : "get", "start\_offset" : 0, "end\_offset" : 3, "type" : "", "position" : 1 }] } You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 10:52pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/11 "2013-07-28T22:52:16Z")
curl --globoff '[http://192.168.11.2:9200/logstash-2013.07.28/\_search?q=method:get&pretty=true](http://192.168.11.2:9200/logstash-2013.07.28/_search?q=method:get&pretty=true)' { "took" : 10, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max\_score" : null, "hits" : [] } } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 28, 2013, 11:14pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/12 "2013-07-28T23:14:54Z")
Could you post a document which have GET in method field? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 juil. 2013 à 00:52, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : curl --globoff '[http://192.168.11.2:9200/logstash-2013.07.28/\_search?q=method:get&pretty=true](http://192.168.11.2:9200/logstash-2013.07.28/_search?q=method:get&pretty=true)' ## { "took" : 10, "timed\_out" : false, "\_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max\_score" : null, "hits" : [] } } You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 11:20pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/13 "2013-07-28T23:20:17Z")
Thanks for your assistence here David. Here is a document returned from a search of the '@message' field for _GET_ { "\_index" : "logstash-2013.07.28", "\_type" : "syslog", "\_id" : "JXX9wpWfTiSJ9yR7UZJuXA", "\_score" : 1.0, "\_source" : {"@source":"tcp://172.16.50.11:36874/","@tags":["nginx","nginx\_upstream"],"@fields":{"syslog\_pri":["13"],"syslog\_timestamp":["Jul 28 22:21:51"],"syslog\_hostname":["indesit1"],"syslog\_program":["upstream.log"],"clientip":["90.244.96.35"],"time":["28/Jul/2013:22:21:51 +0000"],"host":["www.hotpointservice.co.uk"],"method":["GET"],"path":["/wp-content/themes/indesit/fonts/opensans/OpenSans-Regular-webfont.woff"],"httpversion":["1.1"],"response":["200"],"bytes":["13999"],"referrer":["[http://www.hotpointservice.co.uk/manual/?id=EXFL1810G"],"useragent":["Mozilla/5.0](http://www.hotpointservice.co.uk/manual/?id=EXFL1810G%22%5D,%22useragent%22:%5B%22Mozilla/5.0) (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36"],"upstr\_ip":["127.0.0.1"],"upstr\_port":["8000"],"upstrcode":["200"],"upstrtime":["0.002"]},"@timestamp":"2013-07-28T22:21:51.000Z","@source\_host":"indesit1","@source\_path":"/","@message":"\<13\>Jul 28 22:21:51 indesit1 upstream.log: :90.244.96.35 - - [28/Jul/2013:22:21:51 +0000] "www.hotpointservice.co.uk" "GET /wp-content/themes/indesit/fonts/opensans/OpenSans-Regular-webfont.woff HTTP/1.1" 200 13999 "[http://www.hotpointservice.co.uk/manual/?id=EXFL1810G](http://www.hotpointservice.co.uk/manual/?id=EXFL1810G)" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36" "127.0.0.1:8000" 200 0.002\n","@type":"syslog"} } ] } } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 28, 2013, 11:36pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/14 "2013-07-28T23:36:19Z")
I've just run: curl -XGET '[http://192.168.11.3:9200/logstash-2013.07.29/\_mapping?pretty=true](http://192.168.11.3:9200/logstash-2013.07.29/_mapping?pretty=true)' To confirm how the mapping is taking effect. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 30, 2013, 1:43pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/15 "2013-07-30T13:43:22Z")
What happens if you search for @fields.method:get ? It should work. But it sounds like you defined a mapping for method field and not for @fields.method. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 29 juil. 2013 à 01:20, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : > Thanks for your assistence here David. > > Here is a document returned from a search of the '@message' field for _GET_ > > { > "\_index" : "logstash-2013.07.28", > "\_type" : "syslog", > "\_id" : "JXX9wpWfTiSJ9yR7UZJuXA", > "\_score" : 1.0, "\_source" : {"@source":"tcp://172.16.50.11:36874/","@tags":["nginx","nginx\_upstream"],"@fields":{"syslog\_pri":["13"],"syslog\_timestamp":["Jul 28 22:21:51"],"syslog\_hostname":["indesit1"],"syslog\_program":["upstream.log"],"clientip":["90.244.96.35"],"time":["28/Jul/2013:22:21:51 +0000"],"host":["[www.hotpointservice.co.uk](http://www.hotpointservice.co.uk)"],"method":["GET"],"path":["/wp-content/themes/indesit/fonts/opensans/OpenSans-Regular-webfont.woff"],"httpversion":["1.1"],"response":["200"],"bytes":["13999"],"referrer":["[http://www.hotpointservice.co.uk/manual/?id=EXFL1810G"],"useragent":["Mozilla/5.0](http://www.hotpointservice.co.uk/manual/?id=EXFL1810G%22%5D,%22useragent%22:%5B%22Mozilla/5.0) (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36"],"upstr\_ip":["127.0.0.1"],"upstr\_port":["8000"],"upstrcode":["200"],"upstrtime":["0.002"]},"@timestamp":"2013-07-28T22:21:51.000Z","@source\_host":"indesit1","@source\_path":"/","@message":"\<13\>Jul 28 22:21:51 indesit1 upstream.log: :90.244.96.35 - - [28/Jul/2013:22:21:51 +0000] "www.hotpointservice.co.uk" "GET /wp-content/themes/indesit/fonts/opensans/OpenSans-Regular-webfont.woff HTTP/1.1" 200 13999 "[http://www.hotpointservice.co.uk/manual/?id=EXFL1810G\](http://www.hotpointservice.co.uk/manual/?id=EXFL1810G%5C)" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36" "127.0.0.1:8000" 200 0.002\n","@type":"syslog"} > } ] > } > } > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [July 30, 2013, 1:58pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/16 "2013-07-30T13:58:34Z")
Amazing, that worked! I thought I was never going to get anywhere with this. Thanks. So do I need to add further mappings somehow to make this work on method:GET rather than @fields.method:GET ? I've seen nothing online about @fields. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [July 30, 2013, 2:00pm UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/17 "2013-07-30T14:00:10Z")
@fields comes from logstash. I don't think you are pushing any value in _method_ field. ## My 2 cents David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 30 juil. 2013 à 15:58, simon\_thepiman [simon@wpsites.co.uk](mailto:simon@wpsites.co.uk) a écrit : > Amazing, that worked! I thought I was never going to get anywhere with this. Thanks. > > So do I need to add further mappings somehow to make this work on method:GET rather than @fields.method:GET ? > > I've seen nothing online about @fields. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![Simon\_thepiman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thepiman/32/2195_2.png) [@Simon\_thepiman](https://discuss.elastic.co/u/Simon_thepiman) #### Post date: [August 1, 2013, 9:28am UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/18 "2013-08-01T09:28:43Z")
Just wanted to say thanks again for your help David. I've updated my gist with the working Elasticsearch index template that takes into account the fact that additional fields created by grok in Logstash are sent into Elasticsearch to be indexed in the '@fields' object > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [July 6, 2017, 2:23am UTC](https://discuss.elastic.co/t/mapping-modifications-in-index-template-breaks-search-queries/12977/19 "2017-07-06T02:23:23Z")