# Searching indexed fields without analysing **URL:** \ **Category:** Elasticsearch\ **Created:** [January 8, 2014, 3:26pm UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144 "2014-01-08T15:26:03Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Chris\_H\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Chris\_H\_2](https://discuss.elastic.co/u/Chris_H_2)\ **Post date:** [January 8, 2014, 3:26pm UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/1 "2014-01-08T15:26:03Z")
Hi. I've deployed elasticsearch with logstash and kibana to take in Windows logs from my OSSEC log server, following this guide: [http://vichargrave.com/ossec-log-management-with-elasticsearch/](http://vichargrave.com/ossec-log-management-with-elasticsearch/) I've tweaked the logstash config to extract some specific fields from the logs, such as User\_Name. I'm having some issues searching on these fields though. These searches work as expected: - User\_Name: \* - User\_Name: john.smith - User\_Name: john.\* - NOT User\_Name: john.\* But I'm having problems with Computer accounts, which take the format "w-dc-01$" - they're being split on the "-" and the "$" is ignored. So a search for "w-dc-01" returns all the servers named "w-". Also I can't do "NOT User\_Name: \*$" to exclude computer accounts. The mappings are created automatically by logstash, and GET /logstash-2014.01.08/\_mapping shows: "User\_Name": { "type": "multi\_field", "fields": { "User\_Name": { "type": "string", "omit\_norms": true }, "raw": { "type": "string", "index": "_not\_analyzed_", "omit\_norms": true, "index\_options": "docs", "include\_in\_all": false, "ignore\_above": 256 } } }, My (limited) understanding is that the "not\_analyzed" should stop the field being split, so that my searching matches the full name, but it doesn't. I'm trying both kibana and curl to get results. Hope this makes sense. I really like the look of elasticsearch, but being able to search on extracted fields like this is pretty key to me using it. Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/62e3ebfc-aaa3-4af0-b93e-d4454146607b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62e3ebfc-aaa3-4af0-b93e-d4454146607b%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Chris\_H\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Chris\_H\_2](https://discuss.elastic.co/u/Chris_H_2)\ **Post date:** [January 9, 2014, 8:20am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/2 "2014-01-09T08:20:34Z")
Hi, a bit more information. I tried adding a custom analyzer based off a recommendation I saw online somewhere. This partly works in that it's not tokenising. But I can't do wildcard searches in Kibana on the fields, and they're now case sensitive ☹ curl localhost:9200/\_template/logstash-username -XPUT -d '{ "template": "logstash-\*", "settings" : { "analysis": { "analyzer": { "lc\_analyzer": { "type": "custom", "tokenizer": "keyword", "filters": ["lowercase"] } } } }, "mappings": { "_default_": { "properties" : { "User\_Name" : { "type" : "string", "analyzer" : "lc\_analyzer" } } } } }' Thanks On Wednesday, January 8, 2014 3:26:03 PM UTC, Chris H wrote: > Hi. I've deployed elasticsearch with logstash and kibana to take in > Windows logs from my OSSEC log server, following this guide: > [http://vichargrave.com/ossec-log-management-with-elasticsearch/](http://vichargrave.com/ossec-log-management-with-elasticsearch/) > I've tweaked the logstash config to extract some specific fields from the > logs, such as User\_Name. I'm having some issues searching on these fields > though. > > These searches work as expected: > > - User\_Name: \* > - User\_Name: john.smith > - User\_Name: john.\* > - NOT User\_Name: john.\* > > But I'm having problems with Computer accounts, which take the format > "w-dc-01$" - they're being split on the "-" and the "$" is ignored. So a > search for "w-dc-01" returns all the servers named "w-". Also I > can't do "NOT User\_Name: \*$" to exclude computer accounts. > > The mappings are created automatically by logstash, and GET > /logstash-2014.01.08/\_mapping shows: > > "User\_Name": { > > "type": "multi\_field", > "fields": { > "User\_Name": { > "type": "string", > "omit\_norms": true > }, > "raw": { > "type": "string", > "index": "_not\_analyzed_", > "omit\_norms": true, > "index\_options": "docs", > "include\_in\_all": false, > "ignore\_above": 256 > } > } > }, > > My (limited) understanding is that the "not\_analyzed" should stop the > field being split, so that my searching matches the full name, but it > doesn't. I'm trying both kibana and curl to get results. > > Hope this makes sense. I really like the look of elasticsearch, but being > able to search on extracted fields like this is pretty key to me using it. > > Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\ **Post date:** [January 9, 2014, 8:52am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/3 "2014-01-09T08:52:46Z")
Hi Chris, Could you try to escape “-“ in query for “not\_analyzed” field? > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. I hope this helps. Regards, * * * Jun Ohtani [johtani@gmail.com](mailto:johtani@gmail.com) blog : [http://blog.johtani.info](http://blog.johtani.info) twitter : [http://twitter.com/johtani](http://twitter.com/johtani) 2014/01/09 17:20、Chris H [chris.hembrow@gmail.com](mailto:chris.hembrow@gmail.com) のメール： > Hi, a bit more information. > > I tried adding a custom analyzer based off a recommendation I saw online somewhere. This partly works in that it's not tokenising. But I can't do wildcard searches in Kibana on the fields, and they're now case sensitive ☹ > > curl localhost:9200/\_template/logstash-username -XPUT -d '{ > "template": "logstash-\*", > "settings" : { > "analysis": { > "analyzer": { > "lc\_analyzer": { > "type": "custom", > "tokenizer": "keyword", > "filters": ["lowercase"] > } > } > } > }, > "mappings": { > "_default_": { > "properties" : { > "User\_Name" : { "type" : "string", "analyzer" : "lc\_analyzer" } > } > } > } > }' > > Thanks > > On Wednesday, January 8, 2014 3:26:03 PM UTC, Chris H wrote: > Hi. I've deployed elasticsearch with logstash and kibana to take in Windows logs from my OSSEC log server, following this guide: [http://vichargrave.com/ossec-log-management-with-elasticsearch/](http://vichargrave.com/ossec-log-management-with-elasticsearch/) > I've tweaked the logstash config to extract some specific fields from the logs, such as User\_Name. I'm having some issues searching on these fields though. > > These searches work as expected: > • User\_Name: \* > • User\_Name: john.smith > • User\_Name: john.\* > • NOT User\_Name: john.\* > But I'm having problems with Computer accounts, which take the format "w-dc-01$" - they're being split on the "-" and the "$" is ignored. So a search for "w-dc-01" returns all the servers named "w-". Also I can't do "NOT User\_Name: \*$" to exclude computer accounts. > > The mappings are created automatically by logstash, and GET /logstash-2014.01.08/\_mapping shows: > > "User\_Name": { > > "type": "multi\_field", > "fields": { > "User\_Name": { > "type": "string", > "omit\_norms": true > }, > "raw": { > "type": "string", > "index": "not\_analyzed", > "omit\_norms": true, > "index\_options": "docs", > "include\_in\_all": false, > "ignore\_above": 256 > } > } > }, > My (limited) understanding is that the "not\_analyzed" should stop the field being split, so that my searching matches the full name, but it doesn't. I'm trying both kibana and curl to get results. > > Hope this makes sense. I really like the look of elasticsearch, but being able to search on extracted fields like this is pretty key to me using it. > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Chris\_H\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Chris\_H\_2](https://discuss.elastic.co/u/Chris_H_2)\ **Post date:** [January 9, 2014, 1:27pm UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/4 "2014-01-09T13:27:04Z")
Hi, Jun. That doesn't seem to work. For a user with the username bob.smith-jones: - bob.smith-jones -\> matches - bob.smith-aaaa -\> matches - bob.smi\* -\> matches - bob.smith-j\* -\> no results - bob.smith-j\* -\> no results Also, a "$" isn't one of the special characters. Thanks. On Thursday, January 9, 2014 8:52:46 AM UTC, Jun Ohtani wrote: > Hi Chris, > > Could you try to escape “-“ in query for “not\_analyzed” field? > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_reserved_characters) > > I hope this helps. > Regards, > > * * * > > Jun Ohtani > [joh...@gmail.com](mailto:joh...@gmail.com) \ > blog : [http://blog.johtani.info](http://blog.johtani.info) > twitter : [http://twitter.com/johtani](http://twitter.com/johtani) > > 2014/01/09 17:20、Chris H \<[chris....@gmail.com](mailto:chris....@gmail.com) \\> のメール： > > > Hi, a bit more information. > > > > I tried adding a custom analyzer based off a recommendation I saw online > > somewhere. This partly works in that it's not tokenising. But I can't do > > wildcard searches in Kibana on the fields, and they're now case sensitive > > ☹ > > > > curl localhost:9200/\_template/logstash-username -XPUT -d '{ > > "template": "logstash-\*", > > "settings" : { > > "analysis": { > > "analyzer": { > > "lc\_analyzer": { > > "type": "custom", > > "tokenizer": "keyword", > > "filters": ["lowercase"] > > } > > } > > } > > }, > > "mappings": { > > "_default_": { > > "properties" : { > > "User\_Name" : { "type" : "string", "analyzer" : > > "lc\_analyzer" } > > } > > } > > } > > }' > > > > Thanks > > > > On Wednesday, January 8, 2014 3:26:03 PM UTC, Chris H wrote: > > Hi. I've deployed elasticsearch with logstash and kibana to take in > > Windows logs from my OSSEC log server, following this guide: > > [http://vichargrave.com/ossec-log-management-with-elasticsearch/](http://vichargrave.com/ossec-log-management-with-elasticsearch/) > > I've tweaked the logstash config to extract some specific fields from > > the logs, such as User\_Name. I'm having some issues searching on these > > fields though. > > > > These searches work as expected: > > • User\_Name: \* > > • User\_Name: john.smith > > • User\_Name: john.\* > > • NOT User\_Name: john.\* > > But I'm having problems with Computer accounts, which take the format > > "w-dc-01$" - they're being split on the "-" and the "$" is ignored. So a > > search for "w-dc-01" returns all the servers named "w-". Also I > > can't do "NOT User\_Name: \*$" to exclude computer accounts. > > > > The mappings are created automatically by logstash, and GET > > /logstash-2014.01.08/\_mapping shows: > > > > "User\_Name": { > > > > "type": "multi\_field", > > "fields": { > > "User\_Name": { > > "type": "string", > > "omit\_norms": true > > }, > > "raw": { > > "type": "string", > > "index": "not\_analyzed", > > "omit\_norms": true, > > "index\_options": "docs", > > "include\_in\_all": false, > > "ignore\_above": 256 > > } > > } > > }, > > My (limited) understanding is that the "not\_analyzed" should stop the > > field being split, so that my searching matches the full name, but it > > doesn't. I'm trying both kibana and curl to get results. > > > > Hope this makes sense. I really like the look of elasticsearch, but > > being able to search on extracted fields like this is pretty key to me > > using it. > > > > Thanks. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96e74e53-54f9-48ec-9e5c-8f1354b264be%40googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/22cf533e-eab8-468b-9b9a-55bbe12b3d62%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/22cf533e-eab8-468b-9b9a-55bbe12b3d62%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [January 9, 2014, 5:09pm UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/5 "2014-01-09T17:09:41Z")
Chris, I updated one of my tests to reproduce your issue. My text field is a multi-field where _text.na_ is the text field without any analysis at all. This Lucene query does not find anything at all: { "bool" : { "must" : { "query\_string" : { "query" : "_text.na:Immortal-Li_\*" } } } } But this one works fine: { "bool" : { "must" : { "prefix" : { "text.na" : { "_prefix_" : "_Immortal-Li_" } } } } } And returns the two documents that I expected: { "\_index" : "mortal" , "\_type" : "elf" , "\_id" : "1" , "\_version" : 1 , "\_score" : 1.0 , "\_source" : { "cn" : "Celeborn" , "text" : "Immortal-Lives forever" } } { "\_index" : "mortal" , "\_type" : "elf" , "\_id" : "2" , "\_version" : 1 , "\_score" : 1.0 , "\_source" : { "cn" : "Galadriel" , "text" : "Immortal-Lives forever" } } Note that in both cases, the query's case must match since the field value is not analyzed at all. I'm not sure if this is a true bug. In general, I find Lucene syntax somewhat useful for ad-hoc queries, and I find their so-called Simple Query Parser syntax to be completely unable to find anything when there is no \_all field, whether or not I specify a default field. (But that's another issue I'm going to ask about in the near future.) Brian On Thursday, January 9, 2014 8:27:04 AM UTC-5, Chris H wrote: > Hi, Jun. > > That doesn't seem to work. For a user with the username bob.smith-jones: > > - bob.smith-jones -\> matches > - bob.smith-aaaa -\> matches > - bob.smi\* -\> matches > - bob.smith-j\* -\> no results > - bob.smith-j\* -\> no results > > Also, a "$" isn't one of the special characters. > > Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6cb908eb-9ca7-4f05-815f-a868c45f9f66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6cb908eb-9ca7-4f05-815f-a868c45f9f66%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\ **Post date:** [January 10, 2014, 3:38am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/6 "2014-01-10T03:38:15Z")
Hi Chris, I recreate your issue to the following gist. > And I try to change query as follows: User\_Name.raw:bob.smith-jones -\> matches User\_Name.raw:bob.smi\* -\> matches User\_Name.raw:bob.smith-j\* -\> matches User\_Name.raw:bob.smith-j\* -\> matches I use User\_Name.raw field instead of User\_Name. Sorry, not necessary to escape… And I don’t know why do not work Brian example’s query\_string query… Does it make sense? Is this understanding mistaken? * * * Jun Ohtani [johtani@gmail.com](mailto:johtani@gmail.com) blog : [http://blog.johtani.info](http://blog.johtani.info) twitter : [http://twitter.com/johtani](http://twitter.com/johtani) 2014/01/10 2:09、InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com) のメール： > Chris, > > I updated one of my tests to reproduce your issue. My text field is a multi-field where text.na is the text field without any analysis at all. > > This Lucene query does not find anything at all: > > { > "bool" : { > "must" : { > "query\_string" : { > "query" : "text.na:Immortal-Li\*" > } > } > } > } > > But this one works fine: > > { > "bool" : { > "must" : { > "prefix" : { > "text.na" : { > "prefix" : "Immortal-Li" > } > } > } > } > } > > And returns the two documents that I expected: > > { "\_index" : "mortal" , "\_type" : "elf" , "\_id" : "1" , "\_version" : 1 , "\_score" : 1.0 , "\_source" : > { "cn" : "Celeborn" , "text" : "Immortal-Lives forever" } } > > { "\_index" : "mortal" , "\_type" : "elf" , "\_id" : "2" , "\_version" : 1 , "\_score" : 1.0 , "\_source" : > { "cn" : "Galadriel" , "text" : "Immortal-Lives forever" } } > > Note that in both cases, the query's case must match since the field value is not analyzed at all. > > I'm not sure if this is a true bug. In general, I find Lucene syntax somewhat useful for ad-hoc queries, and I find their so-called Simple Query Parser syntax to be completely unable to find anything when there is no \_all field, whether or not I specify a default field. (But that's another issue I'm going to ask about in the near future.) > > Brian > > On Thursday, January 9, 2014 8:27:04 AM UTC-5, Chris H wrote: > Hi, Jun. > > That doesn't seem to work. For a user with the username bob.smith-jones: > • bob.smith-jones -\> matches > • bob.smith-aaaa -\> matches > • bob.smi\* -\> matches > • bob.smith-j\* -\> no results > • bob.smith-j\* -\> no results > Also, a "$" isn't one of the special characters. > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6cb908eb-9ca7-4f05-815f-a868c45f9f66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6cb908eb-9ca7-4f05-815f-a868c45f9f66%40googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [January 10, 2014, 4:07am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/7 "2014-01-10T04:07:06Z")
If it helps, here are my index settings and mappings. Note that I chose the name text.na as the non-analyzed form, not text.raw. Perhaps I should follow convention? But for now, a rose by any other name is still not analyzed: { "settings" : { "index" : { "number\_of\_shards" : 1, "refresh\_interval" : "1s", "analysis" : { "char\_filter" : { }, "filter" : { "english\_snowball\_filter" : { "type" : "snowball", "language" : "English" } }, "analyzer" : { "english\_stemming\_analyzer" : { "type" : "custom", "tokenizer" : "standard", "filter" : [ "standard", "lowercase", "asciifolding", "english\_snowball\_filter" ] }, "english\_standard\_analyzer" : { "type" : "custom", "tokenizer" : "standard", "filter" : ["standard", "lowercase", "asciifolding"] } } } } }, "mappings" : { "_default_" : { "dynamic" : "strict" }, "ghost" : { "\_all" : { "enabled" : false }, "\_ttl" : { "enabled" : true, "default" : "1.9m" }, "properties" : { "cn" : { "type" : "string", "analyzer" : "english\_stemming\_analyzer" }, "text" : { "type" : "multi\_field", "fields" : { "text" : { "type" : "string", "analyzer" : "english\_stemming\_analyzer", "position\_offset\_gap" : 4 }, "std" : { "type" : "string", "analyzer" : "english\_standard\_analyzer", "position\_offset\_gap" : 4 }, "na" : { "type" : "string", "index" : "not\_analyzed" } } } } }, "_elf_" : { "\_all" : { "enabled" : false }, "\_ttl" : { "enabled" : true }, "properties" : { "cn" : { "type" : "string", "analyzer" : "english\_stemming\_analyzer" }, "_text_" : { "type" : "multi\_field", "fields" : { "text" : { "type" : "string", "analyzer" : "english\_stemming\_analyzer", "position\_offset\_gap" : 4 }, "std" : { "type" : "string", "analyzer" : "english\_standard\_analyzer", "position\_offset\_gap" : 4 }, "_na_" : { "type" : "string", "index" : "_not\_analyzed_" } } } } } } } Brian On Thursday, January 9, 2014 10:38:15 PM UTC-5, Jun Ohtani wrote: > Hi Chris, > > I recreate your issue to the following gist. > > [elasticsearch-ML : "Searching indexed fields without analysing" sample · GitHub](https://gist.github.com/johtani/8346404) > > And I try to change query as follows: > > User\_Name.raw:bob.smith-jones -\> matches > User\_Name.raw:bob.smi\* -\> matches > User\_Name.raw:bob.smith-j\* -\> matches > User\_Name.raw:bob.smith-j\* -\> matches > > I use User\_Name.raw field instead of User\_Name. > > Sorry, not necessary to escape… > > And I don’t know why do not work Brian example’s query\_string query… -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b0aece84-052c-4efc-8a25-1b42850fefe4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b0aece84-052c-4efc-8a25-1b42850fefe4%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\ **Post date:** [January 10, 2014, 4:39am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/8 "2014-01-10T04:39:52Z")
Hi Brian, Thanks! I understand that your query does not match anything at all. “query\_string” query is changed automatically query-terms to lower-case in some cases. i.e. wildcard, prefix, fuzzy… See : [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html) I change your query as follow : { "query": { "bool" : { "must" : { "query\_string" : { "query" : "text.na:Immortal-Li\*", "lowercase\_expanded\_terms" : false } } } } } Then returns the two documents. I've learned a great deal! Regards, * * * Jun Ohtani [johtani@gmail.com](mailto:johtani@gmail.com) blog : [http://blog.johtani.info](http://blog.johtani.info) twitter : [http://twitter.com/johtani](http://twitter.com/johtani) 2014/01/10 13:07、InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com) のメール： > If it helps, here are my index settings and mappings. Note that I chose the name text.na as the non-analyzed form, not text.raw. Perhaps I should follow convention? But for now, a rose by any other name is still not analyzed: > > { > "settings" : { > "index" : { > "number\_of\_shards" : 1, > "refresh\_interval" : "1s", > "analysis" : { > "char\_filter" : { }, > "filter" : { > "english\_snowball\_filter" : { > "type" : "snowball", > "language" : "English" > } > }, > "analyzer" : { > "english\_stemming\_analyzer" : { > "type" : "custom", > "tokenizer" : "standard", > "filter" : ["standard", "lowercase", "asciifolding", "english\_snowball\_filter"] > }, > "english\_standard\_analyzer" : { > "type" : "custom", > "tokenizer" : "standard", > "filter" : ["standard", "lowercase", "asciifolding"] > } > } > } > } > }, > "mappings" : { > "_default_" : { > "dynamic" : "strict" > }, > "ghost" : { > "\_all" : { > "enabled" : false > }, > "\_ttl" : { > "enabled" : true, > "default" : "1.9m" > }, > "properties" : { > "cn" : { > "type" : "string", > "analyzer" : "english\_stemming\_analyzer" > }, > "text" : { > "type" : "multi\_field", > "fields" : { > "text" : { > "type" : "string", > "analyzer" : "english\_stemming\_analyzer", > "position\_offset\_gap" : 4 > }, > "std" : { > "type" : "string", > "analyzer" : "english\_standard\_analyzer", > "position\_offset\_gap" : 4 > }, > "na" : { > "type" : "string", > "index" : "not\_analyzed" > } > } > } > } > }, > "elf" : { > "\_all" : { > "enabled" : false > }, > "\_ttl" : { > "enabled" : true > }, > "properties" : { > "cn" : { > "type" : "string", > "analyzer" : "english\_stemming\_analyzer" > }, > "text" : { > "type" : "multi\_field", > "fields" : { > "text" : { > "type" : "string", > "analyzer" : "english\_stemming\_analyzer", > "position\_offset\_gap" : 4 > }, > "std" : { > "type" : "string", > "analyzer" : "english\_standard\_analyzer", > "position\_offset\_gap" : 4 > }, > "na" : { > "type" : "string", > "index" : "not\_analyzed" > } > } > } > } > } > } > } > > Brian > > On Thursday, January 9, 2014 10:38:15 PM UTC-5, Jun Ohtani wrote: > Hi Chris, > > I recreate your issue to the following gist. > > [elasticsearch-ML : "Searching indexed fields without analysing" sample · GitHub](https://gist.github.com/johtani/8346404) > > And I try to change query as follows: > > User\_Name.raw:bob.smith-jones -\> matches > User\_Name.raw:bob.smi\* -\> matches > User\_Name.raw:bob.smith-j\* -\> matches > User\_Name.raw:bob.smith-j\* -\> matches > > I use User\_Name.raw field instead of User\_Name. > > Sorry, not necessary to escape… > > And I don’t know why do not work Brian example’s query\_string query… > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email [toelasticsearch+unsubscribe@googlegroups.com](mailto:toelasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b0aece84-052c-4efc-8a25-1b42850fefe4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b0aece84-052c-4efc-8a25-1b42850fefe4%40googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Chris\_H\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Chris\_H\_2](https://discuss.elastic.co/u/Chris_H_2)\ **Post date:** [January 13, 2014, 9:51am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/9 "2014-01-13T09:51:17Z")
Thanks, everybody. It does look like the issue is with the "lowercase\_expanded\_terms". I've also discovered that logstash by default creates both an analyzed and non-analyzed field, which helps a bit. However, I've worked around my specific issue (differentiating Windows User and Computer accounts) in logstash by extracting them into separate fields. Thanks On Wednesday, January 8, 2014 3:26:03 PM UTC, Chris H wrote: > Hi. I've deployed elasticsearch with logstash and kibana to take in > Windows logs from my OSSEC log server, following this guide: > [http://vichargrave.com/ossec-log-management-with-elasticsearch/](http://vichargrave.com/ossec-log-management-with-elasticsearch/) > I've tweaked the logstash config to extract some specific fields from the > logs, such as User\_Name. I'm having some issues searching on these fields > though. > > These searches work as expected: > > - User\_Name: \* > - User\_Name: john.smith > - User\_Name: john.\* > - NOT User\_Name: john.\* > > But I'm having problems with Computer accounts, which take the format > "w-dc-01$" - they're being split on the "-" and the "$" is ignored. So a > search for "w-dc-01" returns all the servers named "w-". Also I > can't do "NOT User\_Name: \*$" to exclude computer accounts. > > The mappings are created automatically by logstash, and GET > /logstash-2014.01.08/\_mapping shows: > > "User\_Name": { > > "type": "multi\_field", > "fields": { > "User\_Name": { > "type": "string", > "omit\_norms": true > }, > "raw": { > "type": "string", > "index": "_not\_analyzed_", > "omit\_norms": true, > "index\_options": "docs", > "include\_in\_all": false, > "ignore\_above": 256 > } > } > }, > > My (limited) understanding is that the "not\_analyzed" should stop the > field being split, so that my searching matches the full name, but it > doesn't. I'm trying both kibana and curl to get results. > > Hope this makes sense. I really like the look of elasticsearch, but being > able to search on extracted fields like this is pretty key to me using it. > > Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1746973d-75ba-4dbd-a026-f5bfce663899%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1746973d-75ba-4dbd-a026-f5bfce663899%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:57am UTC](https://discuss.elastic.co/t/searching-indexed-fields-without-analysing/15144/10 "2017-07-06T01:57:11Z")