Logstash how to send string as not_analyzed into elasticsearch

Elastic Stack Logstash
#1

Hi ,

I am usng logstash 1.5.4 , while indexing the data through logstash, all the string data type indexing as analyzed , but my requirement is it should be not_analyzed ..
So I had changed the file elastic-template.json

"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "not_analyzed", "omit_norms" : true

but still the string type is indexing as anayzed , why ?
how to overcome this ...

Thanks

#2

Assuming you made the change in the right place, it'll only take effect for newly created indexes. Are you taking that into account?

#3

yes, after that I had changed the template file and again trying to index the doc , still showing analyzed in kibana

my elastic-template.json file is

    {
      "template" : "logstash-*",
      "settings" : {
        "index.refresh_interval" : "5s"
      },
      "mappings" : {
        "_default_" : {
           "_all" : {"enabled" : true, "omit_norms" : true},
           "dynamic_templates" : [ {
             "message_field" : {
               "match" : "message",
               "match_mapping_type" : "string",
               "mapping" : {
                 "type" : "string", "index" : "not_analyzed", "omit_norms" : true
               }
             }
           }, {
             "string_fields" : {
               "match" : "*",
               "match_mapping_type" : "string",
               "mapping" : {
                 "type" : "string", "index" : "not_analyzed", "omit_norms" : true,
                   "fields" : {
                     "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : true}
                   }
               }
             }
           } ],
           "properties" : {
             "@version": { "type": "string", "index": "not_analyzed" },
             "geoip"  : {
               "type" : "object",
                 "dynamic": true,
                 "properties" : {
                   "location" : { "type" : "geo_point" }
                 }
             }
           }
        }
      }

}

and Logstash output is

elasticsearch
		{
			action => "index"
			host => "localhost"
			cluster => "navneet"		
			template =>  "D:/es_tools/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.7-java/lib/logstash/outputs/elasticsearch/elasticsearch-template.json"
			index => "ems"
		}
1 Like
#4

Your template is for indexes whose name matches the pattern logstash-* but you're pushing data to an index named ems so it won't use your template.

How to change fields to not_analyzed in kibana
#5

I have changed the template with "*" , but still the same
all the string fields are occurring as analyzed only , am I doing something wrong somewhere ?

#6

Any update on this?
I have the same issue.

#7

No one?

#8

@zappe: To maximize your chances of getting help, start a new thread and post a complete recipe for reproducing your problem. You might be experiencing the same symptom as the original poster in this thread but the root cause could be something completely different.

#9

@zappe

It was not working when referring the template from logstash so I had created the template in elastic only ..
if your index name starts with elastic- ... you can create template (dynamic template ) for elastic-* (this works the same as referring the template from logstash ).

#10

Hi Navneet. I just wanted to know that you created template in elasticsearch using curl command etc, then did you use manage_template filed in logstash configuration for elasticsearch output?

#11

As you say, if creating a dynamic template in Elasticsearch works, then why not referring in Logstash is not working?

#12