Logstash how to send string as not_analyzed into elasticsearch

Hi ,

I am usng logstash 1.5.4 , while indexing the data through logstash, all the string data type indexing as analyzed , but my requirement is it should be not_analyzed ..
So I had changed the file elastic-template.json

"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "not_analyzed", "omit_norms" : true

but still the string type is indexing as anayzed , why ?
how to overcome this ...

Thanks

Assuming you made the change in the right place, it'll only take effect for newly created indexes. Are you taking that into account?

yes, after that I had changed the template file and again trying to index the doc , still showing analyzed in kibana

my elastic-template.json file is

    {
      "template" : "logstash-*",
      "settings" : {
        "index.refresh_interval" : "5s"
      },
      "mappings" : {
        "_default_" : {
           "_all" : {"enabled" : true, "omit_norms" : true},
           "dynamic_templates" : [ {
             "message_field" : {
               "match" : "message",
               "match_mapping_type" : "string",
               "mapping" : {
                 "type" : "string", "index" : "not_analyzed", "omit_norms" : true
               }
             }
           }, {
             "string_fields" : {
               "match" : "*",
               "match_mapping_type" : "string",
               "mapping" : {
                 "type" : "string", "index" : "not_analyzed", "omit_norms" : true,
                   "fields" : {
                     "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : true}
                   }
               }
             }
           } ],
           "properties" : {
             "@version": { "type": "string", "index": "not_analyzed" },
             "geoip"  : {
               "type" : "object",
                 "dynamic": true,
                 "properties" : {
                   "location" : { "type" : "geo_point" }
                 }
             }
           }
        }
      }

}

and Logstash output is

elasticsearch
		{
			action => "index"
			host => "localhost"
			cluster => "navneet"		
			template =>  "D:/es_tools/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.7-java/lib/logstash/outputs/elasticsearch/elasticsearch-template.json"
			index => "ems"
		}
1 Like

Your template is for indexes whose name matches the pattern logstash-* but you're pushing data to an index named ems so it won't use your template.

I have changed the template with "*" , but still the same
all the string fields are occurring as analyzed only , am I doing something wrong somewhere ?

Any update on this?
I have the same issue.

No one?

@zappe: To maximize your chances of getting help, start a new thread and post a complete recipe for reproducing your problem. You might be experiencing the same symptom as the original poster in this thread but the root cause could be something completely different.

@zappe

It was not working when referring the template from logstash so I had created the template in elastic only ..
if your index name starts with elastic- ... you can create template (dynamic template ) for elastic-* (this works the same as referring the template from logstash ).

Hi Navneet. I just wanted to know that you created template in elasticsearch using curl command etc, then did you use manage_template filed in logstash configuration for elasticsearch output?

As you say, if creating a dynamic template in Elasticsearch works, then why not referring in Logstash is not working?