Message.raw in logstash 2


(Jyotir Test) #1

Hi,

We were using logstash 1.5 and recently upgraded to logstash 2. We are unable to get message.raw in elasticsearch.

We followed mentioned link to make it work on logstash 1.5

But it is not working in logstash 2.0

Jyotir


(Magnus Bäck) #2

The change described in that issue will work fine in Logstash 2.0 too. What did you do to modify the index template?


(Jyotir Test) #3

PFA the code from logstash 2, elasticsearch-template.json file

{
  "template" : "logstash-*",
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
       "_all" : {"enabled" : true, "omit_norms" : true},
       "dynamic_templates" : [ {
         "message_field" : {
           "match" : "message",
           "match_mapping_type" : "string",
           "mapping" : {
             "type" : "string", "index" : "analyzed", "omit_norms" : true,
               "fields" : {
                 "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
             }
           }
         }
       }, {
         "string_fields" : {
           "match" : "*",
           "match_mapping_type" : "string",
           "mapping" : {
             "type" : "string", "index" : "analyzed", "omit_norms" : true,
               "fields" : {
                 "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
               }
           }
         }
       } ],
       "properties" : {
         "@version": { "type": "string", "index": "not_analyzed" },
         "geoip"  : {
           "type" : "object",
             "dynamic": true,
             "properties" : {
               "location" : { "type" : "geo_point" }
             }
         }
       }
    }
  }
}

(Magnus Bäck) #4

Okay, but what, exactly, did you do with this file? And whatever you did, have you verified that the index template indeed was updated by your actions?


(Jyotir Test) #5

added the mentioned line as been prescribed in github

"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}

I checked in again, and I can see message.raw in the elasticsearch. Thanks for response.

Jyotir


(Magnus Bäck) #6

Where did you add that line? To which file? Did you do anything with that file after modifying it?


(system) #7