Mutate not converting text to integer

Elastic Stack Logstash
1

I'm trying to grok a field with regexp and convert it to an integer, but it's not converting to an integer.

if [type] == "server_log" {
  grok {
             match => [ "message", ".*mediaDurationMs=(?<mediaDurationMs>\d+)" ]
       }
  mutate {
    convert => { "mediaDurationMs" => "integer" }
  }
}

In Kibana I can see that the 'mediaDurationMs field exists now, but it's 't', not #.. What is the correct way to make sure the mediaDurationMs field is set to integer?

2

Hi Mike,

It's not a correct approach to manage field types, look into mapping. If your index mapping already has mediaDurationMs -> string than logstash cannot overwrite it.

3

I've tried setting a mapping for my logstash indexes and it still isn't taking:

{
    "template" : "logstash-*",
    "settings" : {
        "number_of_shards" : 2,
        "number_of_replicas" : 1,
        "refresh_interval" : "30s",
        "index.mapper.dynamic":true
    },
    "mappings": {
        "server_log": { 
    		"properties": { 
        		"mediaDurationMs":    { "type": "long"  }
    		}
      }
    }
}

What is the correct pattern for setting the data type of our log fields?

4

I've tried setting a mapping for my logstash indexes and it still isn't taking:

Did you create a new index after you uploaded this template? What mappings does that index have?

5

Looking at this mapping:

GET /logstash-2018.01.29/_mapping

I can see that my mapping template is in place, because in the mapping its show as:

                "mediaDurationMs": {
                    "type": "long"
                },

However, in Kibana, it still shows as a string...

53
Screen Shot 2018-01-28 at 20.35.53.png828×172 16.4 KB

6

Have you refreshed the fields in Kibana?

7

I did, and it looks like there is a conflict on that field.. I guess I need to grok to a new field and make sure the mapping is in place on the index and template before the field is created.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.