Logstash configurtaion for convert to Number

Elastic Stack Logstash
1

Hi , We are trying a configuration to convert the message field (which has number coming as Text) to integer as we have to create some line graphs based on the number .

We tried below configuration but its converting only first digit.
ex: if message field has "1234"after conversion its showing only 1 .
Kindly suggest any ideas.

input {
  beats {
    port => 5046
    ecs_compatibility => "v8"
    include_codec_tag => false
    }
}
filter {
        grok {
        match => { "message" => "%{NUMBER:count}" }
        }
        mutate {
        convert => { "count" => "integer" }
        }
}
2

I can say that the problem is not from the filter as I running from my local machine, it look like it run as you expected:

{
  "@version": "1",
  "@timestamp": "2024-07-16T07:58:46.730900926Z",
  "message": "12345678912348",
  "count": 12345678912348,
  "event": {
    "original": "12345678912348"
  }
}

You should check for the input message or the event.original.

3

Hi ,

We are getting the output like this :

Here you can see Table view - Count is truncated or converted only first digit and and event.original is in text form.
Json view - Unicode is coming for event.original and count is coming as single digit .

image

"event": {
      "original": "\u00006\u00006\u00007\u00000\u00008\u0000\r\u0000"
    },
4

Hi,

You can try using the gsub function in the mutate filter to remove these non-printable characters before applying the grok pattern. Here's how you can modify your configuration:

filter {
    mutate {
        gsub => ["message", "[\x00-\x1F]", ""]
    }

Regards

5

Should try use other regex form, like BASE16NUM or BASE16FLOAT rather then NUMBER.

        grok {
        match => { "message" => "%{NUMBER:count}" }
        }

Here is the link for more infor about them: logstash-patterns-core/patterns/ecs-v1/grok-patterns at main · logstash-plugins/logstash-patterns-core (github.com)

6

Hi ,

Its getting converted after trying below code:

mutate {
        gsub => ["message", "[\x00-\x1F]", ""]
        }
        grok {
        match => { "message" => "%{BASE16NUM:count}" }
        }
        mutate {
        convert => { "count" => "integer" }
        }

Thanks for your help!

1 Like