Extracting a value(Integer) from the logs and plotting it in kibana VS time

Elastic Stack Logstash
1

Hi,

I am tying to parse logs using grok filters.

if [type]=="xyz"{
multiline {
pattern => "^%{SYSLOG5424SD} "
negate => true
what => previous
}
grok{
match => {
"message" => ["%{SYSLOG5424SD:Timestamp} [%{LOGLEVEL:Severity}] [helium[.]grid[.]env[.]%{WORD:DataLakeLoader}[$]] [com.ca.ri.hercules.dataloader.push.PushSchedulerTimerTask] %{NUMBER:Event_Loaded:int}",
"%{SYSLOG5424SD:Timestamp} [%{LOGLEVEL:Severity}] %{GREEDYDATA:Message}"
]
}
}
}

As u can see in my grok filter i am parsing an int value using {NUMBER:Event_Loaded:int}, but when i try to plot this Event_loaded on a line graph VS time, it is not coming up under the number field on y axis.

Need help

2

That's most likely because the Event_Loaded field has already been mapped as a string in ES, so your change to have grok emit an integer field doesn't make a difference (an index's field mappings can't be changed after the fact). You either have to reindex or wait until tomorrow when a new index is created.

3

Thanks for the reply!!

"You either have to reindex or wait until tomorrow when a new index is created"
logstash is running for more than 10 days, with the same configuration.

Also, is their any difference between {NUMBER:Event_Loaded:int} and {NUMBER:Event_Loaded:float} with respect to kibana UI number field. What i mean is will it not show as number if I use float instead of int??

4

logstash is running for more than 10 days, with the same configuration.

Hmm, okay. What does the mapping look like in ES (use the get mapping API)? Can you show an example message? And have you reloaded the field list in Kibana?

Also, is their any difference between {NUMBER:Event_Loaded:int} and {NUMBER:Event_Loaded:float} with respect to kibana UI number field. What i mean is will it not show as number if I use float instead of int??

I don't think it matters.