Kibana is not recognizing integer


(Nil) #1

Tried inboth Kibana 4.4.1/2 version. It is not recognizing integer as int. Also tried with float.
However logstash is processing as integer. Have tried with mutating the same to a different field but same result.

input {
file {
path => "/u01/MyLagInfo/.txt."
type => "MyLagInfo"
#sincedb_path => "/dev/null"
}
}

filter {
if [type] =~ "MyLagInfo" {

grok {
    match => { "path" => "%{GREEDYDATA}/%{GREEDYDATA:SrvName}.txt" }
    #add_field => { "RepSrvName" => "%{SrvName}" }
}

grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
    #match => { "message" =>  "%{GREEDYDATA:SchemaName}\|%{GREEDYDATA:DestinationTS}\|%{GREEDYDATA:SourceTS}\|%{INT:ReplicationLag}" }
    match => { "message" =>  "%{GREEDYDATA:SchemaName}\|%{GREEDYDATA:DestinationTS}\|%{GREEDYDATA:SourceTS}\|(?<RepLag>%{NUMBER})" }
}
date {
    match => [ "SourceTS", "yyyy-MM-dd HH:mm:ss.SSS", "yyyy-MM-dd HH:mm:ss:SSS" ]
}
date {
    match => [ "DestinationTS", "yyyy-MM-dd HH:mm:ss.SSS", "yyyy-MM-dd HH:mm:ss:SSS" ]
    target => "@timestamp"
    #locale => "en"
}

mutate {
  convert => { "RepLag" => "integer" }
   #add_field =>
   #     {
   #        "MyLagtime" => "%{ReplicationLag}"
   #     }
   #remove_field => [ "ReplicationLag", "DestinationTS" ]
}

}
}

output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["localhost:9200"]
#index => "MyLag-%{+YYYY.MM.dd}"
#template => "/u01/logstash/template/replatency_index_template.json"
#template_overwrite => true
}
}


(Christian Dahlqvist) #2

What does the output look like? Do you get a match for the field ReplicationLag (which you do not seem to cast as an integer) or RepLag in the records?


(Nil) #3

Yes! It matches the data! I need to cast as integer for aggregation.

{
"message" => "GPST029|2016-03-16 15:10:01.176|2016-03-16 15:09:57.736|3",
"@version" => "1",
"@timestamp" => "2016-03-16T19:10:01.176Z",
"path" => "/u01/RepLatencyInfo/IDAS_PRD.txt.031616",
"host" => "itsrhv22111.it.statestr.com",
"type" => "RepLatencyInfo",
"SrvName" => "MYSRV",
"SchemaName" => "MYT029",
"DestinationTS" => "2016-03-16 15:10:01.176",
"SourceTS" => "2016-03-16 15:09:57.736",
"ReplicationLag" => 3
}


(system) #4