Logstash 2.4 grok mutate real with comma delimiter

Elastic Stack Logstash
1

I need to parse the following line:
2016-12-02 11:54:03,107 Tests run: 2344, Failures: 0, Errors: 1, Skipped: 14, Time elapsed: 1,480.535 sec <<< FAILURE! - in com.werum.pasx.servicetests.ebrequipment.TestSuiteEbrEquipmentBf

Here my grok pattern:

grok {
	  break_on_match => true
	  match => [ "message", "(?m)%{TIMESTAMP_ISO8601}\s*%{TIMESTAMP_ISO8601:logdate}\s.*INFO\s.*?Time elapsed:\s(?<duration>[0-9.,+-]+)\s*sec\s.*com\.werum\.pasx\.%{DATA:probe}(\s|\n)%{GREEDYDATA:msg}"]
	  match => [ "message", "(?m)%{TIMESTAMP_ISO8601}\s*%{TIMESTAMP_ISO8601:logdate}\s.*INFO\s.*?Time elapsed:\s(?<duration>[0-9.,+-]+)\s*sec\s.*com\.werum\.pasx\.%{GREEDYDATA:probe}"]
	  match => [ "message", "(?m)%{TIMESTAMP_ISO8601}\s*%{TIMESTAMP_ISO8601:logdate}\s%{GREEDYDATA:msg}"]
	  match => [ "message", "(?m)%{TIMESTAMP_ISO8601:logdate}\s%{GREEDYDATA:msg}"]
	}

	mutate {
	  convert => [ "duration", "float" ]
	}

It works, if the numbers follwing "Time collapsed:" don't contain a comma as delimiter. In my example duration contains 1 after mutating.
Any idea, how I can get the correct number ?

2

Use gsub in mutate filter before mutate convert to get rid of the comma? https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html

1 Like
3

Thank, works, but only if I place gsub and convert in different mutates

mutate {
gsub => [
  # remove all comma 
  "duration", ",", ""
]
add_field => {
  "org_duration" => "copy from %{duration}"
}
}
mutate {
	convert => [ "duration", "float" ]
}

Any idea why?
It's not essential, but I'd like to know the reason :slight_smile:

Next inconsequential detail:
Kibana shows the number with comma again :confused:

4

Known issue Each mutate must be in its own code block · Issue #27 · logstash-plugins/logstash-filter-mutate · GitHub

You can change the display format of number fields in Kibana. Go to Settings > Indices > <your_index> and change the field format to your need.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.