Problem with duplicate data in field

Hi all

I have an issue with the data that is passed into elasticsearch from logstash
below is my logstash config.

if [type] == "coreservices-fs" or "coreservices-net" {
grok {
match => { "message" => [
"^%{TIMESTAMP_ISO8601:Time} [%{DATA:Thread}] %{GREEDYDATA:Logger} : %{LOGLEVEL:Severity} - %{GREEDYDATA:KEYMessage}",
"Process with pid=%{NOTSPACE:PID} %{GREEDYDATA:Info}\nService stopped at %{GREEDYDATA:StopTime} by",
"Start time %{GREEDYDATA:StartTime} by" ] } }
grok {
match => { "source" => "/appdir/%{NOTSPACE:Env_Name}/logs/%{WORD:Server}.%{WORD}.%{WORD:NPID}" } } }

the data being seen on kibana are as follows:

as you can see, for several fields, the data is repeated multiple times.

I cant seem to figure out why.

Any help would be greatly appreciated.

Thank you

Please show the full configuration. All files in /etc/logstash/conf.d.

if [type] == "coreservices-fs" or "coreservices-mxnet" {

This conditional expression doesn't mean what you think it means. See Logstash sending same message to multiple indexes in ES.

Perfect.

Thanks

This is great.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.