Remove logstash default timestamp and host info

Elastic Stack Logstash
1

Hi there:

My env info

Logstash 2.4.1
Kafka 2.10-0.8.2.1

My logstash config is

input {
	kafka {
	group_id => "salo"
	zk_connect=>"192.168.5.126:2181,192.168.5.21:2181,192.168.5.225:2181"
	topic_id=>"info_flow"
	}
}
filter {
	mutate {
	remove_field => ["%{timestamp}","%{host}"]
	}
}
output {
	s3{
 		bucket => "....."                    
 		size_file => 5242880
 		time_file => 4
 		prefix => "logs/info_flow/"
	}
}

My original kafka message is like

1528707594000|192.168.5.1|{"appkey":"Un7ey3h","d":{"et":[.....

But after logstash process , this turns to be

2018-06-11T08:37:38.728Z %{host} 1528707594000|192.168.5.1|{"appkey":"Un7ey3h","d":{"et":[.....

We don't want

  • 2018-06-11T08:37:38.728Z
  • %{host}

at all. Could you help to figure out how to delete them ? Thanks

2

I think these are being added by the output codec. Can you try adding this to your s3 output:

codec => line { format => "%{message}" }
1 Like
4

Yes, exactly! Thank you very much.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.