Remove logstash default timestamp and host info

[simple365 (Tom Luo)]

Hi there:

My env info

Logstash 2.4.1
Kafka 2.10-0.8.2.1

My logstash config is

input {
	kafka {
	group_id => "salo"
	zk_connect=>"192.168.5.126:2181,192.168.5.21:2181,192.168.5.225:2181"
	topic_id=>"info_flow"
	}
}
filter {
	mutate {
	remove_field => ["%{timestamp}","%{host}"]
	}
}
output {
	s3{
 		bucket => "....."                    
 		size_file => 5242880
 		time_file => 4
 		prefix => "logs/info_flow/"
	}
}

My original kafka message is like

1528707594000|192.168.5.1|{"appkey":"Un7ey3h","d":{"et":[.....

But after logstash process , this turns to be

2018-06-11T08:37:38.728Z %{host} 1528707594000|192.168.5.1|{"appkey":"Un7ey3h","d":{"et":[.....

We don't want

• 2018-06-11T08:37:38.728Z
• %{host}

at all. Could you help to figure out how to delete them ? Thanks

[Badger]

@simple365

Could you help to figure out how to delete them ?

I think these are being added by the output codec. Can you try adding this to your s3 output:

codec => line { format => "%{message}" }

[simple365 (Tom Luo)]

Yes, exactly! Thank you very much.