Logstash adding date at the beginning of the each record

[sekharvijju (Vijay)]

Hi Logstash gurus,

I m trying to setup a basic logstash client to push server logs into S3 bucket using Input plugin & s3 output plugin.

input {
file {
path => "/logs/sys*"
exclude => "*.tar"
start_position => "beginning"
sincedb_path => "/var/logstash/.sincedb9"
sincedb_clean_after => "4"
}
}
output {
s3{
access_key_id => "XXXXXXXXXXXXXXXXXXXX"
secret_access_key => "XXXXXXXXXXXX"
region => "xx-xxxx-1"
bucket => "xyz
size_file => 10480000
time_file => 5 #5 minutes
codec => "line"
canned_acl => "private"
prefix => "logs"
}
}

At the beginning of each log event (row/line), it's adding the timestamp in UTC and then the actual event itself from the logfile as below:

2019-10-16T21:26:14.530Z XXXX XXXX XXXXXXXXXXXX XXXXXXXXXXX "GET XXXXXXX HTTP/1.1" 200 2057

Went through logstash documentation for input plugin and S3 output plugin and did not find anything talking about the timestamp being added automatically. time_file was added to rollup the events at every 5 minutes interval.

How do I get rid of that default timestamp? Any ideas or help is appreciated.

Regards,

Vijay

[Badger]

The default message format for the line codec is to add the timestamp and hostname at the beginning of the line. Specify the format option if you do not want that.