How to change logstash default timezone

[Manoj_Hettiarachchi]

Hello,

In my system, I use filebeat and logstash. The data written to the log files are sent to logstash via filebeat. Some fields in the logs are filtered and saved in csv format using logstash.

Problem:

When saving data in CSV files I need to save data belongs to each day in sperate files. So my "output" is as follows.

output {
  csv {
     path => "/var/csv_reports/%{+YYYY}-%{+MM}-%{+dd}/transaction-report.csv"
     fields => ["timestamp","tid","api","user","application","app_id", "body"]
  }
}

The file is shifting process happens in UTC time. So the generated files contain data belongs to 2 days.
How can I change the default timezone of logstash to my timezone ( +5:30).

Your help is much appreciated.

Thanks,

[asp]

Hi,

don't know how to change it globally, but you can define the time zone used in your logs.
To do so, you need to use the date filter plugin:

date 
{
	match => ['LogTime', 'dd.MM.YYYY HH:mm:ss']
	timezone => "Europe/Berlin"
	
	# remove fields no longer needed
	remove_field => ['LogTime']
}

In the example above my date field is stored as "logTime".
Since I am not specifying a target, logtime is automatically stored in @timestamp.

So log time is Berlin time. Logstash converts it to UTC.
Kibana is converting UTC to browser's timestamp on loading the visualization. Internally elastic is storing the timestamp in utc.

Regards, Andreas

[Manoj_Hettiarachchi]

Hello @asp

I have added the "date" field to my filter {} area. But no success.
Still, the directory name is created according to UTC time zone.

Thanks,

[Manoj_Hettiarachchi]

Hello again

I have found a solution here: https://stackoverflow.com/questions/29302444/convert-timestamp-timezone-in-logstash-for-output-index-name

so my new logstash config is as fallows

filter{
..........
ruby {
        code => "event.set('index_day', event.get('[@timestamp]').time.localtime.strftime('%Y%m%d'))"
    }
}

output {
  csv {
     path => "/var/csv_reports/%{index_day}/transaction-report.csv"
     fields => ["timestamp","tid","api","publisher","user","consumerKey","application","app_id"]
  }
}

Now the directory name can is created with the timestamp in my log lines.
This solved my issue.