Input file has PST timestamp like "2020 Feb 12 13:54:07:447 GMT -0800" in a message event like below:
2020 Feb 12 13:54:07:447 GMT -0800 myservice Error [mybusinessworks] BWENGINE-100001 process initialization failed for CommonProcesses/ExceptionHandling/ExcepOverJMSProxy.process
I am trying to convert the Timestamp from PST to EST using the below snippet in Filebeat.
------
processors:
- dissect:
tokenizer: "%{log_time} %{+log_time} %{+log_time} %{+log_time}:%{+log_time}:%{+log_time}:%{+log_time} %{} %{} %{service_name} %{log_level} [%{process_name->}] %{msg}"
field: "message"
target_prefix: ""
- drop_event:
when:
not:
equals:
log_level: Error
- timestamp:
field: log_time
layouts:
- '2020 Feb 12 13:54:07:447'
test:
- '2020 Feb 12 13:54:07:447'
- drop_fields:
fields: ["log_time","message"]
------
Error details:
********
c:\Program Files\filebeat>"C:\Program Files\Filebeat\\filebeat.exe" -c "C:\Program Files\Filebeat\\filebeat.yml" -path.home "C:\Program Files\Filebeat" -path.data "C:\\ProgramData\\filebeat" -path.logs "C:\\ProgramData\\filebeat\logs"
Exiting: Failed to start crawler: starting input failed: Error while initializing input: failed parsing time field log_time='2020 Feb 12 13:54:07:447': failed using layout [2020 Feb 12 13:54:07:447] cannot parse [ Feb 12 13:54:07:447] as [0 Feb ]
failed to parse test timestamp
github.com/elastic/beats/v7/libbeat/processors/timestamp.newFromConfig
/go/src/github.com/elastic/beats/libbeat/processors/timestamp/timestamp.go:79
********
I have removed the GMT part from the time field thinking I can do the conversion easily. but it didn't work. Wondering if I should somehow shuttle the time field to match with one of the layouts like below to convert the time stamp to EST:
layouts:
- '2006-01-02T15:04:05Z'
- '2006-01-02T15:04:05.999Z'
- '2006-01-02T15:04:05.999-07:00'
The default value of the Timezone for the Timestamp processor is UTC. Which is what I want the timestamp value to be converted into.
I am using Filebeat 7.11