How to keep original date data from json as source to ES

[Sirius (Jorge Alfredo Sánchez Lamas)]

Hello,

I'm shipping json data logs with filebeat to ELK 7.6 version. For some reason date data are substracted by hours when is ingested in ES,, how can I keep original source dates in ES without be modified ?

FIlebeat configuration is:

filebeat.inputs:
json:
keys_under_root: true
add_error_key: true
overwrite_key: true

processors:

• decode_json_fields:
  fields: ["message"]
  max_depth: 2
  target: ""

Filebeat output is sending to logstash. Logstash has not any special configuration just send the data to ES.

[Christian_Dahlqvist]

All time stamps in Elasticsearch must be in UTC to work properly with e.g. Kibana. Indexing them in another timezone will cause problems.

[Sirius (Jorge Alfredo Sánchez Lamas)]

Thanks Christian, even using UTC timezone same problem maybe something else I'm doing wrong. However, using pipeline as below resolved this, overall setting in formats "X" instead of "Z" made keep original dates without changes ...

PUT _ingest/pipeline/mypipeline
{
"description" : "pipeline test",
"processors" : [
{
"date" : {
"field": "date",
"target_field": "date",
"formats": ["yyyy-MM-dd'T'HH:mm:ss.SSSX"],
"timezone": "Mexico/General"
}
}
]
}