Impossible format date

Hello, I need to format a date.

The log line is this:
2019-01-29 12:59:51,8345358246519 ERROR - RGKAS055551 * INFO * 00003 * method * initSystem * =====> fail system down

I have this:
2019-01-29 12:59:51,8345358246519

and I want this:
2019-01-29 12:59:51,834 (This information will send to elasticsearch) I need to save as a date field.

How can I make it?

You can extract the first 23 characters of a string using

mutate { gsub => [ "ts", "^(.{23}).*", "\1" ] }

Yes, it works, but it only got a string of 23 characters.

Now I have this:
2019-01-29 12:59:51,834

I want to send to Elasticsearch a DATE field or some kind of field for I can do filter with Dates

What do I need?

Use a date filter. That format fits the ISO8601 pattern.

date { match => [ "ts", "ISO8601" ] }

That will modify the @timestamp field.

Elasticsearch always get that field like a String. I don't understand because It has this performances

Elasticsearch has @timestamp mapped as a string? You might want to try creating a new index and see if it works in that.

now I have the date:
2019-01-29T12:59:51.834 (12:59)

When it saved to elasticsearch
2019-01-29T11:59:51.834 (11:59)

Why is ocurring this?

I think that I need the time zone

How do it make?

All timestamps in Elasticsearch are in UTC timezone, so depending on which time zone you are in this may be correct.

I send the current hour and elasticsearch saves one hour more

How do I change it in elastisearch?.

Which time zone are you in? Are the timestamps in the logs in local time?

I work with time zone of Europa Madrid.

Then it looks correct as 12:59 CET corresponds to 11:59 UTC. Elasticsearch requires all timestamps to be in UTC, and Kibana will translate to the local timezone where applicable, although it will not change the source of the document.

The register always saves with a hour more.
For example I have a register with date

Elasticsearhc saves:
2019-01-30T00:00:51.834 (I have and hour more and it is next day)

I want to save 2019-01-29T23:00:51.834

Finally I change the parameter in Kibana -> Management ->advanced settings

Thank you very much

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.