Impossible format date

[Viti]

Hello, I need to format a date.

The log line is this:
2019-01-29 12:59:51,8345358246519 ERROR - RGKAS055551 * INFO * 00003 * method * initSystem * =====> fail system down

I have this:
2019-01-29 12:59:51,8345358246519

and I want this:
2019-01-29 12:59:51,834 (This information will send to elasticsearch) I need to save as a date field.

How can I make it?

[Badger]

@Viti

2019-01-29 12:59:51,8345358246519

You can extract the first 23 characters of a string using

mutate { gsub => [ "ts", "^(.{23}).*", "\1" ] }

[Viti]

Yes, it works, but it only got a string of 23 characters.

Now I have this:
2019-01-29 12:59:51,834

I want to send to Elasticsearch a DATE field or some kind of field for I can do filter with Dates

What do I need?

[Badger]

@Viti

2019-01-29 12:59:51,834

Use a date filter. That format fits the ISO8601 pattern.

date { match => [ "ts", "ISO8601" ] }

That will modify the @timestamp field.

[Viti]

Elasticsearch always get that field like a String. I don't understand because It has this performances

[Badger]

Elasticsearch has @timestamp mapped as a string? You might want to try creating a new index and see if it works in that.

[Viti]

now I have the date:
2019-01-29T12:59:51.834 (12:59)

When it saved to elasticsearch
2019-01-29T11:59:51.834 (11:59)

Why is ocurring this?

I think that I need the time zone

How do it make?

[Christian_Dahlqvist]

All timestamps in Elasticsearch are in UTC timezone, so depending on which time zone you are in this may be correct.

[Viti]

I send the current hour and elasticsearch saves one hour more

How do I change it in elastisearch?.

[Christian_Dahlqvist]

Which time zone are you in? Are the timestamps in the logs in local time?

[Viti]

I work with time zone of Europa Madrid.

[Christian_Dahlqvist]

Then it looks correct as 12:59 CET corresponds to 11:59 UTC. Elasticsearch requires all timestamps to be in UTC, and Kibana will translate to the local timezone where applicable, although it will not change the source of the document.

[Viti]

@Christian_Dahlqvist

9 UTC

The register always saves with a hour more.
For example I have a register with date
2019-01-29T23:00:51.834

Elasticsearhc saves:
2019-01-30T00:00:51.834 (I have and hour more and it is next day)

I want to save 2019-01-29T23:00:51.834

[Viti]

Finally I change the parameter in Kibana -> Management ->advanced settings

Thank you very much