Date, change day and month position

Elastic Stack Logstash
1

I have a field called "timeanddate" and I use it as timestamp.

date {
       match => ["timeanddate", "HH:mm:ss MM/dd/yyyy"]
       target => "@timestamp"
 }

But I want to change the order of the date and put the day first and then the month, how can I achieve this?

dd/MM/yyyy
2

If you want to change the format in kibana then kibana lets you do that. If you want to store a string in a different format in elasticsearch then use a ruby filter and strftime. If you want to change the format in which elasticsearch stores dates then you cannot do so.

3

The format string in Logstash's date filter determines how the input data is interpreted. The filter converts the input into a format that Elasticsearch understands natively.

So, if your input looks like 12:34:56 20/02/2020 (February 20th, 2020), the format string should be HH:mm:ss dd/MM/yyy.

4

Thanks Magnus but i forgot to tell that the date in the logs is MM/dd/yyyy, so if I change it trows me a ["_dateparsefailure"]

this is the format of the date in the logs

11:28:11 03/02/2020
5

Apologies for the typo in my previous message. Please try again with this format string in the mapping: HH:mm:ss dd/MM/yyyy. Please not that this is the mapping used for the Elasticsearch index. With the correct mapping in place there, the string from the original log message can be interpreted correctly in Elasticsearch, and no further transformation is needed in e.g. Logstash.

I'd also add some other commonly used formats into the format string, which will help with the queries generated by Kibana. Date fields can only be queried in one of the formats defined in the format string. So, for example the full format string might look like:

HH:mm:ss dd/MM/yyyy||strict_date_optional_time||epoch_millis

6

Hi badger dateandtime is a field that I join, before that exists separated time and date fields, so I was thinking maybe to do this procedure only to the date field, do you think my code below is correct?

"Why don't he test this by himself" you will think.....today is 03/03/2020 :grinning: so I wont notice any change

    ruby {
        code => '
            t = event.get("date")
            event.set("date", Time.at(t.to_f).strftime("%d/%m/%Y"))
        '
    }
7

If I remember correctly, if you event.set a field that already exists then it becomes an array, so you would want to event.remove("date") between the event.get and event.set.

9

Doesn't seems to create an array based on the rubydebug outpupt, but it send me back to the previous millennium :grin:

{
          "tags" => [],
       "message" => "14:34:47 03/03/2020 name:monitor_Uptime targets:server.win state:System\\System Up Time=432472.45596 System\\System Up Time=432472.45596 type:Windows Resources unique:1952452212",
        "fields" => {
        "metrica" => "uptime"
    },
          "hora" => "14:34:47",
     "sitescope" => "claro",
      "@version" => "1",
      "segundos" => 432472.45596,
         "date" => "31/12/1969",
           "log" => {
        "offset" => 472513,
          "file" => {
            "path" => "E:\\uptime_03-03-2020_14-37-22.log"
        }
    },
          "tipo" => "Windows Resources",
    "horayfecha" => "14:34:47 31/12/1969",
            "id" => 1952452212,
         "input" => {
        "type" => "log"
    },
           "ecs" => {
        "version" => "1.4.0"
    },
       "monitor" => "monitor_Uptime",
      "hostname" => "server.win",
    "@timestamp" => 1969-12-31T17:34:47.000Z
}
10

What does the date field look like in the rubydebug output if you remove the ruby filter?

11

"date" => "03/03/2020",

12

The "Time.at(t.to_f).strftime" assumes that t is a LogStash::Timestamp. You need to use a date filter to parse date (and overwrite it) then the ruby filter should work.

1 Like
13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.