Return (emit) @timestamp date value in runtime field

Dear all,

I am trying to output the value of @timestamp in a runtime field (configured in Kibana 7.13.3). My goal is then to get only the month as display value with a modified format 'MMM'. Unfortunately, I have been unsuccessful so far with all possible combinations that can be found on Google. I have not even managed to output the "normal" value of @timestamp or any other date field in a new runtime field.

emit(doc['@timestamp'].value);
                      ^---- HERE

In Discover, I then get this error message:

cannot convert MethodHandle(Dates)JodaCompatibleZonedDateTime to (Object)long

I also tried a gazillion of other things... with no success.

The same works great with a keyword field if I am using this:

emit(doc['user_name'].value);

How the heck does that work with dates :slightly_smiling_face: ???

Thanks a lot for your help!!!

-- Cheers, Nils

emit(doc['@timestamp'].value.toString())
2022-07-23T22:31:10.171Z

emit(doc['@timestamp'].value.month.getDisplayName(TextStyle.FULL, Locale.ROOT))
July

emit(doc['@timestamp'].value.month.getDisplayName(TextStyle.SHORT, Locale.ROOT));
Jul

ZonedDateTime zdt = ZonedDateTime.parse(doc['@timestamp'].value.toString());
emit(zdt.getMonthValue());
7 (integer)

ZonedDateTime zdt = ZonedDateTime.parse(doc['@timestamp'].value.toString());
emit(zdt.getMonthValue().toString());
7 (keyword)

emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))
Saturday

1 Like

Hey Stephen,

thanks a lot. This is a great resource of examples.

Could you help me further to return the value of @timestamp as a Date type in the runtime field? Using .toString(), which requires the runtime field to be a Keyword field, brings the problem that I cannot configure any custom date formatting, like MMM,

If I try to use emit(doc['@timestamp'].value.toString()) and runtime field type Date, the Discover app shows an error:

cannot convert MethodHandle(Object)String to (Object)long

-- Cheers, Nils

Apologies I am Confused what you want then

@timestamp is already a date

emit(doc['@timestamp'].value.toString()) is a string

Can you show me exactly what you want before and after and what data types you want before and after and how you plan to use the result and what type you want the result... I am lost / confused at this point.

Hi Stephen,

thank you very much for your help!

I have data from about the last 10 years and would like an evaluation of which month has the most entries. As a date field I have the @timestamp field available. I need an evaluation of the data per month. Month should be for example "7" or "July". That would not matter. My plan was to create a runtime field, which gets the value of @timestamp and is displayed in the format MMM. So I still have the @timestamp field and additionally the month in a separate field.

-- Cheers, Nils

That is exactly what I showed you... confused... you have to name it as a new field..

You add a new field...this is in the Index Pattern if you which will work for visualizations.

BUT if you actually want to do searches / aggregation etc.. etc.. you need to add it to the mapping

PUT my-index-000001/
{
  "mappings": {
    "runtime": {
      "date_month_MMM": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['@timestamp'].value.month.getDisplayName(TextStyle.SHORT, Locale.ROOT));"
        }
      }
    }

Hi Stephen,

thanks for the clarification of the difference between mapping and index pattern. I am aware of this now. I also understand now how to return the month name as a string (Keyword runtime field), but I am still unsure how to return the value of @timestamp as a date (Date runtime field). Maybe I am just confused and the answer is too simple :slightly_smiling_face: Sorry if that is the case...

-- Cheers, Nils

@timestamp is already always available as a "indexed / concrete field" as a date type... why do you need it as a runtime field? @timestamp will always be available.

What do you mean the the value of @timestamp as a date (Date runtime field)

Can you show me what you mean? Are you just trying to format the date in a certain format?

Are you trying to build visualizations or DSL queries?

Hi Stephen,

I want to show and use the Month and the Year in separate additional fields. My idea is to keep the fields as Date fields and just apply a custom formatting as needed.

-- Cheers, Nils

Ok just create another field with the Year part... you can create a field if you like for every part of the date if you like.

Hi Stephen,

let us just assume that I want to duplicate the @timestamp field into a runtime field. Let us call this runtime field my_date. How do I return the date value of the @timstamp field in the my_date field? The my_date field is a Date field type.

:slightly_smiling_face:

-- Cheers, Nils

Well then I would probably use an alias... not a runtime field...

And remember all dates are actually stored as epoch millis long... all you ever really see is the formatted date output...

I will look at how to duplicate a field later...

Think this will work, but put it as a mapping

ZonedDateTime zdt = ZonedDateTime.parse(doc['@timestamp'].value.toString());
emit(zdt.toEpochMilli());

I changed the name so I could see them side by side

I had to set the date formatters to look the same, That formats the output that is set in the Data View / Index Pattern

Hi Stephen,

when you said ...

... that helped me a lot! I now have what I was looking for.

My solution is the following:

// Create a zoned datetime 
ZonedDateTime zdt = ZonedDateTime.parse(doc['@timestamp'].value.toString());

then:

// Return as long milli
emit(zdt.toInstant().toEpochMilli());

This discussion also helped me to find the correct syntax:

-- Cheers, Nils

1 Like

Hi Stephen,

great, this is exactly what I was looking for!

Thank you so much!

-- Cheers, Nils

1 Like

Cool that is good too!!

Why you can not just turn around an emit the value is unclear to me... (bugging me actually)

That is what I was trying first, but without any success. So I thought that must be the wrong way.

-- Cheers, Nils

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.