Find current event logs

Elastic Stack Kibana
1

Hello

I have set of log documents categorized with event-ids which is string in form of timestamp.
I want to find the document with the earliest timestamp so I can always visualize the current set of logs.

This is what I have done with scripted field but not working, please I need help.

```painless
 def event_latest = 0L; 
    for (def i = 0; i < doc['event_id'].length; i++) {
    def current_date = doc['event_id'][i].getValue().toInstant().toEpochMilli();
         if (current_date > event_latest)
            event_latest = current_date;
     }
    return event_latest;
2

If event_id is of type string and not date, you have to parse it first - see https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-datetime.html#_datetime_parsing_examples for examples how to do it.

3

@flash1293 . Thanks for your suggestion. I parsed into date but did not work.

Maybe someone can help me modify the code, I want to find the latest document based on the event_id (timestamp).

Sorry for late in reply.

4

Can you share the code where you attempt the date parsing? An example document would also be helpful to find a solution.

5

@flash1293

Thank you. This is my code below:

 def event_latest = 0L; 
    for (def i = 0; i < doc['indexer_job_id'].length; i++) {
    def datetime = doc['indexer_job_id'][i].getValue().toInstant().toEpochMilli();
    ZonedDateTime current_date = ZonedDateTime.parse(datetime);
         if (current_date > event_latest)
            event_latest = current_date;
     }
    return event_latest;

I still get shards failures !!

6

Can you show an example of an indexer_event_id?

7

indexer_job_id is an event-id in a form of string eg. "2020-05-28T00:03:00.001106Z"

8

You have to call parse with the value from the doc, you can only call toInstant on an already parsed date:

def d = ZonedDateTime.parse(doc['indexer_job_id'][i].getValue());
def timestamp = d.toInstant().toEpochMilli();

But re-reading your question I'm not even sure whether the thing you want to do is possible using scripted fields. To you want to find the latest event id within the current document (indexer_job_id being an array field with multiple entries), or are there multiple documents you want to find the latest?

For the first use case this is the right approach, but it's not for the latter one. A scripted field is executed once for each document, you can't look up values within other documents inside of it.

To visualize the value of a field from the latest document in the current time range, you can use a metric visualization and the "Top Hit" aggregation. If indexer_job_id is always a date, you should specify it as date in the index mapping, then Elasticsearch will know how to handle it.

9

Thanks @flash1293.
Yes, it is the later, I want to find the latest event of multiple documents so I can dynamically visual the values of the latest event; ie I want visualisation that will always find the latest values of the current or latest event not manually with a specific event_id.

If not possible with scripted field, Is any approach to dynamically visualized values based on the latest event. ?

10

See the last paragraph in my previous post.

11

Thanks @flash1293 . I will try that and give you feedback.

Best regards

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.