Adding new field to existing index where field does not exist

Firstly, totally new to the ELK stack so still struggling with the terminology.

I have been trying to create some visualizations and discovered that merging fields via a JSON scipt is kind of finicky, and apparently not recommended.

eg, I want to visualize the number of unique sessions but the sessions are only unique to a given hostname, therefore this script in the aggregation achieves that.

    {
      "script": {
      "source": "doc['hostname.keyword'].value + doc['session_id.keyword'].value",
      "lang": "painless"
      }
    }

So it turns out from my research that what I should do is add another Field during my GROK parsing so that this field exists in each document, making the search faster.

So, the Grok part can be done with the following:

add_field => { "hostname_session_id" => "%{hostname}:%{session_id}" }

So what's missing for me, is how I go about updating all the entries already in the index so they have this field added historically and what's the best way to do this on a live system where the index is getting appended too?

It seems like I need to do a script with a check if exists, and if not create. But i cannot seem to get the syntax correct.

My Grok currently creates a new index each day, and I am up too day seven, so six static Indexes and another one that's still being appended too.

Thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Hey,

you can use the update by query API using a script.

--Alex