Hi,
We are storing our product logs into Elasticsearch. I want to count the number of occurrences of each log to find the high runners. Now problem is many logs contain some session specific numerical ID. Due to that the same log message with two different IDs will be shown as separate occurrence when done using terms aggregation.
I understand one option would be to mask the numbers during ingestion but I don't want change the message. Second option would be to add this as a new field but that would require more storage.
The option I am thinking is to query the logs and then do all the transformation/calculation in a separate client application.
Wanted to check with experts here if there is any better way that I am unaware of.
Thanks
Nikhil