Apologies, I don't know whether this is the correct forum for this question, please bear with me.
I have an elasticsearch index containing mail logging events from syslog, however as with postfix the complete transaction for an email is difficult to determine from an individual event and must be correlated together using the queue id across multiple logging events to piece the whole story together
I'm used to doing this sort of log analysis manually with scripts I have developed, but I'm not sure even the terminology I could use to search for documentation on something like this in the elasticsearch ecosystem ... what should I be looking for?
we're using td-agent/fluent to do log shipping from our fleet to our elasticsearch nodes and if this is something that needs to happen prior to elasticsearch indexing and then kibana performing searches then we can look at that - my assumption is that we might want to pull the relevant raw records from the index, do $magic, and then push that back into the index later to search on
Any pointers appreciated, thanks