I have a web application that generate log files which are then ingested to via Beat-Logstash to ElasticSearch. Now I need to combine information from different log lines or even different log files before insert the log documents to elasticsearch.
E.g. the username below appears in line 50 but I need to also include it in the document of line 100, which refers to a user action. The glue between the two lines in the session-id.
50: [2018-12-13 13:45:32] [INFO] [Yud52I234hGVgdj] User with username=test loggedin with success 51: [2018-12-13 13:45:33] [INFO] [Yud52I234hGVgdj] Initializing user assets ... 100: [2018-12-13 13:48:15] [INFO] [Yud52I234hGVgdj] Page Catalogue entered
I want username from line 50 to populate to the document of line 100.
Are there approaches to enrich the data during ingest, other than add the username in the log of line 100?