Hello,
I wonder, if there is a way to join data that I'm currently receiving (in real time) with existing data (in elasticsearch).
If yes, could you point me to how to do it?
Thank you!
Join, no. But you can look it up https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html and then do stuff.
It is kind of unclear. The filter is appling only on the current message if I understand it well. So can a filter help me filter on existing data?
In fact I'm receving actions/events done by users (I have their userid) and I would like to know who is the user (I have an account table).
Thank you!
No, you need to reindex that.
Apologies, I'm still learning..
I cannot understant , event if I reindex everything, how can the filter help me? I will hot have the account message and the account action message in the same time...
Thank you!
If you have data in ES in an index that you want to enrich other incoming events with, then you need to use the filter I linked to to read that data that is in ES to then do something with.
If you have older data that you want to do this enrichment on, you need to reindex it through Logstash.