Problem Statement

Elastic Stack Elasticsearch
1

I have 3 different files ( 3 diff csv files) - transaction data, card info data and user data. I need to merge all these data into single file (I mean to perform SQL type join based on a key). I am aware that we can’t perform joins in elasticsearch.

So what approach to solve this problem. I need to join these documents and inject the data into single index on Kibana.

My options – let me know if this can be possible.

Use logstash- ETL tool, do data structuring and transformation there ( is it possible to do joins in logstash? )
Or use join queries and mapping documents and inner_hits concepts in Kibana console ?
Which of the above is feasible or if any new recommendations to solve this problem?

2

I'd use Logstash to do it, have a look at the translate filter for eg.

3

Thanks a ton. Can I know if any example solving related issue using translate filter?

4

There's examples in the documentation, but also lots on here if you do a search :slight_smile:

5

Writing a python script on the Elasticsearch end can be easier than merging on the Logstash end?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.