Need to send query in my data view on 2 separate document

Hi,
I’m working with a Dataview where I’m querying to extract specific information from raw data for my use case. The issue I’m facing is that I have multiple documents, and some of the fields I need to query are not in the same document. Is it possible to send a query that targets two different documents within the same index in the Dataview?

Thanks for reaching out, @ermisma. Can you provide more context on how your index and documents are structured? I'd also like to hear more about the results you are looking for from your data view.

Thank you
I have a single data view on top of a single index, but the output I need from my query requires fields that are spread across separate documents/events. In traditional databases, you’d perform a join between tables, but I’m not sure how to handle this in ELK. Do I need to create a new index using a transform, or is there another approach you’d recommend?

For example, I have one document (the raw data in my data view) with fields like [Job_id, Job_status] , and another document with fields like [Job_id, Job_name] . My query needs to use both Job_status and Job_name , but since these fields are in different documents, I’m getting zero results. Is there a way to query fields from different documents in ELK?

This should be a FAQ.

The answer to “how to do what you want” here is very dependent on precisely what you want. There’s no overall join equivalent that matches SQL JOIN, but a lot of problems are solvable with different techniques , maybe correctly formatted ES|QL query, or a transform, or adjusting input pipeline with some sort of enrichment.

As always, a few sample documents help a lot, as well as clear and complete explanation of the requirements.

1 Like