Flatten _source field of query results

Elastic Stack Elasticsearch
1

Hello,

At our company we are using Elasticsearch together with GraphQL
The problem we are facing is that we would like to cleanly fetch and return the _source content of elasticsearch documents using the @elastic/elasticsearch npm package.

We got to the point where we can return documents using the following query:

await elasticsearchClient.search({
            index: 'my_index',
            body: {
              query: {
                match: {
                  topic
                }
              }
            },
            // -> don't return elasticsearch metadata
            filterPath: 'hits.hits._source',
            // NOTE: we are currently filtering out
            //       the fields '@version' and '@timestamp'
            _source: ['id', 'timestamp', 'topic', 'index', 'type']
 })

The problem with this approach is that we get results that are formatted like this:

[ { _source:
     { topic: 'user-logged-in',
       index: 'tracking_events',
       id: '5e1b528aa1ca021659fb1b5b',
       type: 'redis-input',
       timestamp: '2020-01-12T17:08:26.094Z' } },
  { _source:
     { topic: 'user-logged-in',
       index: 'tracking_events',
       id: '5e1b56f0a1ca021659fb1b5c',
       type: 'redis-input',
       timestamp: '2020-01-12T17:27:13.020Z' } } ]

How can we get rid of the top-level _source field?

We would like to format the results and return something that looks like this:

[ { topic: 'user-logged-in',
    index: 'tracking_events',
    id: '5e1b528aa1ca021659fb1b5b',
    type: 'redis-input',
    timestamp: '2020-01-12T17:08:26.094Z' },
  { topic: 'user-logged-in',
    index: 'tracking_events',
    id: '5e1b56f0a1ca021659fb1b5c',
    type: 'redis-input',
    timestamp: '2020-01-12T17:27:13.020Z' } ]

Straightforward approach:

const formattedResults = hits.map(event => event._source)

But this of course is infeasible for obvious performance reasons.

Thank you for your help!

2

Hey,

Elasticsearch has no mechanism to change the JSON output of a search response. the filter_path is just about omitting data, but not about changing the structure.

If you need this, it's best to do it on the client side.

--Alex

3

Ok, thank you for your help.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.