Flatten _source field of query results

Hello,

At our company we are using Elasticsearch together with GraphQL
The problem we are facing is that we would like to cleanly fetch and return the _source content of elasticsearch documents using the @elastic/elasticsearch npm package.

We got to the point where we can return documents using the following query:

await elasticsearchClient.search({
            index: 'my_index',
            body: {
              query: {
                match: {
                  topic
                }
              }
            },
            // -> don't return elasticsearch metadata
            filterPath: 'hits.hits._source',
            // NOTE: we are currently filtering out
            //       the fields '@version' and '@timestamp'
            _source: ['id', 'timestamp', 'topic', 'index', 'type']
 })

The problem with this approach is that we get results that are formatted like this:

[ { _source:
     { topic: 'user-logged-in',
       index: 'tracking_events',
       id: '5e1b528aa1ca021659fb1b5b',
       type: 'redis-input',
       timestamp: '2020-01-12T17:08:26.094Z' } },
  { _source:
     { topic: 'user-logged-in',
       index: 'tracking_events',
       id: '5e1b56f0a1ca021659fb1b5c',
       type: 'redis-input',
       timestamp: '2020-01-12T17:27:13.020Z' } } ]

How can we get rid of the top-level _source field?

We would like to format the results and return something that looks like this:

[ { topic: 'user-logged-in',
    index: 'tracking_events',
    id: '5e1b528aa1ca021659fb1b5b',
    type: 'redis-input',
    timestamp: '2020-01-12T17:08:26.094Z' },
  { topic: 'user-logged-in',
    index: 'tracking_events',
    id: '5e1b56f0a1ca021659fb1b5c',
    type: 'redis-input',
    timestamp: '2020-01-12T17:27:13.020Z' } ]

Straightforward approach:

const formattedResults = hits.map(event => event._source)

But this of course is infeasible for obvious performance reasons.

Thank you for your help!

Hey,

Elasticsearch has no mechanism to change the JSON output of a search response. the filter_path is just about omitting data, but not about changing the structure.

If you need this, it's best to do it on the client side.

--Alex

Ok, thank you for your help.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.