Nested field in app search

What is best practice when needing to store data in a nested field for app search

I have the following example, where the events key can hold an array of
events

How would i perform a search to find all documents where the events.slug = 'demo-sale' for example?

AFAIK our "best practice" is generally to flatten your data. YMMV doing this and I don't know that we have a really good solution right now. But here are some thoughts:

A simple solution would be to do the following:

[{
  id: 1,
  name: 'top-level-thing'
  events: [
    {description: 'desc1', name: 'name1', slug: 'demo-sale'},
    {description: 'desc1', name: 'name2', slug: 'another-slug'}
  ]
}]

You could have:

[{
  id: 1,
  name: 'top-level-thing',
  events.descriptions: ['desc1', 'desc2']
  events.names: ['name1', 'name2']
  events.slugs: ['demo-sale', 'another-slug']
}]

That will get your use case working, searching by event.slug. However, you'll find it difficult if you want to do any facet filtering by sub-fields. I.e., you can't do a facet filter on both events.name and events.slug.

A variant of that would be indexing multiple variants of your top level document, each with a unique event:

[{
  id: 1,
  name: 'top-level-thing'
  event: {description: 'desc1', name: 'name1', slug: 'demo-sale'}
},{
{
  id: 1,
  name: 'top-level-thing'
  event: {description: 'desc1', name: 'name2', slug: 'another-slug'}
}]

You'd then use grouping on your queries to group results: https://www.elastic.co/guide/en/app-search/current/grouping.html by the top level document.

This strategy winds up having some challenges around the management of duplicated documents. Grouping has its own unique challenges as well, which are mentioned in the documentation.

Yet another variant of that would be indexing at an event level and having the top level document be the attribute

[
  {
    description: 'desc1',
    name: 'name1',
    slug: 'demo-sale',
    document.name: 'top-level-thing',
    document.id: 1
  },
  {
    description: 'desc1',
    name: 'name2',
    slug: 'another-slug',
    document.name: 'top-level-thing',
    document.id: 1
  }
]

I'm not sure how that would work out.

That brings me to my last recommendation... which I've heard works but I haven't personally seen in practice, which is to have two separate engines

// Events
[
  {id: 'e1', description: 'desc1', name: 'name1', slug: 'demo-sale'},
  {id: 'e2', description: 'desc1', name: 'name2', slug: 'another-slug'}
]

// Documents
[{
  id: 1,
  name: 'top-level-thing'
  events: ['e1', 'e2']
}]

To do this, I think you would essentially need to query using two distinct queries.

Query 1: A query on the Events engine for slug = 'demo-sale', which would result in a list of engine ids.
Query 2: A query on the Documents engine, which filters on the events attribute by the list of ids.

Huge disclaimer: I personally have not done any of these, so this is all theoretical. I would love to hear from others that may have experience with this. I would also love to hear back from you if any of these solutions work.

2 Likes

Thanks so much for the detailed response Jason!

I think the first solution is probably the best, I'd prefer to keep the simplest solution, without having to add additional documents or engines.

It allows us to get documents part of events and it's sufficient to facet filter on simply one of the event fields.