Multiple Entity Type Indexing and Search

Elastic Search
1

Hi there :wave:

Let's say I have an application where I need to index multiple different types of entities: users, courses, programs, etc. Each of these entities will have a specific schema of their own, and different search tuning settings (searchable fields, weights, etc.).

Now:

  • What is the right way to index them ? (e.g. 1 engine per entity if we are using App search ? )
  • Then, how would I perform a search query request that searches across all different entities and returns results (using app search nodejs client for example):
    • (A) in separated sets if we were to build a UI that displays each type of entity separately.
    • (B) as a unified set if we were to build a UI where all items are presented in a unified list

Thanks in advance

2

Hi!

You're right, 1 engine per entity seems like the best approach here.
A) to get results in separate sets, you'll have to query each engine individually.
B) to get a unified set of results, you'll need to create a meta-engine that includes all the other engines. Then you could query only the meta-engine and get all the results in a single request.

Hope this helps!

3

Thanks for the answer @Vadim_Yakhin.

Regarding:

to get a unified set of results, you'll need to create a meta-engine that includes all the other engines. Then you could query only the meta-engine and get all the results in a single request

I was under the impression, after reading the meta engines guide, that they were meant for grouping engines with identical schemas and not for engines that would represent different entities ? So, the latter (different entities with different schemas) is a valid and encouraged use case ?

to get results in separate sets, you'll have to query each engine individually.

So for engines right now there is not an API like the one for Elasticsearch indexes to target multiple indexes in one HTTP call ? There would need to be one HTTP call per engine ?

Thanks in advance.

4

Great questions.

Just curious, what gave you that impression? Maybe we need to update the docs. I believe meta engines were created specifically for this use-case.

If it exists, I've never heard of it. :slight_smile: The way to do it is via meta-engines. And yes, you'll need one HTTP call per engine if you decide to not use meta-engines.

5

Thanks again @Vadim_Yakhin :+1:

Just curious, what gave you that impression?

I think it was mainly that the example provided only talks about a same entity: a national park (eastern vs western); there is no mention/example for entities with different schemas.

Perhaps also:

  • Unified Schemas: Meta Engine schemas are the union of all Source Engines schemas. Therefore, if two Source Engines share a field, their names must match - if the names are different, they will be treated as a different field! As well, the fields need to have the same type. See more below on how schema conflicts are treated.

It would be nice to see an example result for this section for entities that are completely different. E.g. National Parks and Hotels: for example, do fields that are present in one appear as null when merging schemas ?

I think in general, a brief mention/example of 2 completely different entities (schema wise) would be useful.

Cheers.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.