Use case for App Search?

[ismarslomic]

While setting up and reading online documentation of Elasticsearch, I came over App Search. Since I have never used App Search before I wanted to give it a try by running Elasticsearch and App Search in Docker containers locally. After simple PoC Im not completely sure whether I should use it or not for my use case. So Im reaching out to community to get some inputs.

My use case:
I want to store 2 JSON documents from my Kafka-broker to Elasticsearch for analysis and visualization in custom made React App. User needs to search specific document and retrieve full payload that will be visualized in tabular and map format. Documents will be stored for only 30 days of length.

My original thought was to use App Search for search purpose in React App, and access directly from Elasticsearch the full payload of specific document to represent in tabular and map format. That would required to create documents in both App Search and Elasticsearch from my backend-application (Kotlin/Java) while reading new documents from the Kafka topic.

Characteristics of Document #1:

1. Up to 2MB in size
2. Many nested objects, also in arrays
3. Fields from nested objects will be needed to represent user friendly search results
4. About 400 000 documents in 30 days

Characteristics of Document #2:

1. Up to 100KB in size
2. No nested objects
3. About 2M documents in 30 days

Doubts:

1. Since App Search does not support nested objects, I would need to flatten my data. Based on reply on this thread this doesn't seem to be straight forward and would increase complexity of my solution
2. App Search does not provide API Client for Java/Spring/Kotlin (see Programming language clients), which means increased complexity in backend app while using the low level Rest clients. Elasticsearch on the other hand, has High Level API Client for Spring.
3. Im uncertain about maturity and maintenance of App Search and exactly benefit of using App Search for search compared with accessing REST API of Elasticsearch from React application

I would appreciate if someone with experience with Elasticsearch and App Search can shed light on my doubts so I can perform correct decision.

[nickchow]

Hi ismarslomic,

Interesting use case!

First, to directly answer your questions:

1. Nested objects: that's correct, you'd likely need to reconsider how you structure your data using suggestions from that thread.
2. There isn't a Java-based client for App Search right now. Working directly with our APIs is quite pleasant though – we have put a lot of thought into making our APIs easy to use!
3. There is most certainly active work on App Search – every release brings exciting new features! And App Search is used to power mission-critical search by many companies.

On another note – searching over documents that are 2MB in size may have some performance implications. Would it be possible to break those into smaller individual documents? (Perhaps that may change how you architect the fields as well).

Hope that helps,
Nick