While setting up and reading online documentation of Elasticsearch, I came over App Search. Since I have never used App Search before I wanted to give it a try by running Elasticsearch and App Search in Docker containers locally. After simple PoC Im not completely sure whether I should use it or not for my use case. So Im reaching out to community to get some inputs.
My use case:
I want to store 2 JSON documents from my Kafka-broker to Elasticsearch for analysis and visualization in custom made React App. User needs to search specific document and retrieve full payload that will be visualized in tabular and map format. Documents will be stored for only 30 days of length.
My original thought was to use App Search for search purpose in React App, and access directly from Elasticsearch the full payload of specific document to represent in tabular and map format. That would required to create documents in both App Search and Elasticsearch from my backend-application (Kotlin/Java) while reading new documents from the Kafka topic.
Characteristics of Document #1:
- Up to 2MB in size
- Many nested objects, also in arrays
- Fields from nested objects will be needed to represent user friendly search results
- About 400 000 documents in 30 days
Characteristics of Document #2:
- Up to 100KB in size
- No nested objects
- About 2M documents in 30 days
- Since App Search does not support nested objects, I would need to flatten my data. Based on reply on this thread this doesn't seem to be straight forward and would increase complexity of my solution
- App Search does not provide API Client for Java/Spring/Kotlin (see Programming language clients), which means increased complexity in backend app while using the low level Rest clients. Elasticsearch on the other hand, has High Level API Client for Spring.
- Im uncertain about maturity and maintenance of App Search and exactly benefit of using App Search for search compared with accessing REST API of Elasticsearch from React application
I would appreciate if someone with experience with Elasticsearch and App Search can shed light on my doubts so I can perform correct decision.