ElasticSearch - many data - can ES handle it?

Elastic Stack Elasticsearch
#1

I have Events entity in my database. Event looks like this:

ID, NAME, DATE, CITY

Now.. I integrated ElasticSearch JavaAPI to my application.
There are millions of events in database.

So after typing New in searchbox I want to search events and display them grouped by cities.

Questions:

  1. Is it possible to query ElasticSearch for such data structure having only such document sturture: Events(ID, NAME, DATE, CITY) indexed ?

  2. Should ElasticSearch have Events(ID, NAME, DATE, CITY) records indexed or some more complicated documents, ready for display?

There are 1,5millions of new events daily..

#2

ES should be able to readily handle such a workload but of course it depends on your hardware characteristics, how many nodes, etc. You will want to experiment and profile with your data. For suggesters, you will want to look at the suggester API: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html

I don't follow the question. You should have fields on your documents that are pertinent for searching and retrieving from Elasticsearch. Its up to your application's needs what information would be important to store for each event.

#3

Thanks for your answer. I didn't know the suggesters.

Generally I wonder what to put into ES index. It depends on ES capabilities and best practice / experience which I don't have yet.

#4

You will want to read up on these sections of the guide at minimum:

https://www.elastic.co/guide/en/elasticsearch/guide/current/capacity-planning.html

https://www.elastic.co/guide/en/elasticsearch/guide/current/modeling-your-data.html

In general, there is no better way than to experiment and determine what your capacity needs to be. ES is used by many organizations to handle more events than that per day, but it depends on document size, structure, hardware, query characteristics, etc. etc.

#5