Using ElasticSearch for logs

[hhkx (Hasan)]

Hi there,

I am new to ElasticSearch. So I need your kind helps.

We are developing a huge environment on smart meters and have been researching some searching tools to use it fast for logs.

We are planning to log every action of all users and some behaviours of all meters. Something like "UserX removed meterThis", "meterThat's signal is %75" etc.

So as you may imagine there will be around 10 million logs per day.

What we think to use ElasticSearch in this system is that when a system administrator wants to search a spesific user's activity or a meters behaviour...

So I am asking now firstly can ElasticSearch handle this amount of data?

And why should I use ElasticSearch not a bigdata database solution like MongoDB?

Hopefully, I haven't been the one who expects everything to be handed on a silver platter.

Thanks in advance.

[Christian_Dahlqvist]

From an Elasticsearch perspective 10 million events a day is not necessarily very much. I have seen clusters handling billions of events per day, so suspect data volume will not be a issue here. Elasticsearch offers flexible querying out of the box as well as a wide range of powerful aggregations. Whether these features and flexibility will make it easier to develop your solution or some other data store works just as well for your requirements you will need to decide.

[hhkx (Hasan)]

Well, thanks for your reply Christian.