# Elastic Search to Monitor and store data from client side machine
**URL:** https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723
**Category:** Elasticsearch
**Created:** [October 24, 2018, 6:05am UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723 "2018-10-24T06:05:07Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@somu\_p](https://discuss.elastic.co/u/somu_p)
#### Post date: [October 24, 2018, 6:05am UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/1 "2018-10-24T06:05:07Z")
i am new to ES i would like to monitor 100000 endpoint with Metric Beats.
Can you please help me with below question:
Can i monitor 100000 system with beats. is it possible?
can i store those data to Elastic search, since data generated for those endpoints will be huge.
what are the points i should consider while doing something like this. since all endpoint will be pointing to one Elastic search server.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 24, 2018, 6:14am UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/2 "2018-10-24T06:14:41Z")
> [@somu\_p](#):
>
> Can i monitor 100000 system with beats. is it possible?
> can i store those data to Elastic search, since data generated for those endpoints will be huge.
Yes, that should be possible. Elasticsearch can scale to handle very large data volumes.
> [@somu\_p](#):
>
> what are the points i should consider while doing something like this. since all endpoint will be pointing to one Elastic search server.
If the data volume is huge you will need a sizeable Elasticsearch cluster, not just a single server. It is probably also impractical to have 100000 Beats write directly to Elasticsearch, so you will probably need to design an ingest pipeline like described in [this blog post.](https://www.elastic.co/blog/small-medium-or-large-scaling-elasticsearch-and-evolving-the-elastic-stack-to-fit)
---
### Author:  [@somu\_p](https://discuss.elastic.co/u/somu_p)
#### Post date: [October 25, 2018, 7:05am UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/3 "2018-10-25T07:05:23Z")
can i use kafka as buffer before storing to ES or Directly contacting ES after filtering and processing via Logstash. i want to run some ML algorithm on the monitored data.
will there be any performance issue if i Run my ML algorithm on ES data.
---
### Author:  [@somu\_p](https://discuss.elastic.co/u/somu_p)
#### Post date: [October 31, 2018, 11:51am UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/4 "2018-10-31T11:51:03Z")
> [@somu\_p](#):
>
> will there be any performance issue if i Run my ML algorithm on ES data.
can anyone reply on the thread from ES team.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 31, 2018, 12:04pm UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/5 "2018-10-31T12:04:41Z")
> [@somu\_p](#):
>
> can i use kafka as buffer before storing to ES or Directly contacting ES after filtering and processing via Logstash.
Yes, that should be possible.
> [@somu\_p](#):
>
> i want to run some ML algorithm on the monitored data.
> will there be any performance issue if i Run my ML algorithm on ES data.
There is really not a lot of information about exactly what you will be running, which makes it hard to judge what impact it may have on the performance of the cluster.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 28, 2018, 12:04pm UTC](https://discuss.elastic.co/t/elastic-search-to-monitor-and-store-data-from-client-side-machine/153723/6 "2018-11-28T12:04:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.