# ES load testing?
**URL:** https://discuss.elastic.co/t/es-load-testing/21028
**Category:** Elasticsearch
**Created:** [December 2, 2014, 11:38am UTC](https://discuss.elastic.co/t/es-load-testing/21028 "2014-12-02T11:38:37Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Siddharth\_Trikha](https://discuss.elastic.co/u/Siddharth_Trikha)
#### Post date: [December 2, 2014, 11:38am UTC](https://discuss.elastic.co/t/es-load-testing/21028/1 "2014-12-02T11:38:37Z")
I had tested my ELK setup on a small set of logs and got it working.
Today I started it's testing on actual log files of a server. So one
server's one month data with around 10 type of logs.
So approximately 500 files to be monitored each approx 1k lines, comes upto
5 lac log lines.
\*\*: I am using default setting for elasticsearch.
So when I started logstash ES, first I saw logstash kept running for 2
hours just reading logs and after that it gave following error on console:
_log4j, [2014-12-02T16:03:19.087] DEBUG: org.elasticsearch.transport.netty:
[logstash-XX-] disconnected from
[[#zen\_unicast\_2#][XX[inet[localhost/127.0.0.1:9301]]]log4j,
[2014-12-02T16:03:19.087] DEBUG: org.elasticsearch.discovery.zen:
[logstashXX] filtered ping responses: (filter\_client[true],
filter\_data[false]) {none}log4j, [2014-12-02T16:03:19.089] DEBUG:
org.elasticsearch.transport.netty: [logstashXX] connected to node
[[#zen\_unicast\_2#][inet[localhost/127.0.0.1:9301]]]_
ES console:
_[2014-12-02 12:23:24,289][INFO][cluster.metadata] [Mantis]
[logstash-2014.12.02] creating index, cause [auto(bulk api)], shards
[5]/[1], mappings [default][2014-12-02 12:23:24,972][INFO
][cluster.metadata] [Mantis] [logstash-2014.12.02] update\_mapping
[logs] (dynamic)[2014-12-02 12:23:39,675][INFO][cluster.metadata]
[Mantis] [logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
12:51:13,975][INFO][cluster.metadata] [Mantis]
[logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
12:51:26,389][INFO][cluster.metadata] [Mantis]
[logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
12:55:18,142][INFO][cluster.metadata] [Mantis]
[logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
12:55:31,564][INFO][cluster.metadata] [Mantis]
[logstash-2014.12.02] update\_mapping [logs] (dynamic)Write failed: Broken
pipe_
I guess the connection is lost bw logstash and ES.
I am using ELK first time with such amount data .
Any guidance??
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0d820088-3604-426e-b50e-d5c3dccd7ee5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d820088-3604-426e-b50e-d5c3dccd7ee5%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@eliasah](https://discuss.elastic.co/u/eliasah)
#### Post date: [December 2, 2014, 12:44pm UTC](https://discuss.elastic.co/t/es-load-testing/21028/2 "2014-12-02T12:44:08Z")
I have used Tsung to load test my clusters. It's very easy to install and
configure.
It will give more insight about memory usage, i/o, network i/o etc.
I suggest that you read this blog entry
[https://engineering.helpshift.com/2014/tsung/](https://engineering.helpshift.com/2014/tsung/) about it.
You'll find here [http://tsung.erlang-projects.org/](http://tsung.erlang-projects.org/) the official page for
Tsung.
Le mardi 2 décembre 2014 12:38:37 UTC+1, Siddharth Trikha a écrit :
> I had tested my ELK setup on a small set of logs and got it working.
>
> Today I started it's testing on actual log files of a server. So one
> server's one month data with around 10 type of logs.
> So approximately 500 files to be monitored each approx 1k lines, comes
> upto 5 lac log lines.
>
> \*\*: I am using default setting for elasticsearch.
>
> So when I started logstash ES, first I saw logstash kept running for 2
> hours just reading logs and after that it gave following error on console:
>
> _log4j, [2014-12-02T16:03:19.087] DEBUG:
> org.elasticsearch.transport.netty: [logstash-XX-] disconnected from
> [[#zen\_unicast\_2#][XX[inet[localhost/127.0.0.1:9301]]]log4j,
> [2014-12-02T16:03:19.087] DEBUG: org.elasticsearch.discovery.zen:
> [logstashXX] filtered ping responses: (filter\_client[true],
> filter\_data[false]) {none}log4j, [2014-12-02T16:03:19.089] DEBUG:
> org.elasticsearch.transport.netty: [logstashXX] connected to node
> [[#zen\_unicast\_2#][inet[localhost/127.0.0.1:9301]]]_
> ES console:
>
> _[2014-12-02 12:23:24,289][INFO][cluster.metadata] [Mantis]
> [logstash-2014.12.02] creating index, cause [auto(bulk api)], shards
> [5]/[1], mappings [default][2014-12-02 12:23:24,972][INFO
> ][cluster.metadata] [Mantis] [logstash-2014.12.02] update\_mapping
> [logs] (dynamic)[2014-12-02 12:23:39,675][INFO][cluster.metadata]
> [Mantis] [logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
> 12:51:13,975][INFO][cluster.metadata] [Mantis]
> [logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
> 12:51:26,389][INFO][cluster.metadata] [Mantis]
> [logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
> 12:55:18,142][INFO][cluster.metadata] [Mantis]
> [logstash-2014.12.02] update\_mapping [logs] (dynamic)[2014-12-02
> 12:55:31,564][INFO][cluster.metadata] [Mantis]
> [logstash-2014.12.02] update\_mapping [logs] (dynamic)Write failed: Broken
> pipe_
> I guess the connection is lost bw logstash and ES.
>
> I am using ELK first time with such amount data .
>
> Any guidance??
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/25189c47-a9c4-4933-a29e-9599bec454d9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/25189c47-a9c4-4933-a29e-9599bec454d9%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:46am UTC](https://discuss.elastic.co/t/es-load-testing/21028/3 "2017-07-06T00:46:38Z")