# Nprobe flows drops when sending to elasticsearch cluster

**URL:** <https://discuss.elastic.co/t/nprobe-flows-drops-when-sending-to-elasticsearch-cluster/98878>\
**Category:** Elasticsearch\
**Created:** [August 30, 2017, 3:30pm UTC](https://discuss.elastic.co/t/nprobe-flows-drops-when-sending-to-elasticsearch-cluster/98878 "2017-08-30T15:30:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tomasz\_Zielinski](https://avatars.discourse-cdn.com/v4/letter/t/4491bb/32.png) [@Tomasz\_Zielinski](https://discuss.elastic.co/u/Tomasz_Zielinski)\
**Post date:** [August 30, 2017, 3:30pm UTC](https://discuss.elastic.co/t/nprobe-flows-drops-when-sending-to-elasticsearch-cluster/98878/1 "2017-08-30T15:30:22Z")

</div>

Hi, i have an nprobe probe what collect a flows from network and transport it to elastic cluster.  
For testing purposes i have one elastic node with following hardware

o CPU: 8C  
o RAM: 16GB  
o HDD: 1T

This is my nprobe configuration  
nprobe -b 1 -V 5 --collector-port 2055 -i eth0 -n none --elastic "flows;nprobe2-%Y.%m.%d;[http://192.168.0.197:9200/\_bulk](http://192.168.0.197:9200/_bulk)"

When i exceeded more than 1k flow per seconds i get a  
WARNING: [ES] Export queue too long [32768]: expect drops  
and this is true, most of flows is really dropped.

When i re-direct output of nprobe to tcp port and try to receive it by netcat i see all of them nothing is missing, so it is definitely elastic search issue.

Strange is that CPU is almost all the time 80% idle, iowait is about 5% there is some RAM free also.  
I set limits in my OS to

elasticsearch soft memlock unlimited  
elasticsearch hard memlock unlimited  
and set LimitMEMLOCK=infinity  
also change  
bootstrap.memory\_lock: true  
and  
-Xms5g  
-Xmx5g

But no luck .. I was trying to change setting of thread\_pool in elasticsearch.yml to  
thread\_pool.bulk.size: 9  
thread\_pool.bulk.queue\_size: 100  
but it seems to have no effect at all

I’m using elastic search version 5.5.2, what I’m missing ? why it does not work ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 27, 2017, 3:31pm UTC](https://discuss.elastic.co/t/nprobe-flows-drops-when-sending-to-elasticsearch-cluster/98878/2 "2017-09-27T15:31:06Z")

</div>

Closed on September 27, 2017, 3:31pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.