# Logstash backfilling two type of logs, problem!

**URL:** <https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646>\
**Category:** Logstash\
**Created:** [February 1, 2016, 3:52pm UTC](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646 "2016-02-01T15:52:07Z")\
**Posts on this page:** 4\
**Page:** 3

<div class="post-metadata">

**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\
**Post date:** [February 19, 2016, 9:32pm UTC](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646/45 "2016-02-19T21:32:20Z")

</div>

solved it ...

needed to use UNIX not UNIX\_MS

the indexing looks a bit slow ...

still 4cpu 8gig of ram :\ ~ 2000log seconds... a bit low :\

---

<div class="post-metadata">

**Author:** ![anhlqn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anhlqn/32/5454_2.png) [@anhlqn](https://discuss.elastic.co/u/anhlqn)\
**Post date:** [February 20, 2016, 5:19am UTC](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646/46 "2016-02-20T05:19:12Z")

</div>

Regarding indexing rate. I just did a perf test using Logstash to receive netflow data:

- 1xCPU 6 physical cores or 12 logical cores, 16GB RAM.
- The server is dedicated for netflow generator and Logstash only, ES is on another server.
- 1Gbps network between 2 servers.
- 20K flow/sec generated

Flows sent from Logstash/received by Elasticsearch

No filter at all: 15K flows/sec  
GeoIP filter for 2 IP fields: 7K flows/sec  
GeoIP filter + logstash-cidr-filter to tag network based on IP + some if statements: 2.5K flows/sec

I guess playing with Filter Worker may help [https://www.elastic.co/guide/en/logstash/2.2/breaking-changes.html#\_filter\_worker\_default\_change](https://www.elastic.co/guide/en/logstash/2.2/breaking-changes.html#_filter_worker_default_change)

---

<div class="post-metadata">

**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\
**Post date:** [February 20, 2016, 5:34am UTC](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646/47 "2016-02-20T05:34:39Z")

</div>

I have geofilter on two field  
and its a VM on ESX server the ressources are not dedicated. but allowed.

so I assume, according to your stats.. its decent then

thank, im already running 20workers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:10am UTC](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646/48 "2017-07-06T05:10:30Z")

</div>

Closed on July 6, 2017, 5:10am UTC



[Previous page](https://discuss.elastic.co/t/logstash-backfilling-two-type-of-logs-problem/40646.md?page=2)