# How to setup logstash workers or batch size

**URL:** <https://discuss.elastic.co/t/how-to-setup-logstash-workers-or-batch-size/337903>\
**Category:** Logstash\
**Created:** [July 7, 2023, 3:22pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-workers-or-batch-size/337903 "2023-07-07T15:22:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lz840408](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lz840408/32/106315_2.png) [@lz840408](https://discuss.elastic.co/u/lz840408)\
**Post date:** [July 7, 2023, 3:22pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-workers-or-batch-size/337903/1 "2023-07-07T15:22:50Z")

</div>

i have es cluster 6.1.3  
i want migrate data to es 7.17.9  
i used logstash 7.17.9 to make it  
my index count has only 1200 doc

my logstash config file:

```auto
input {
  elasticsearch {
    hosts => ["10.251.0.11:39202","10.251.0.12:39202","10.251.0.13:39202","10.251.0.14:39202","10.251.0.15:39202"]
    index => "new_index_001"
    query => '{ "query": { "match_all": {} } }'
    docinfo => true
  }
}

filter {
}

output {
  elasticsearch {
    hosts => [["10.251.0.11:39201","10.251.0.12:39201","10.251.0.13:39201","10.251.0.14:39201","10.251.0.15:39201"]]
    index => "new_index_001_v7"
  }
}

```

startup cmd:  
../bin/logstash -f es6\_to\_es7.conf -b 20000 -w 500

but when i was startup logstash, it's shutdown immediate and data only sync little

```auto
error:
[2023-07-07T23:18:12,280][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>500, "pipeline.batch.size"=>16000, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>8000000, "pipeline.sources"=>["/home/kafka/logstash-7.17.9/config/es6_to_es7.conf"], :thread=>"#<Thread:0x264ebb94 run>"}
[2023-07-07T23:18:12,283][WARN][logstash.javapipeline][main] CAUTION: Recommended inflight events max exceeded! Logstash will run with up to 8000000 events in memory in your current configuration. If your message sizes are large this may cause instability with the default heap size. Please consider setting a non-standard heap size, changing the batch size (currently 16000), or changing the number of pipeline workers (currently 500) {:pipeline_id=>"main", :thread=>"#<Thread:0x264ebb94 run>"}
[2023-07-07T23:18:18,209][INFO][logstash.javapipeline][main] Pipeline Java execution initialization time {"seconds"=>5.92}
[2023-07-07T23:18:18,778][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2023-07-07T23:18:18,904][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2023-07-07T23:18:22,021][INFO][logstash.javapipeline][main] Pipeline terminated {"pipeline.id"=>"main"}
[2023-07-07T23:18:22,150][INFO][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:main}
[2023-07-07T23:18:22,227][INFO][logstash.runner] Logstash shut down.

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2023, 3:23pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-workers-or-batch-size/337903/2 "2023-08-04T15:23:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.