Hi elastic community!
I really need some guidiance to speed up the starting of logstash service.
Following is logstash restart history.
[2019-12-20T12:20:12,591][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"D:/elk/logstash-6.1.1/modules/fb_apache/configuration"}
[2019-12-20T12:20:12,682][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"D:/elk/logstash-6.1.1/modules/netflow/configuration"}
[2019-12-20T12:20:13,673][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"D:\\elk\\logstash-6.1.1\\data/queue"}
[2019-12-20T12:20:13,685][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"D:\\elk\\logstash-6.1.1\\data/dead_letter_queue"}
[2019-12-20T12:20:14,685][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-12-20T12:20:14,982][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"19a53c6e-1c7f-40fb-96db-3bde86b4b3b9", :path=>"D:\\elk\\logstash-6.1.1\\data/uuid"}
[2019-12-20T12:20:20,279][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.1.1"}
[2019-12-20T12:20:33,873][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-12-20T12:39:05,158][INFO ][logstash.outputs.amazones] Automatic template management enabled {:manage_template=>"true"}
[2019-12-20T12:39:05,252][INFO ][logstash.outputs.amazones] Using mapping template {:template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "ignore_above"=>256}}}}}], "properties"=>{"@version"=>{"type"=>"string", "index"=>"not_analyzed"}, "geoip"=>{"type"=>"object", "dynamic"=>true, "properties"=>{"location"=>{"type"=>"geo_point"}}}}}}}}
[2019-12-20T12:39:08,517][INFO ][logstash.outputs.amazones] New Elasticsearch output {:hosts=>["myhost"], :port=>443}
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,361][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:16,377][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,377][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,377][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:16,377][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:17,220][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:17,220][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:17,517][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:17,517][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:17,517][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:17,517][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:19,859][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:19,859][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:19,859][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:19,924][INFO ][logstash.filters.geoip ] Using geoip database {:path=>"D:/elk/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.2-java/vendor/GeoLite2-City.mmdb"}
[2019-12-20T12:39:23,563][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 1800 seconds
[2019-12-20T12:39:23,658][INFO ][logstash.filters.elapsed ] Elapsed, timeout: 2100 seconds
[2019-12-20T12:39:23,958][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>2000, "pipeline.batch.delay"=>80, "pipeline.max_inflight"=>2000, :thread=>"#<Thread:0x65a5939c run>"}
[2019-12-20T12:39:27,658][INFO ][logstash.pipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-12-20T12:39:30,261][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
I usually restart logstash, it takes about 6 mins.
But after I add geoip code in filter part, starting speeding is too slow.
Not sure that's why. just sharing...
Running on
logstash version: 6.1.1
server ram: 4GB
server processor: 2 cores