Warn on testing the conf of logstash

Elastic Stack Logstash
1

Hi,
Please why do I have this warn ?

[root@frghcslnetv10 conf.d]# /usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/parsing-log.conf
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
[2018-07-10T15:08:50,216][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2018-07-10T15:08:59,058][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
2

It's just saying that it's ignoring your pipelines.yml file because you're passing -f /etc/logstash/conf.d/parsing-log.conf.

3

i am blocked I don't know what's going on :

[root@frghcslnetv10 logstash]# tail -25 logstash-plain.log
[2018-07-10T15:44:49,642][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>33, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.16-java/lib/logstash/inputs/beats.rb:198:in `run'"}], ["LogStash::Filters::Fingerprint", {"method"=>"SHA1", "key"=>"KEY", "id"=>"7ff28d71c023feebb4fea4184c7e4ebf3c316e1edf1036d5858aeb41e05e04f1"}]=>[{"thread_id"=>28, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in `read_batch'"}, {"thread_id"=>29, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in `read_batch'"}, {"thread_id"=>30, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in `read_batch'"}, {"thread_id"=>31, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in `read_batch'"}]}}
[2018-07-10T15:44:49,678][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2018-07-10T15:44:52,697][INFO ][logstash.pipeline        ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x784c274 run>"}
[2018-07-10T15:45:34,640][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.1"}
[2018-07-10T15:45:45,379][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-07-10T15:45:46,134][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-07-10T15:45:46,150][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-07-10T15:45:46,487][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-07-10T15:45:46,594][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-07-10T15:45:46,602][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-07-10T15:45:46,657][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-07-10T15:45:46,693][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-07-10T15:45:46,759][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-07-10T15:45:48,791][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2018-07-10T15:45:48,921][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x19253a9c run>"}
[2018-07-10T15:45:49,082][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2018-07-10T15:45:49,125][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-07-10T15:45:49,653][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-07-10T15:46:37,939][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"7c6ca8acf5153e75d558cc4dda814162830c6d16", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x1f5d03b4>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"7c6ca8acf5153e75d558cc4dda814162830c6d16", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}
[2018-07-10T15:47:02,572][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"b9e05983b064a0f2ea99648043e881a6d6cba1ea", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x275e3fde>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"b9e05983b064a0f2ea99648043e881a6d6cba1ea", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-release' is not an IP string literal."}}}}}
4 
[2018-07-10T15:47:02,576][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"6b5dc13a0de5c7d48b13a639f94ca3261cfcd7c5", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x5d2ac6ce>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"6b5dc13a0de5c7d48b13a639f94ca3261cfcd7c5", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}
[2018-07-10T15:49:21,054][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"bfdbcff8f0c40086e55e2a7c7ce1d653fa68b731", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x709a6947>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"bfdbcff8f0c40086e55e2a7c7ce1d653fa68b731", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}
[2018-07-10T15:49:45,749][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"b23ef6f4364770f9adabd34224a24bf2394d82ab", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x453cd198>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"b23ef6f4364770f9adabd34224a24bf2394d82ab", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}
[2018-07-10T15:49:59,773][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"4f72addea152b5ed890c6b29a5ad6147177a0e95", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x188004b4>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", "_id"=>"4f72addea152b5ed890c6b29a5ad6147177a0e95", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-release' is not an IP string literal."}}}}}
[2018-07-10T15:49:59,775][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"2e98031012946a11b4fd11029727772ebb625ab4", :_index=>"logstash-2018.07.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x6d942141>], :response=>{"index"=>{"_index"=>"logstash-2018.07.10", "_type"=>"doc", 
"_id"=>"2e98031012946a11b4fd11029727772ebb625ab4", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}
5

"_id"=>"2e98031012946a11b4fd11029727772ebb625ab4", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [dst_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'globalprotectgateway-config-succ' is not an IP string literal."}}}}}

Okay, but this one should be quite clear. The dst_ip field is mapped as an IP address but you're trying to index a document where that field contains the string "globalprotectgateway-config-succ".

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.