Hi,
I am loading a csv file data into elasticsearch using Logstash. I have installed and configured all the necessary details in the Linux server. when I tried to run the config file to load the data, I am receiving the below warning and I couldn't find the index created in elasticsearch. Also, I did receive any error specific to data failure. please help me here to fix the issue.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2018-12-25 15:02:56.303 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-12-25 15:02:56.368 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.5.3"}
[INFO ] 2018-12-25 15:03:00.741 [Converge PipelineAction::Create<main>] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-12-25 15:03:01.344 [[main]-pipeline-manager] elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://adsuedscaispk04.aipcore.local:9200/]}}
[WARN ] 2018-12-25 15:03:01.634 [[main]-pipeline-manager] elasticsearch - Restored connection to ES instance {:url=>"http://adsuedscaispk04.aipcore.local:9200/"}
[INFO ] 2018-12-25 15:03:01.901 [[main]-pipeline-manager] elasticsearch - ES Output version determined {:es_version=>6}
[WARN ] 2018-12-25 15:03:01.906 [[main]-pipeline-manager] elasticsearch - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[INFO ] 2018-12-25 15:03:01.937 [[main]-pipeline-manager] elasticsearch - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//adsuedscaispk04.aipcore.local:9200"]}
[INFO ] 2018-12-25 15:03:01.961 [Ruby-0-Thread-5: :1] elasticsearch - Using mapping template from {:path=>nil}
[INFO ] 2018-12-25 15:03:01.990 [Ruby-0-Thread-5: :1] elasticsearch - Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[INFO ] 2018-12-25 15:03:02.234 [Converge PipelineAction::Create<main>] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2809872 run>"}
[INFO ] 2018-12-25 15:03:02.290 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2018-12-25 15:03:02.317 [[main]<file] observingtail - START, creating Discoverer, Watch with file and sincedb collections
[INFO ] 2018-12-25 15:03:02.664 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[WARN ] 2018-12-25 15:03:03.445 [Ruby-0-Thread-6: :1] elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"city", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x6139ff42>], :response=>{"index"=>{"_index"=>"city", "_type"=>"doc", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security*,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*]) doesn't match", "index_uuid"=>"_na_", "index"=>"city"}}}}
[WARN ] 2018-12-25 15:03:03.445 [Ruby-0-Thread-10: :1] elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"city", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x370a3f24>], :response=>{"index"=>{"_index"=>"city", "_type"=>"doc", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security*,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*]) doesn't match", "index_uuid"=>"_na_", "index"=>"city"}}}}
[WARN ] 2018-12-25 15:03:03.447 [Ruby-0-Thread-8: :1] elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"city", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x1e46702a>], :response=>{"index"=>{"_index"=>"city", "_type"=>"doc", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security*,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*]) doesn't match", "index_uuid"=>"_na_", "index"=>"city"}}}}
[WARN ] 2018-12-25 15:03:03.445 [Ruby-0-Thread-13: :1] elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"city", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x761caeb2>], :response=>{"index"=>{"_index"=>"city", "_type"=>"doc", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security*,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*]) doesn't match", "index_uuid"=>"_na_", "index"=>"city"}}}}
[WARN ] 2018-12-25 15:03:03.446 [Ruby-0-Thread-9: :1] elasticsearch - Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>nil, :_index=>"city", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x7fd9253e>], :response=>{"index"=>{"_index"=>"city", "_type"=>"doc", "_id"=>nil, "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index and [action.auto_create_index] ([.security*,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*]) doesn't match", "index_uuid"=>"_na_", "index"=>"city"}}}}