I am Trying to use logstash oss 7.9.1 to push csv files into Open Distro Elasticsearch 1.13.2

Elastic Stack Logstash
1

Please suggest me with version combabilities, I am not sure if thats the case of my errors

When I try to run logstash i get these errors

[2021-05-30T03:34:51,112][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-05-30T03:34:52,549][INFO ][org.reflections.Reflections] Reflections took 28 ms to scan 1 urls, producing 22 keys and 45 values 
[2021-05-30T03:34:54,483][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2021-05-30T03:34:54,660][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2021-05-30T03:34:54,723][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-05-30T03:34:54,731][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2021-05-30T03:34:54,762][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
[2021-05-30T03:34:54,883][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Amazon/opendistro/logstash-7.9.1/logstash_test.conf"], :thread=>"#<Thread:0x79b07031 run>"}
[2021-05-30T03:34:54,847][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2021-05-30T03:34:54,980][INFO ][logstash.outputs.elasticsearch][main] Index Lifecycle Management is set to 'auto', but will be disabled - Index Lifecycle management is not installed on your Elasticsearch cluster
[2021-05-30T03:34:54,987][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2021-05-30T03:34:55,036][INFO ][logstash.outputs.elasticsearch][main] Installing elasticsearch template to _template/logstash
[2021-05-30T03:34:56,058][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.17}
[2021-05-30T03:34:56,500][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-05-30T03:34:56,543][INFO ][filewatch.observingtail  ][main][9cde7087ee7d6c6b69453da0340fc3ea72fb8e0ad28af77d8154224474e00a0f] START, creating Discoverer, Watch with file and sincedb collections
[2021-05-30T03:34:56,550][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-05-30T03:34:56,865][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-05-30T03:35:55,164][WARN ][logstash.outputs.elasticsearch][main] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out {:url=>http://localhost:9200/, :error_message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2021-05-30T03:35:55,164][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :backtrace=>["C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:334:in `perform_request_to_url'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:319:in `block in perform_request'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:414:in `with_connection'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:318:in `perform_request'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:326:in `block in Pool'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:352:in `template_put'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in `template_install'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/template_manager.rb:31:in `install'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/template_manager.rb:17:in `install_template'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/common.rb:212:in `install_template'", "C:/Amazon/opendistro/logstash-7.9.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.6.2-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]}
[2021-05-30T03:35:55,282][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2021-05-30T03:36:55,674][WARN ][logstash.outputs.elasticsearch][main][56b0aa42b453579833fc101715487131c2df974c561b92ab1128435b535bbb1a] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out {:url=>http://localhost:9200/, :error_message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}```
 


```input{
	file{
	path => "C:/Users/aswanth.jabba/Documents/Task_records.csv"
	start_position => "beginning"
	sincedb_path => "NULL"
	}
}
filter{
	csv{
	separator => ","
	columns => ["dbid","Name","Description","StartDate","EndDate"]
	}
}
output{
	elasticsearch{
	hosts => "http://localhost:9200"
	index => "logstashtest"
	}
	stdout{}
}```
2

We don't support opendistro as it's a fork of Elasticsearch, sorry. You will need to ask the authors directly.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.