LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError, -
I am moving this discussion over to this forum - it was suggested that it was more to do with logstash connecting to ECE 2.4.3 -
The error is that the certificate is not recognized. we are in the middle of a lab rebuild or I could have grabbed the error. I will try the ssl verification mode - I had heard of that but not yet tried it. As a side note - I do need to update the logstash yml still don't I or can I just point to the server in the logstash.conf for the output
output {
elasticsearch {
hosts => ["http://elasticsearch docker container info:port"]
username= 'elastic'
password ='xlfksfljeoiroeri.sample'
index => "linda_sample"
document_type => "Sample1_20_20"
}
stdout {codec => rubydebug }
}
We have tried every rendition of the suggestions that have been put out there by all the other folks here. Most of their suggestions are between logstash and other "ELK" solutions not ECE with dockers. We have all the logs from all the different configurations that we have tried. the most recent is the curl to make sure that all of our user/pw combos were correct. We can ping the Elasticsearch environment, we can do the standard output of a logstash implementation. But we keep getting errors. Please see the items we have tried today
| Remove the certificate/ssl | NO impact |
|---|---|
| Remove the username/password<<no_cert_sample_logs>> | No impact |
| Remove the cloud ui and id from yml file | |
| Change the IP to the FQDN – mantech-ece.cde.defende.local after validating that it pings | No impact |
| bin/logstash --config.test_and_exit -f logstash-sample2.conf --path.settings /etc/logstash |
The –path.settings is to be sure that the yml file is found – this does not try to write to the elasticsearch – it just checks the config file. It is below with all the changes that I made.
[ece_admin@dev-logstash-01 logstash]$ cat logstash-sample2.conf
Logstash configuration for creating a simple
input {
file {
path => "/home/ece_admin/Sample_CVE.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
}
output {
elasticsearch {
hosts => ["[https://c8e98726538340d892c810ed49c9ef6e.server.domain.local.ip.es.io:9243](https://c8e98726538340d892c810ed49c9ef6e.mantech-ece.cde.defende.local.ip.es.io:9243/)"]
index => "LindaSample"
}
stdout {}
}
***********************************||
|Permission Denied - permission settings? - #3 by DevT
Tried to ensure AGAIN that the logstash user had rights – reapplied the following
Sudo chown –recursive /var/logs/logstash||
|Checked our questions -
https://discuss.elastic.co/t/cert-error-connecting-standalone-vm-logstash-to-docker-ece-build/221109/4||
|Validating that the standard logstash is working -
[ece_admin@dev-logstash-01 logstash]$ bin/logstash -e 'input { stdin { } } output { stdout {} }'
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2020-03-16 13:08:23.563 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-03-16 13:08:23.573 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.6.1"}
[INFO ] 2020-03-16 13:08:25.571 [Converge PipelineAction::Create] Reflections - Reflections took 35 ms to scan 1 urls, producing 20 keys and 40 values
[WARN ] 2020-03-16 13:08:26.622 [[main]-pipeline-manager] LazyDelegatingGauge - A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization. It is recommended to log an issue to the responsible developer/development team.
[INFO ] 2020-03-16 13:08:26.625 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x6b4798d2 run>"}
[INFO ] 2020-03-16 13:08:27.613 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[INFO ] 2020-03-16 13:08:27.669 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[INFO ] 2020-03-16 13:08:27.839 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601}
hello world
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
"host" => "dev-logstash-01",
"message" => "hello world",
"@timestamp" => 2020-03-16T17:08:33.580Z,
"@version" => "1"
}||
|Tested the url connection to validate the configuration of the url within the config file - SUCCESSFUL
=curl "https://elastic:Ixxxxxxxx@c8e98726538340d892c810ed49c9ef6e.172.22.82.158.ip.es.io:9243/" --insecure
{
"name" : "instance-0000000008",
"cluster_name" : "c8e98726538340d892c810ed49c9ef6e",
"cluster_uuid" : "iNbMYx38Si2D7iafyQneUA",
"version" : {
"number" : "7.5.0",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "e9ccaed468e2fac2275a3761849cbee64b39519f",
"build_date" : "2019-11-26T01:06:52.518245Z",
"build_snapshot" : false,
"lucene_version" : "8.3.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}||
Let us know your thoughts/suggestions and any websites that we have missed