Error management in elasticsearch output plugin

Elastic Stack Logstash
1

Hello everyone
Hello to all of you
I have a problem with logstash and elasticsearch
I have this configuration file for logstash

input {
	file {
		path => "${FILE_TO_SEND}"
		sincedb_path => "/dev/null"
		mode => read
		exit_after_read => true
		file_completed_action => log
		file_completed_log_path => "/dev/null"
	}
}

filter {
    json {
        source => "message"
		remove_field => ["@version","host","log","event","message"]
	}
}

output {
	elasticsearch {
		user => "${ELASTIC_USER}"
		password => "${ELASTIC_PASS}"
		hosts => "${ELASTIC_HOST}"
		index => "${INDEX}"
		ssl_certificate_verification => false
	}
}

But when I send the data to elasticsearch and the connection fails, I get this message in the log file:

[WARN ] 2024-01-15 16:26:25.644 [[main]-pipeline-manager] elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elastic:xxxxxx@host.docker.internal:9201/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://host.docker.internal:9201/][Manticore::SocketException] Connect to host.docker.internal:9201 [host.docker.internal/192.168.5.2] failed: Connection refused"}
[INFO ] 2024-01-15 16:26:25.775 [[main]-pipeline-manager] elasticsearch - Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=>"testlogs-testlogs-2024.01.15"}

And I need it to show as an error not as a warning
I need Logstash to tell me that there is an error with the elasticsearch connection and terminate the logstash process.

2

Hi,

connection issues are logged as warnings, not errors, because that's how they're classified in the Logstash code.

If you want to treat these warnings as errors, you would need to implement a custom solution.

Regards

3

Hello, thanks for your answer
Can you help me with that, there are something in the documentation about how to treat the warnings as error in a custom solution?

4

This is unrelated to Logstash, this kind of error is always logged as WARN, if you need to stop Logstash when this happen you need to use a third-party tool or write some script to check the logs and stop the service.

For example, you could have a bash script that checks the log for this error and stop the service.

5

Ok thaks a lot!!
Regards

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.