error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch

Elastic Stack Elasticsearch
1

I am setting up EFK stack using docker-compose. Fluentd is able to receive the logs from http-endpoint.

But the logs are getting rejected by elasticsearch.
Though I am able to receive the error as bad argument and rejected, parsed logs are able to view.
But after certain period of time elasticsearch stops accepting the logs completely. No logs are shown in the kibana dashboard.

The below error is thrown

#0 dump an error event: error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch [error type]: mapper_parsing_exception [reason]: 'failed to parse'" location=nil tag="http-tester.log" time=2022-12-01 06:49:21.498341864

The elasticsearch and kibana version - 8.5.0

My configuration in fluent.conf file for elasticsearch and source alone

<source>
  @type http
  format json
  @log_level debug
  port 9880
  bind 0.0.0.0
  body_size_limit 32m
  keepalive_timeout 10s
</source>
  
 <store>
     @type elasticsearch
     @log_level debug
     host elasticsearch
     port 9200
     enable_ilm true
     index_name fluentd
     rollover_index true
     include_timestamp true
     include_tag_key true
     tag_key @log_name
     scheme https
     ssl_verify true
     ca_file /usr/share/fluentd/certs/ca/ca.crt
     client_cert /usr/share/fluentd/certs/fluentd/fluentd.crt
     client_key /usr/share/fluentd/certs/fluentd/fluentd.key
     user elastic
     password xxxxx
     log_es_400_reason true
     reconnect_on_error true
     reload_on_failure true
     flatten_hashes true
     flatten_hashes_separator .
     <buffer tag>
       @type memory
       flush_mode interval
       flush_interval 5m
       retry_type exponential_backoff
       retry_forever true
     </buffer>
   </store>

Kindly help in solving this error

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.