Filebeat warning log "Cannot index event"

Elastic Stack Elasticsearch
1

When sending logs from Filebeat to Elasticsearch, when there is an indexing error, Filebeat will log the message

Cannot index event (status=400): dropping event! Look at the event log to view the event and cause."

The log doesn't show "reason" or "caused_by".

When I use Elastic Agent, this error shows more details, including "reason" and "caused_by".

Is there a way to achieve the same with Filebeat?

2

Hi @rodolk Welcome to the community!

You can turn on logging debug and see.... Are you sure there isn't a following line with the mapping conflict, perhaps that is now logged at debug level

It is often a mapping issue ... The document being written has a mapping conflict.

What version are you on?

3

Hi @stephenb

I am also getting a similar issue, i.e., "Failed to index 4 events in last 10s: events were dropped! Look at the event log of filebeat to view the event and cause."

Set logging.level: debug in filebeat, but still unable to understand what the issue is.

Filebeat version: 8.17.2
Elastic version: v8.17.4

{"log.level":"debug","@timestamp":"2025-05-29T11:41:55.611Z","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":301},"message":"doBulkRequest: 4 events have been sent to elasticsearch in 840.569459ms.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-05-29T11:41:55.614Z","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.(*ackLoop).handleBatchSig","file.name":"memqueue/ackloop.go","file.line":80},"message":"ackloop: return ack to broker loop:4","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-05-29T11:41:55.615Z","log.logger":"publisher","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.(*ackLoop).handleBatchSig","file.name":"memqueue/ackloop.go","file.line":82},"message":"ackloop:  done send ack","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-05-29T11:41:58.776Z","log.logger":"input.filestream","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/filestream.(*logFile).Read","file.name":"filestream/filestream.go","file.line":131},"message":"End of file reached: /var/log/app/app.log; Backoff now.","service.name":"filebeat","id":"app-logs","source_file":"filestream::app-logs::native::3623436-60","path":"/var/log/app/app.log","state-id":"native::3623436-60","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2025-05-29T11:42:00.652Z","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":175},"message":"Failed to index 4 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"}
4

Have you checked ES CPU utilization?

5

Hello @Mukul_Srivastava1
As with the filebeat logs we are not able to find the cause, maybe for same time we can check the Elasticsearch logs as well to see why the events were not indexed?
Also as Rios suggested maybe check the ES CPU as well.
Thanks!!

6

Have you looked at the event log? If so, what does it say?

7

As mentioned you need to look a the event log file, it is not the same log as the filebeat log.

It will probably be inside /var/lib/filebeat on some subfolder, not sure exactly which one.

Probably the events were dropped because of mapping conflicts, this is the most common cause for events to be dropped.

1 Like
8

I meet the same log on version 8.17.2 when I use filebeat to collect json log and set the json.overwrite_keys to true

{"log.level":"warn","@timestamp":"2025-09-23T16:16:20.671+0800","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":175},"message":"Failed to index 1 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"}