# Filebeat: connection to Elasticsearch OK but no documents indexed **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [September 29, 2025, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285 "2025-09-29T14:03:03Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\ **Post date:** [September 29, 2025, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/1 "2025-09-29T14:03:03Z")
Hi everyone, I’m facing an issue with **Filebeat 8.19.3** on Debian 12 (Bookworm). ### Symptoms - Filebeat starts without errors and connects successfully to Elasticsearch: - No `Failed to index event` or `status code` errors appear in the logs. - However, in Kibana I can’t see any documents in the `filebeat-*` or `logs-*` data streams. ### Environment - OS: Debian GNU/Linux 12 (bookworm), kernel 6.1.0-38-amd64 - Filebeat: 8.19.3 (installed from official Elastic package) - Output: direct to Elasticsearch 8.19.3 over HTTPS - Minimal configuration: enabled `system (auth)` and `system (syslog)` modules. Configured paths: ### Filebeat configuration file ```auto filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false setup.template.settings: index.number_of_shards: 1 setup.kibana: host: localhost:5601 space.id: output.elasticsearch: hosts: ["https://:9200"] preset: balanced protocol: "https" api_key: "" username: "filebeat_setup" password: "" output.logstash: enabled: false processors: - add_host_metadata: when.not.contains.tags: forwarded - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ logging.level: debug logging.selectors: ["elasticsearch"] setup.ilm.check_exists: true ``` ### Log excerpt ```plaintext {"log.level":"info","message":"filebeat start running.","service.name":"filebeat"} {"log.level":"info","message":"Enabled modules/filesets: system (auth), system (syslog)","service.name":"filebeat"} {"log.level":"info","message":"Configured paths: [/var/log/auth.log*]","service.name":"filebeat"} {"log.level":"info","message":"Configured paths: [/var/log/syslog*]","service.name":"filebeat"} {"log.level":"info","message":"Connection to backoff(elasticsearch(https://:9200)) established","service.name":"filebeat"} {"log.level":"debug","message":"doBulkRequest: 1600 events have been sent to elasticsearch in 69ms.","service.name":"filebeat"} ``` ### What I’ve already checked - Filebeat communicates with Elasticsearch without errors. - `/var/log/syslog` and `/var/log/auth.log` contain new events. - In Kibana → Data view `filebeat-*` shows no documents. - ILM is enabled (`lifecycle policy filebeat exists already.`) and the `filebeat-8.19.3` template exists. ### Question What am I missing? If bulk requests are being successfully sent to Elasticsearch, why can’t I find any documents in the `filebeat-*` data streams? Thanks a lot in advance! ---
**Author:** ![AndersonQ](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andersonq/32/112214_2.png) [@AndersonQ](https://discuss.elastic.co/u/AndersonQ)\ **Post date:** [September 30, 2025, 7:29am UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/2 "2025-09-30T07:29:50Z")
Hi @decibel83 , would you have the [30s metrics log to share](https://www.elastic.co/docs/reference/beats/filebeat/understand-filebeat-logs)? or the query the [monitoring `/inputs` endpoint](https://www.elastic.co/docs/reference/beats/filebeat/http-endpoint) and share the result. It might explain what is happening with the events. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 30, 2025, 3:26pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/3 "2025-09-30T15:26:20Z")
> [@decibel83](#): > > However, in Kibana I can’t see any documents in the `filebeat-*` or `logs-*` data streams. Do you have other data streams? Like `logs-system.*` ? Your filebeat shows that events were sent to Elasticsearch. I'm not sure it will write into `filebeat-*` or into `logs-system.*` ---
**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\ **Post date:** [October 2, 2025, 12:55pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/4 "2025-10-02T12:55:29Z")
> [@leandrojmp](#): > > Do you have other data streams? Like `logs-system.*` ? Yes, but `logs-*` is also empty: ![Screenshot 2025-10-02 at 14.54.13](https://us1.discourse-cdn.com/elastic/original/3X/1/2/12a7b6e33d25a4a80b7e794ee27d87586c56272d.png) ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [October 2, 2025, 12:58pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/5 "2025-10-02T12:58:57Z")
What is the time frame? This is a data view (kibana feature to show data streams), to see the data streams you need to go into Stack Management \> Index Management \> Data Streams or run `GET _cat/indices?v` on Kibana Dev Tools. Can you run `GET _cat/indices?v` on Kibana Dev Tools and share the result? If the connection is ok and the modules are enabled then I would not expect any issue, something may not be configured correctly or working correctly. ---
**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\ **Post date:** [October 2, 2025, 1:03pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/6 "2025-10-02T13:03:21Z")
> [@AndersonQ](#): > > would you have the [30s metrics log to share](https://www.elastic.co/docs/reference/beats/filebeat/understand-filebeat-logs)? This is another log excerpt from today: ```auto Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.405+0200","log.logger":"publisher_pipeline_output","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run","file.name":"pipeline/client_worker.go","file.line":138},"message":"Connecting to backoff(elasticsearch(https://elk.my.domain:9200))","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.409+0200","log.logger":"elasticsearch.esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.(*Connection).Ping","file.name":"eslegclient/connection.go","file.line":324},"message":"Attempting to connect to Elasticsearch version 8.19.3 (default)","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.420+0200","log.logger":"index-management","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/idxmgmt.(*indexManager).Setup","file.name":"idxmgmt/index_support.go","file.line":254},"message":"Auto lifecycle enable success.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.422+0200","log.logger":"index-management.ilm","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/idxmgmt/lifecycle.(*stdManager).EnsurePolicy","file.name":"lifecycle/standard_manager.go","file.line":111},"message":"lifecycle policy filebeat exists already.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.422+0200","log.logger":"index-management","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/idxmgmt.applyLifecycleSettingsToTemplate","file.name":"idxmgmt/index_support.go","file.line":402},"message":"Set settings.index.lifecycle.name in template to filebeat as ILM is enabled.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.431+0200","log.logger":"template_loader","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/template.(*ESLoader).Load","file.name":"template/load.go","file.line":121},"message":"Template \"filebeat-8.19.3\" already exists and will not be overwritten.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.431+0200","log.logger":"index-management","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/idxmgmt.(*indexManager).Setup","file.name":"idxmgmt/index_support.go","file.line":299},"message":"Loaded index template.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:57:57.432+0200","log.logger":"publisher_pipeline_output","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run","file.name":"pipeline/client_worker.go","file.line":146},"message":"Connection to backoff(elasticsearch(https://elk.my.domain:9200)) established","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:57:57 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:57:57.438+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 111 events have been sent to elasticsearch in 5.917839ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:07 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:07.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 111 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:07 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:58:07.405+0200","log.logger":"input.harvester","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/log.(*Harvester).Run","file.name":"log/harvester.go","file.line":313},"message":"Harvester started for paths: [/var/log/auth.log*]","service.name":"filebeat","input_id":"3802b47a-4011-43fc-a425-a5cd4eaabe88","source_file":"/var/log/auth.log","state_id":"native::96-64773","finished":false,"os_id":"96-64773","old_source":"/var/log/auth.log","old_finished":true,"old_os_id":"96-64773","harvester_id":"941c8ba5-915c-446b-8c69-75b9e189e9a4","ecs.version":"1.6.0"} Oct 2 14:58:07 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:58:07.417+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 64 events have been sent to elasticsearch in 7.92079ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:17 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:17.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 64 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:17 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:58:17.305+0200","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpu":{"id":"filebeat.service"},"memory":{"id":"filebeat.service","mem":{"usage":{"bytes":183050240}}}},"cpu":{"system":{"ticks":50,"time":{"ms":50}},"total":{"ticks":190,"time":{"ms":190},"value":190},"user":{"ticks":140,"time":{"ms":140}}},"handles":{"limit":{"hard":524288,"soft":524287},"open":15},"info":{"ephemeral_id":"7d2a3b4f-6851-45fd-a877-ab585974d991","name":"filebeat","uptime":{"ms":33057},"version":"8.19.3"},"memstats":{"gc_next":47130290,"memory_alloc":35617992,"memory_sys":57169160,"memory_total":69264920,"rss":157614080},"runtime":{"goroutines":60}},"filebeat":{"events":{"active":32,"added":213,"done":181},"harvester":{"open_files":2,"running":2,"started":2}},"libbeat":{"config":{"module":{"running":1,"starts":1},"reloads":1,"scans":1},"output":{"events":{"active":0,"batches":2,"dropped":175,"total":175},"read":{"bytes":9506,"errors":2},"type":"elasticsearch","write":{"bytes":31897,"latency":{"histogram":{"count":2,"max":7,"mean":6,"median":6,"min":5,"p75":7,"p95":7,"p99":7,"p999":7,"stddev":1}}}},"pipeline":{"clients":2,"events":{"active":32,"filtered":6,"published":207,"total":213},"queue":{"acked":175,"added":{"bytes":284208,"events":207},"consumed":{"bytes":242974,"events":175},"filled":{"bytes":41234,"events":32,"pct":0.01},"max_bytes":0,"max_events":3200,"removed":{"bytes":242974,"events":175}}}},"registrar":{"states":{"current":4,"update":181},"writes":{"success":3,"total":3}},"system":{"cpu":{"cores":4},"load":{"1":5.62,"15":4.3,"5":4.72,"norm":{"1":1.405,"15":1.075,"5":1.18}}}},"ecs.version":"1.6.0"}} Oct 2 14:58:18 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:58:18.421+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 62 events have been sent to elasticsearch in 6.589463ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:27 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:27.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 62 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:30 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:58:30.426+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 58 events have been sent to elasticsearch in 6.599465ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:37 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:37.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 58 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:42 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:58:42.430+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 56 events have been sent to elasticsearch in 6.622743ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:47 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:47.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 56 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:47 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T14:58:47.306+0200","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":183586816}}}},"cpu":{"system":{"ticks":60,"time":{"ms":10}},"total":{"ticks":220,"time":{"ms":30},"value":220},"user":{"ticks":160,"time":{"ms":20}}},"handles":{"limit":{"hard":524288,"soft":524287},"open":13},"info":{"ephemeral_id":"7d2a3b4f-6851-45fd-a877-ab585974d991","uptime":{"ms":63057},"version":"8.19.3"},"memstats":{"gc_next":47130290,"memory_alloc":39530616,"memory_total":73177544,"rss":158572544},"runtime":{"goroutines":56}},"filebeat":{"events":{"active":1,"added":145,"done":176},"harvester":{"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0,"batches":3,"dropped":176,"total":176},"read":{"bytes":1847,"errors":3},"write":{"bytes":28389,"latency":{"histogram":{"count":5,"max":7,"mean":6,"median":6,"min":5,"p75":6.5,"p95":7,"p99":7,"p999":7,"stddev":0.6324555320336759}}}},"pipeline":{"clients":2,"events":{"active":1,"published":145,"total":145},"queue":{"acked":176,"added":{"bytes":188362,"events":145},"consumed":{"bytes":228442,"events":176},"filled":{"bytes":1154,"events":1,"pct":0.0003125},"max_bytes":0,"max_events":3200,"removed":{"bytes":228442,"events":176}}}},"registrar":{"states":{"current":4,"update":176},"writes":{"success":3,"total":3}},"system":{"load":{"1":5.57,"15":4.34,"5":4.8,"norm":{"1":1.3925,"15":1.085,"5":1.2}}}},"ecs.version":"1.6.0"}} Oct 2 14:58:54 elk filebeat[313502]: {"log.level":"debug","@timestamp":"2025-10-02T14:58:54.437+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).doBulkRequest","file.name":"elasticsearch/client.go","file.line":317},"message":"doBulkRequest: 31 events have been sent to elasticsearch in 6.34368ms.","service.name":"filebeat","ecs.version":"1.6.0"} Oct 2 14:58:57 elk filebeat[313502]: {"log.level":"warn","@timestamp":"2025-10-02T14:58:57.302+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.NewClient.func3","file.name":"elasticsearch/client.go","file.line":179},"message":"Failed to index 31 events in last 10s: events were dropped! Look at the event log to view the event and cause.","service.name":"filebeat","ecs.version":"1.6.0"} ``` Below you can find the 30s metrics log: ```auto Oct 2 15:00:17 elk filebeat[313502]: {"log.level":"info","@timestamp":"2025-10-02T15:00:17.305+0200","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).logSnapshot","file.name":"log/log.go","file.line":192},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"memory":{"mem":{"usage":{"bytes":184909824}}}},"cpu":{"system":{"ticks":80,"time":{"ms":10}},"total":{"ticks":310,"time":{"ms":30},"value":310},"user":{"ticks":230,"time":{"ms":20}}},"handles":{"limit":{"hard":524288,"soft":524287},"open":13},"info":{"ephemeral_id":"7d2a3b4f-6851-45fd-a877-ab585974d991","uptime":{"ms":153057},"version":"8.19.3"},"memstats":{"gc_next":47739730,"memory_alloc":30341296,"memory_sys":262144,"memory_total":83799728,"rss":158572544},"runtime":{"goroutines":56}},"filebeat":{"events":{"active":29,"added":119,"done":146},"harvester":{"open_files":2,"running":2}},"libbeat":{"config":{"module":{"running":1}},"output":{"events":{"active":0,"batches":3,"dropped":146,"total":146},"read":{"bytes":1748,"errors":3},"write":{"bytes":23786,"latency":{"histogram":{"count":13,"max":7,"mean":5.923076923076923,"median":6,"min":5,"p75":6,"p95":7,"p99":7,"p999":7,"stddev":0.6153846153846154}}}},"pipeline":{"clients":2,"events":{"active":29,"published":119,"total":119},"queue":{"acked":146,"added":{"bytes":155555,"events":119},"consumed":{"bytes":189579,"events":146},"filled":{"bytes":37131,"events":29,"pct":0.0090625},"max_bytes":0,"max_events":3200,"removed":{"bytes":189579,"events":146}}}},"registrar":{"states":{"current":4,"update":146},"writes":{"success":3,"total":3}},"system":{"load":{"1":4.07,"15":4.29,"5":4.53,"norm":{"1":1.0175,"15":1.0725,"5":1.1325}}}},"ecs.version":"1.6.0"}} ``` > [@AndersonQ](#): > > or the query the [monitoring `/inputs` endpoint](https://www.elastic.co/docs/reference/beats/filebeat/http-endpoint) and share the result. This is the `/inputs` endpoint result: ```auto root@elk /e/filebeat# http 127.0.0.1:5066/inputs/ HTTP/1.1 200 OK Content-Encoding: gzip Content-Length: 27 Content-Type: application/json; charset=utf-8 Date: Thu, 02 Oct 2025 13:01:25 GMT Vary: Accept-Encoding [] ``` In fact, I don’t have any input enabled in the Filebeat configuration, but I have the system module enabled: ```auto root@elk /e/filebeat# cat /etc/filebeat/modules.d/system.yml # Ansible managed # Filebeat default configuration for module system on Linux - module: system syslog: enabled: true var.paths: ["/var/log/syslog*"] auth: enabled: true var.paths: ["/var/log/auth.log*"] ``` Thank you! ---
**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\ **Post date:** [October 2, 2025, 1:18pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/7 "2025-10-02T13:18:11Z")
> [@leandrojmp](#): > > Can you run `GET _cat/indices?v` on Kibana Dev Tools and share the result? This is the result: ```auto health status index uuid pri rep docs.count docs.deleted store.size pri.store.size dataset.size green open .internal.alerts-transform.health.alerts-default-000001 sWH16blfRN2nYfqwcisluQ 1 0 0 0 249b 249b 249b green open .internal.alerts-ml.anomaly-detection.alerts-default-000001 yqwbR1hOSlaWTYIvYAsaHw 1 0 0 0 249b 249b 249b green open .ds-metricbeat-8.18.0-2025.08.18-000004 xJOH4qg1SQanDaVO35gaBg 1 0 62119534 0 22.9gb 22.9gb 22.9gb green open .internal.alerts-observability.slo.alerts-default-000001 KIPmb7dbSqKAsfbOyZ6Ehg 1 0 0 0 249b 249b 249b green open .internal.alerts-default.alerts-default-000001 d3f-IGrsQ_GrRdYJC2VBKA 1 0 0 0 249b 249b 249b green open .internal.alerts-streams.alerts-default-000001 jT1Ea5x1R9Sn10gGnfKK9g 1 0 0 0 249b 249b 249b green open .internal.alerts-observability.apm.alerts-default-000001 p5OF5C9KQbCkPwftph8sPg 1 0 0 0 249b 249b 249b green open .ds-metricbeat-8.14.1-2025.08.18-000008 wV4Kw2d4SbiZrI9r6AP6hA 1 0 42728873 0 10gb 10gb 10gb yellow open .ds-metricbeat-8.14.1-2025.09.17-000012 7V-K60ZURmqwnQtl2hnGDg 1 1 24664721 0 5.9gb 5.9gb 5.9gb green open .internal.alerts-security.attack.discovery.alerts-default-000001 91_6LkPTSlW-qO0XYPjBUQ 1 0 0 0 249b 249b 249b yellow open .ds-metricbeat-8.19.3-2025.09.28-000002 Iv5SfpyDR9Oz0tQO87jnQw 1 1 11845711 0 5.1gb 5.1gb 5.1gb green open .internal.alerts-observability.metrics.alerts-default-000001 zshIHHb6T6ibnDbV70Y_cA 1 0 0 0 249b 249b 249b yellow open .ds-metricbeat-8.18.0-2025.09.17-000008 2ydPcUJITi-Hn2OEwtQKSw 1 1 41806453 0 15.5gb 15.5gb 15.5gb green open .ds-metricbeat-8.18.3-2025.08.18-000002 3Dgqb6hRSbizlgcFgwgT8w 1 0 17527949 0 5gb 5gb 5gb green open .internal.alerts-ml.anomaly-detection-health.alerts-default-000001 mb5AWlCSS1iVMtLcWSyMbQ 1 0 0 0 249b 249b 249b green open .internal.alerts-security.alerts-default-000001 f5_z3qb7RjuNmD-OHijFlA 1 0 0 0 249b 249b 249b green open .internal.alerts-stack.alerts-default-000001 gXTKuJVbQhmh32_BcjER4Q 1 0 0 0 249b 249b 249b yellow open .ds-metricbeat-8.17.0-2025.09.26-000038 yXtn5nVXTROnqyncQ-SOtg 1 1 55503141 0 25.6gb 25.6gb 25.6gb green open .ds-metricbeat-8.18.1-2025.08.18-000003 voi4enygTbKiGsy3sgJkyw 1 0 0 0 249b 249b 249b green open .ds-metricbeat-8.17.0-2025.08.29-000029 l60q8TFnS4erJJjhaQ1f6g 1 0 107333766 0 50gb 50gb 50gb green open .internal.alerts-observability.logs.alerts-default-000001 YdUuPea5S1qeP1sxCvUupw 1 0 0 0 249b 249b 249b green open .ds-metricbeat-8.19.2-2025.08.18-000001 5x2iAg1eSX-v_D9_AuzLTw 1 0 43755886 0 19.2gb 19.2gb 19.2gb green open .internal.alerts-observability.uptime.alerts-default-000001 MOHMxRdDSVCdDRsZ_H31iQ 1 0 0 0 249b 249b 249b green open .ds-metricbeat-8.17.0-2025.09.10-000030 GyfK_Gs_RFGLdRjgDffRdQ 1 0 107847695 0 49.9gb 49.9gb 49.9gb green open .ds-metricbeat-8.17.0-2025.08.18-000028 UcJW2ZkSQYK__zHWxeg6rA 1 0 105907946 0 50gb 50gb 50gb yellow open .ds-metricbeat-8.18.3-2025.09.17-000004 iIVbqWZdTviivCewUB46CA 1 1 10315184 0 2.9gb 2.9gb 2.9gb green open .kibana-observability-ai-assistant-conversations-000001 eB35q-YiTLerLR4Sav3oYQ 1 0 0 0 249b 249b 249b yellow open .ds-winlogbeat-8.14.1-2025.09.18-000001 YPxzJRDDQButZhYRUdV4Tw 1 1 36433713 0 18.3gb 18.3gb 18.3gb green open .ds-metricbeat-8.19.3-2025.08.29-000001 bBdU7qCYQVqF-Cdz0uhscA 1 0 48895428 0 21gb 21gb 21gb green open .internal.alerts-observability.threshold.alerts-default-000001 sb2k-X2DQ5W7TtLsZ2TGYw 1 0 0 0 249b 249b 249b yellow open .ds-filebeat-8.19.3-2025.09.25-000001 Y_-CPWDbT_CD9VPfJtToNg 1 1 0 0 249b 249b 249b green open .internal.alerts-dataset.quality.alerts-default-000001 qQx2K-L8Sp2uOJacEeQa1g 1 0 0 0 249b 249b 249b green open .kibana-observability-ai-assistant-kb-000001 5xTfI1D9RzKBb7HUlwyEhA 1 0 0 0 249b 249b 249b yellow open .ds-metricbeat-8.19.2-2025.09.17-000002 rkm3gGXRQ9mt8p52XWG76A 1 1 0 0 249b 249b 249b ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [October 2, 2025, 2:04pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/8 "2025-10-02T14:04:14Z")
Hi @decibel83 `"output":{"events":{"active":0,"batches":2,"dropped":175,"total":175},` `"output":{"events":{"active":0,"batches":3,"dropped":146,"total":146}` Looks like you are perhaps dropping events. Did you run `./filebeat setup -e` Can you run `GET _index_template/filebeat-8.19.3` Under the logs directory is there and `events` directory if so check in there. ---
**Author:** ![decibel83](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@decibel83](https://discuss.elastic.co/u/decibel83)\ **Post date:** [October 2, 2025, 2:31pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/9 "2025-10-02T14:31:11Z")
> [@stephenb](#): > > Did you run > > `./filebeat setup -e` Yes, I ran it and ran it again right now. > [@stephenb](#): > > Can you run > > `GET _index_template/filebeat-8.19.3` Yes, I receive a 28000+ rows JSON output. > [@stephenb](#): > > Under the logs directory is there and `events` directory if so check in there. Ooooookay, maybe I got it: ```auto {"log.level":"warn","@timestamp":"2025-10-02T16:24:50.001+0200","log.logger":"elasticsearch","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/outputs/elasticsearch.(*Client).applyItemStatus","file.name":"elasticsearch/client.go","file.line":533},"message":"Cannot index event '{\"@timestamp\":\"2025-10-02T14:24:39.994Z\",\"event\":{\"dataset\":\"system.syslog\",\"module\":\"system\",\"timezone\":\"+02:00\"},\"fileset\":{\"name\":\"syslog\"},\"input\":{\"type\":\"log\"},\"ecs\":{\"version\":\"1.12.0\"},\"log\":{\"offset\":11806925633,\"file\":{\"path\":\"/var/log/syslog\"}},\"service\":{\"type\":\"system\"},\"host\":{\"os\":{\"platform\":\"debian\",\"version\":\"12 (bookworm)\",\"family\":\"debian\",\"name\":\"Debian GNU/Linux\",\"kernel\":\"6.1.0-38-amd64\",\"codename\":\"bookworm\",\"type\":\"linux\"},\"id\":\"3680414d6afd4c7a8b11cadfd31dfaac\",\"containerized\":false,\"ip\":[\"192.168.110.140\",\"fe80::8c2d:e9ff:fe57:aa37\",\"172.18.0.1\",\"fe80::8cb2:22ff:feb5:d201\",\"172.19.0.1\",\"fe80::b49b:efff:fef9:7a63\",\"172.240.0.1\",\"fe80::dcf5:4eff:fe77:9d0c\",\"fe80::9c13:1dff:fefe:e801\"],\"mac\":[\"8E-2D-E9-57-AA-37\",\"8E-B2-22-B5-D2-01\",\"9E-13-1D-FE-E8-01\",\"B6-9B-EF-F9-7A-63\",\"D6-DC-3A-7C-F6-22\",\"DE-F5-4E-77-9D-0C\"],\"name\":\"elk\",\"hostname\":\"elk\",\"architecture\":\"x86_64\"},\"agent\":{\"name\":\"elk\",\"type\":\"filebeat\",\"version\":\"8.19.3\",\"ephemeral_id\":\"ec7287ab-e926-4684-81d0-7805fd6fcad1\",\"id\":\"15b0aca0-3127-4275-94f9-a93f239d8696\"},\"message\":\"Oct 2 16:24:39 elk logstash[409844]: [2025-10-02T16:24:39,240][INFO][logstash.runner] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]\"}\n, Meta: {\"pipeline\":\"filebeat-8.19.3-system-syslog-entrypoint\"}' (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:data/write/bulk[s]] is unauthorized for user [filebeat_setup] with effective roles [filebeat_setup,filebeat_writer,ingest_admin,kibana_admin] on indices [filebeat-8.19.3], this action is granted by the index privileges [create_doc,create,delete,index,write,all]\"}, dropping event!","service.name":"filebeat","log.type":"event","ecs.version":"1.6.0"} ``` I mistakenly created the `filebeat_writer` role: I fixed the permissions adding the `create_doc` permission to `filebeat-*`. Thank you! ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [October 2, 2025, 2:57pm UTC](https://discuss.elastic.co/t/filebeat-connection-to-elasticsearch-ok-but-no-documents-indexed/382285/10 "2025-10-02T14:57:11Z")
@decibel83 Cool glad you found it. Yeah and the bummer about an error like that the `filebeat test output` will still connect but it doesn't actually try to write a document... so you need to check the event