# Filebeat does not insert data into elasticsearch using ILM (node default index)

**URL:** https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288
**Category:** Beats
**Tags:** filebeat
**Created:** [April 22, 2020, 2:51pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288 "2020-04-22T14:51:00Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Alon\_Eldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alon_eldi/32/66878_2.png) [@Alon\_Eldi](https://discuss.elastic.co/u/Alon_Eldi)
#### Post date: [April 22, 2020, 2:51pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288/1 "2020-04-22T14:51:01Z")

</div>

Hi , We have filebeat agents which write data directly into elasticsearch cloud service.

For some reason the filebeat is not writing the data into the index

The filbeat use ILM policy :

setup.ilm.enabled: auto  
setup.ilm.rollover\_alias: "data-prod"  
setup.ilm.pattern: "{now/d}-000001"  
setup.ilm.policy\_name: "data-prod-%{[agent.version]}"  
setup.ilm.check\_exists: false  
setup.ilm.overwrite: true  
setup.ilm.policy\_file: /etc/filebeat/index.json

## /etc/filebeat/index.json

{  
"policy": {  
"phases": {  
"hot": {  
"min\_age": "0ms",  
"actions": {  
"rollover": {  
"max\_age": "24h",  
"max\_size": "50gb",  
"max\_docs": 100000000  
},  
"set\_priority": {  
"priority": 100  
}  
}  
},  
"delete": {  
"min\_age": "61d",  
"actions": {  
"delete": {}  
}  
}  
}  
}  
}

the data-prod alias have 1 writing index .

"data-prod-2020.04.16-000600" : {  
"aliases" : {  
"data-prod" : {  
"is\_write\_index" : true  
}  
}  
}

I wrote to the alias from the filebeat machine successfully :

curl -k -u elastic -X PUT "https:///data-prod/\_doc/1?pretty" -H 'Content-Type: application/json' -d'  
{  
"name": "John Doe"  
}'

So the filebeat agent can write to elasticsearch and to the correct index .

I run filebeat in debug mode :  
filebeat -e -d "\*" -c /etc/filebeat/filebeat.yml

dont get any errors and output show correct values :

2020-04-22T13:10:56.509Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'data-prod' as ILM is enabled.  
2020-04-22T13:10:56.509Z INFO elasticsearch/client.go:170 Elasticsearch url: https://:443

Do I miss anything

Thanks

Alon

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [April 22, 2020, 8:30pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288/2 "2020-04-22T20:30:07Z")

</div>

Hi @Alon_Eldi, welcome to the Elastic community forums!

Just want to clarify something: is Filebeat not writing data into _any_ Elasticsearch index? Or is it writing data but just not into the `data-prod-2020.04.16-000600` index (via the `data-prod` write alias)?

Shaunak

---

<div class="post-metadata">

### Author: ![Alon\_Eldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alon_eldi/32/66878_2.png) [@Alon\_Eldi](https://discuss.elastic.co/u/Alon_Eldi)
#### Post date: [April 22, 2020, 9:29pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288/3 "2020-04-22T21:29:04Z")

</div>

It’s not writing to any index .  
And does not show any error in filebeat debug output .

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [April 23, 2020, 6:49pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288/4 "2020-04-23T18:49:00Z")

</div>

Okay, thanks. In that case I would try to first narrow down where the problem is happening — Filebeat or Elasticsearch. To do this, could you temporarily disable the Elasticsearch output in Filebeat by setting `output.elasticsearch.enabled: false` and enable the Console output instead by setting `output.console.enabled: true`? Then restart Filebeat and let us know if you are seeing any output from Filebeat to your console.

Thanks,

Shaunak

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 21, 2020, 6:49pm UTC](https://discuss.elastic.co/t/filebeat-does-not-insert-data-into-elasticsearch-using-ilm-node-default-index/229288/5 "2020-05-21T18:49:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
