# Filebeat index is getting created but with 0 documents

**URL:** <https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 26, 2020, 6:34pm UTC](https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667 "2020-12-26T18:34:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![anujaggarwal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anujaggarwal/32/81418_2.png) [@anujaggarwal](https://discuss.elastic.co/u/anujaggarwal)\
**Post date:** [December 26, 2020, 6:34pm UTC](https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667/1 "2020-12-26T18:34:35Z")

</div>

Hi

I am trying to index my custom log file using filebeat. I am successfully running filebeat with pre-built modules like mysql, nginx etc. But when I actually try to use it with my application specific log file, index is created with 0 documents.

I could not find anywhere in the filebeats document if there are any specific steps need to be taken to ensure indexing takes place for the custom log files.

I did not get any error when I setup filebeats or run filebeats post setup.

Below is the filebeat.yml:

> filebeat.inputs:  
> - type: log  
> enabled: true  
> paths:  
> - /Applications/MAMP/htdocs/247around-adminp-aws/application/logs/log-2020-12-21.log  
> include\_lines: ['^INFO', '^ERROR']  
> fields:  
> app\_id: crm  
> filebeat.config.modules:  
> setup.template.settings:  
> index.number\_of\_shards: 1  
> path: ${path.config}/modules.d/\*.yml  
> setup.kibana:  
> output.elasticsearch:  
> hosts: ["localhost:9200"]  
> processors:

As can be seen, it is majorly default .yml file with very minor changes.

My custom log file log-2020-12-21.php is:

> ```
> INFO - 2020-12-21 15:10:26 --> index Logging details have been captured for employee. Details are : Array
> INFO - 2020-12-21 15:10:36 --> editpartner partner_id:1
> INFO - 2020-12-21 15:10:36 --> SELECT DISTINCT service_id, brand, active
> ERROR - 2020-12-21 15:10:36 --> Query error: Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'boloaaka.collateral.id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
> INFO - 2020-12-21 15:10:36 --> Database Error: A Database Error Occurred<br/>Array
> ERROR - 2020-12-21 15:10:54 --> Query error: Expression #5 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'boloaaka.service_centres.district' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
> INFO - 2020-12-21 15:10:54 --> Database Error: A Database Error Occurred<br/>Array
> INFO - 2020-12-21 23:53:21 --> Loginindex
> INFO - 2020-12-21 23:54:50 --> Loginindex
> INFO - 2020-12-21 23:55:42 --> Loginindex
> INFO - 2020-12-21 23:56:24 --> Loginindex
> 
> ```

Index file is getting created with 0 documents:

 ![Index-Management-Elastic](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26619f23055317f031bae5e82020cebe85372691.jpeg)

Log file showing logs for filebeats setup and filebeats running:

> **[Filebeat Log - 1 - Pastebin.com](https://pastebin.com/TK6uYXuq)**
>
> Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Please help:

1. Why there are no error messages if something is wrong because of which documents are not getting indexed? I should be getting some error if things are not right.
2. How should I index my log file?
3. Where should I add pattern for my log file like key-value pair which would help me in searching the documents for relevant values later on?

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [December 29, 2020, 1:27am UTC](https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667/2 "2020-12-29T01:27:22Z")

</div>

When you configure Filebeat for your custom log file, can you temporarily set `output.elasticsearch.enabled: false` and `output.console.enabled: true`? Then start Filebeat and check if there are any events are being output to the console.

Shaunak

---

<div class="post-metadata">

**Author:** ![anujaggarwal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anujaggarwal/32/81418_2.png) [@anujaggarwal](https://discuss.elastic.co/u/anujaggarwal)\
**Post date:** [December 30, 2020, 4:49pm UTC](https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667/3 "2020-12-30T16:49:57Z")

</div>

There was one error in my configuration file: log file name was incorrect. Log file extension was .PHP whereas in the log file, it was mentioned as .LOG which was causing the issue.

Once I fixed that, file was started getting indexed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 27, 2021, 6:50pm UTC](https://discuss.elastic.co/t/filebeat-index-is-getting-created-but-with-0-documents/259667/4 "2021-01-27T18:50:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.