# Filebeat not creating index in ES 6.5.1

**URL:** https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785
**Category:** Beats
**Tags:** filebeat
**Created:** [December 13, 2018, 4:59pm UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785 "2018-12-13T16:59:42Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![vishnujyothi](https://avatars.discourse-cdn.com/v4/letter/v/ba9def/32.png) [@vishnujyothi](https://discuss.elastic.co/u/vishnujyothi)
#### Post date: [December 13, 2018, 4:59pm UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785/1 "2018-12-13T16:59:42Z")

</div>

Hi

Am using filebeat 6.5.1 connected to ES 6.5 , was trying/googling to find a fix to index creation problem. metricbeat and heartbeat installed in same machine are working fine (creating index and all)

ALL my ES/Kibana and beats are in same machine so configuration are pretty simple . Tried many steps given in similar issues and didn't succeeded any help would be much useful

```
**File beat log**

```

2018-12-13T16:56:16.043Z DEBUG [elasticsearch] elasticsearch/client.go:711 Ping status code: 200  
2018-12-13T16:56:16.043Z INFO elasticsearch/client.go:712 Connected to Elasticsearch version 6.5.1  
2018-12-13T16:56:16.043Z DEBUG [elasticsearch] elasticsearch/client.go:730 HEAD [http://localhost:9200/\_template/filebeat](http://localhost:9200/_template/filebeat)  
2018-12-13T16:56:16.047Z INFO template/load.go:129 Template already exists and will not be overwritten.  
2018-12-13T16:56:16.047Z DEBUG [modules] fileset/pipelines.go:45 Required processors:   
2018-12-13T16:56:16.047Z DEBUG [elasticsearch] elasticsearch/client.go:730 GET [http://localhost:9200/\_ingest/pipeline/filebeat-6.5.1-logstash-log-pipeline-plain](http://localhost:9200/_ingest/pipeline/filebeat-6.5.1-logstash-log-pipeline-plain)  
2018-12-13T16:56:16.048Z DEBUG [modules] fileset/pipelines.go:71 Pipeline filebeat-6.5.1-logstash-log-pipeline-plain already loaded  
2018-12-13T16:56:16.048Z DEBUG [modules] fileset/pipelines.go:45 Required processors:   
2018-12-13T16:56:16.048Z DEBUG [elasticsearch] elasticsearch/client.go:730 GET [http://localhost:9200/\_ingest/pipeline/filebeat-6.5.1-logstash-slowlog-pipeline-plain](http://localhost:9200/_ingest/pipeline/filebeat-6.5.1-logstash-slowlog-pipeline-plain)  
  
2018-12-13T16:56:16.049Z DEBUG [modules] fileset/pipelines.go:71 Pipeline filebeat-6.5.1-logstash-slowlog-pipeline-plain already loaded  
2018-12-13T16:56:16.049Z DEBUG [modules] fileset/pipelines.go:45 Required processors:   
2018-12-13T16:56:16.049Z DEBUG [fileset] fileset/fileset.go:220 Comparing ES version 6.5.1 with requirement of 6.1.0  
2018-12-13T16:56:16.049Z DEBUG [elasticsearch] elasticsearch/client.go:730 GET [http://localhost:9200/\_ingest/pipeline/filebeat-6.5.1-system-syslog-pipeline](http://localhost:9200/_ingest/pipeline/filebeat-6.5.1-system-syslog-pipeline)  
2018-12-13T16:56:16.050Z DEBUG [modules] fileset/pipelines.go:71 Pipeline filebeat-6.5.1-system-syslog-pipeline already loaded  
2018-12-13T16:56:16.050Z DEBUG [modules] fileset/pipelines.go:45 Required processors:   
2018-12-13T16:56:16.050Z DEBUG [fileset] fileset/fileset.go:220 Comparing ES version 6.5.1 with requirement of 6.1.0

**Filebeat YML**

```
filebeat.inputs:

    # Each - is an input. Most options can be set at the input level, so
    # you can use different inputs for various configurations.
    # Below are the input specific configurations.

    - type: log

      # Change to true to enable this input configuration.
      enabled: true

      # Paths that should be crawled and fetched. Glob based paths.
      paths:
       #- /var/log/*.log
        - /data/logs/grafana/*.log
        - /data/logs/metricbeat/*

        #- c:\programdata\elasticsearch\logs\*

    #==================== Elasticsearch template setting ==========================

    setup.template.name: "filebeat"
    setup.template.pattern: "filebeat-*"
    setup.template.overwrite: false
    setup.template.settings:
      index.number_of_shards: 1
      #index.codec: best_compression
      #_source.enabled: false

    #-------------------------- Elasticsearch output ------------------------------
    output.elasticsearch:
      # Array of hosts to connect to.
      hosts: ["localhost:9200"]
      index: "filebeat-%{[beat.version]}-%{+yyyy.MM.dd}"
      filebeat.registry_file: /var/lib/filebeat/registry

      # Optional protocol and basic auth credentials.
      #protocol: "https"
      #username: "elastic"
      #password: "changeme"
```

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [December 14, 2018, 11:54pm UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785/2 "2018-12-14T23:54:29Z")

</div>

The `filebeat-*` index is created when Filebeat first tries to index a log event from one of your log files. I see you included the Filebeat log above. Is this the complete log? Are there any mentions of "Harvester" in your Filebeat log?

---

<div class="post-metadata">

### Author: ![vishnujyothi](https://avatars.discourse-cdn.com/v4/letter/v/ba9def/32.png) [@vishnujyothi](https://discuss.elastic.co/u/vishnujyothi)
#### Post date: [December 17, 2018, 9:53am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785/3 "2018-12-17T09:53:22Z")

</div>

Its says Harvester for file still running

```
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:494 Update existing file for harvesting: /var/log/messages, offset: 411624
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:546 Harvester for file is still running: /var/log/messages
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:195 input states cleaned up. Before: 1, After: 1, Pending: 0
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:404 Check file for harvesting: /var/log/auth.log
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:494 Update existing file for harvesting: /var/log/auth.log, offset: 0
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:548 File didn't change: /var/log/auth.log
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:404 Check file for harvesting: /var/log/secure
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:494 Update existing file for harvesting: /var/log/secure, offset: 178251
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:546 Harvester for file is still running: /var/log/secure
2018-12-17T09:51:12.920Z DEBUG [input] log/input.go:195 input states cleaned up. Before: 2, After: 2, Pending: 0
2018-12-17T09:51:18.401Z DEBUG [input] input/input.go:152 Run input
2018-12-17T09:51:18.401Z DEBUG [input] log/input.go:174 Start next scan
2018-12-17T09:51:18.401Z DEBUG [input] input/input.go:152 Run input
2018-12-17T09:51:18.401Z DEBUG [input] log/input.go:174 Start next scan
2018-12-17T09:51:18.401Z DEBUG [input] log/input.go:195 input states cleaned up. Before: 0, After: 0, Pending: 0
2018-12-17T09:51:18.401Z DEBUG [input] log/input.go:195 input states cleaned up. Before: 0, After: 0, Pending: 0
2018-12-17T09:51:22.920Z DEBUG [input] input/input.go:152 Run input
2018-12-17T09:51:22.920Z DEBUG [input] log/input.go:174 Start next scan
2018-12-17T09:51:22.920Z DEBUG [input] input/input.go:152 Run input
2018-12-17T09:51:22.920Z DEBUG [input] log/input.go:174 Start next scan
2018-12-17T09:51:22.920Z DEBUG [input] log/input.go:404 Check file for harvesting: /var/log/messages
2018-12-17T09:51:22.921Z DEBUG [input] log/input.go:494 Update existing file for harvesting: /var/log/messages, offset: 411624
2018-12-17T09:51:22.921Z DEBUG [input] log/input.go:546 Harvester for file is still running: /var/log/messages
2018-12-17T09:51:22.921Z DEBUG [input] log/input.go:195 input states cleaned up. Before: 1, After: 1, Pending: 0
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 14, 2019, 9:53am UTC](https://discuss.elastic.co/t/filebeat-not-creating-index-in-es-6-5-1/160785/4 "2019-01-14T09:53:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
