# Index in filebeat.yml doesn't match index in elasticsearch dataview

**URL:** <https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543>\
**Category:** Elasticsearch\
**Created:** [April 16, 2024, 7:08pm UTC](https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543 "2024-04-16T19:08:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vickistan](https://avatars.discourse-cdn.com/v4/letter/v/94ad74/32.png) [@Vickistan](https://discuss.elastic.co/u/Vickistan)\
**Post date:** [April 16, 2024, 7:08pm UTC](https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543/1 "2024-04-16T19:08:13Z")

</div>

I added filebeat to a Linux Mint system in an attempt to upload logs from it to my elk stack. The index is set to this in filebeat.yml:

index: "vicki-desktop-filestream"

When I started filebeat, I guess it created the index on the elasticsearch server as "filebeat-8.12.2" because that is the index that I can add to the data view to see the log traffic from that system. I tried to add "vicki-desktop-filesteam", but that isn't in the dropdown and isn't accepted if I type it in. I even created an index called that on the elasticsearch server using curl, but even though it allows me to create it, it doesn't ingest any data. I did create a data view by selecting the index "filebeat-8.12.2' though, and I see the data. I would like to be able to use the hostname as the index name since I have multiple systems that I want to pull in logs from and they will likely all use the same filebeat version. I have read through the index documentation, but I must be missing something.

Here is the filebeat.yml file without the comments:filebeat.inputs:

```auto
- type: log
  id: vicki-desktop-filestream-id
  enabled: true
  paths:
    - "/var/log/syslog"
    - "/var/log/messages"
    - "/var/log/*.log"
  multiline:
    pattern: '^[A-Z]\S{2,8} {1,2}\d{1,2} \d{2}:\d{2}:\d{2}\b'
    match: after
    negate: true
    processors:
    - add_locale: ~
output.elasticsearch:
    hosts: ["http://ubuntu-elk.mydomain.com:9200"]
    index: "vicki-desktop-filestream"
    pipeline: "vicki-desktop-filesteam-pipeline"
setup:
    template.enabled: false
    ilm.enabled: false
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "ubuntu-elk.mydomain.com:5601"
output.elasticsearch:
  hosts: ["ubuntu-elk.mydomain.com:9200"]
  preset: balanced
  username: "elastic"
  password: "changeme"
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

```

Can anyone clarify what I am missing here?

Vickistan

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 16, 2024, 9:56pm UTC](https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543/2 "2024-04-16T21:56:42Z")

</div>

> [@Vickistan](#):
>
> Can anyone clarify what I am missing here?

Not sure if this is the issue, but you have 2 elasticsearch outputs configured, this is not supported, you can have only **one** output.

If I'm not wrong if you have two, it will just ignore everything except the last one.

---

<div class="post-metadata">

**Author:** ![Vickistan](https://avatars.discourse-cdn.com/v4/letter/v/94ad74/32.png) [@Vickistan](https://discuss.elastic.co/u/Vickistan)\
**Post date:** [April 17, 2024, 1:24pm UTC](https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543/3 "2024-04-17T13:24:38Z")

</div>

That was absolutely what was happening. Thanks. Now I need to understand why this doesn't work:

```auto
    output.elasticsearch:
    hosts: ["ubuntu-elk.mydomain.com:9200"]
    preset: balanced
    username: "elastic"
    password: "changeme"
    index: "vicki-desktop-filestream"
    pipeline: "vicki-desktop-filesteam-pipeline"

```

when I comment out the second output and restart. I copied the uncommented lines from the one that was working up to this one. I expected that restarting filebeat on this system would create the index vicki-desktop-filestream, but it didn't. I know how to use a curl to create an index on the elasticsearch server, but I am not sure what the json body should look like or whether this is something that I should do:

```auto
curl -X PUT "localhost:9200/vicki-desktop-filestream?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "properties": {
      "host.name.keyword": { "type": "text" }
    }
  }
}
'

```

I really just want to rename the index to the hostname instead of filebeat-8.12..2

Can you point me to an example or documentation or tell me what to do to rename the index?

Vickistan

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 17, 2024, 10:38pm UTC](https://discuss.elastic.co/t/index-in-filebeat-yml-doesnt-match-index-in-elasticsearch-dataview/357543/4 "2024-04-17T22:38:15Z")

</div>

Perhaps look at the section of the document for this very use case.

> **[Change the index name | Filebeat Reference \[8.12\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.12/change-index-name.html)**

```auto
output.elasticsearch.index: "vicki-desktop-filestream-%{[agent.version]}"
setup.template.name: "vicki-desktop-filestream-%{[agent.version]}"
setup.template.pattern: "vicki-desktop-filestream-%{[agent.version]}"

```

We recommend the agent version but if not try exactly what you want

```auto
output.elasticsearch.index: "vicki-desktop-filestream"
setup.template.name: "vicki-desktop-filestream"
setup.template.pattern: "vicki-desktop-filestream"

```

That will create a data stream named `vicki-desktop-filestream` with a backing index...

```auto
GET _cat/indices/v*?v
GET _data_stream/vicki-desktop-filestream

# GET _cat/indices/v*?v 200 OK
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size dataset.size
yellow open .ds-vicki-desktop-filestream-2024.04.17-000001 qswF7nxdS7OnUiwd7Bj4OQ 1 1 10617 0 1.2mb 1.2mb 1.2mb

# GET _data_stream/vicki-desktop-filestream 200 OK
{
  "data_streams": [
    {
      "name": "vicki-desktop-filestream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-vicki-desktop-filestream-2024.04.17-000001",
          "index_uuid": "qswF7nxdS7OnUiwd7Bj4OQ",
          "prefer_ilm": true,
          "ilm_policy": "filebeat",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "status": "YELLOW",
      "template": "vicki-desktop-filestream",
      "ilm_policy": "filebeat",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}

```