# Why there are not any index on elasticsearch after run filebeat

**URL:** <https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422>\
**Category:** Beats\
**Created:** [November 4, 2023, 10:08am UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422 "2023-11-04T10:08:46Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 4, 2023, 10:08am UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/1 "2023-11-04T10:08:47Z")

</div>

My elasticsearch version = 8.10.4  
My filebeat version : 8.7

Also these are outputs :

```auto
filebeat test output
elasticsearch: https://172.10.110.29:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 172.10.110.29
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.10.4

```

```auto
curl -XGET "https://172.10.110.29:9200" -u elastic --cacert /etc/filebeat/certs/http_ca.crt
Enter host password for user 'elastic':
{
  "name" : "elstack",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "3lZ748BgSiG2KRijMrorgQ",
  "version" : {
    "number" : "8.10.4",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "b4a62ac808e886ff032700c391f45f1408b2538c",
    "build_date" : "2023-10-11T22:04:35.506990650Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

```

and my filebeat service is running on the server but it does not create an index on the elasticsearch

 ![inn](https://us1.discourse-cdn.com/elastic/original/3X/0/a/0af95d70033723d1d77d3a9a9234c0288ec59419.jpeg)

This is part of my filebeat log :

```auto
{"log.level":"warn","@timestamp":"2023-11-04T13:34:11.144+0330","log.logger":"add_cloud_metadata","log.origin":{"file.name":"add_cloud_metadata/provider_aws_ec2.go","file.line":81},"message":"read token request for getting IMDSv2 token returns empty: Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded (Client.Timeout exceeded while awaiting headers). No token in the metadata request will be used.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-11-04T13:34:11.146+0330","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.146+0330","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://172.10.110.29:9200","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.168+0330","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":162},"message":"'ca_trusted_fingerprint' set, looking for matching fingerprints","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.168+0330","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":173},"message":"CA certificate matching 'ca_trusted_fingerprint' found, adding it to 'certificate_authorities'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.190+0330","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":162},"message":"'ca_trusted_fingerprint' set, looking for matching fingerprints","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.191+0330","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":173},"message":"CA certificate matching 'ca_trusted_fingerprint' found, adding it to 'certificate_authorities'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-04T13:34:11.194+0330","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.10.4","service.name":"filebeat","ecs.version":"1.6.0"}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 4, 2023, 1:08pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/2 "2023-11-04T13:08:53Z")

</div>

Filebeat 8 writes to data streams, not normal indices, check the Data Streams tab on Index Management to validate if it is sending data.

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 4, 2023, 5:18pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/3 "2023-11-04T17:18:59Z")

</div>

Yes. Filebeat has created in data streams tab . But as we have different servers and filebeat 8.7 has been installed on all of them how can change data stream file name from [filebeat-8.7.0] to hostname or filebeat +hostname ?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 4, 2023, 7:26pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/4 "2023-11-04T19:26:38Z")

</div>

> [@baber1223](#):
>
> how can change data stream file name from [filebeat-8.7.0] to hostname or filebeat +hostname ?

Hi @baber1223 that is generally is really not a good plan, the best practice is to put 1 to many host into the `filebeat-n.n.n` data stream and then filter on `host.name` when you want to look at specifics.

I am not saying you can not do what you want but generally will lead to other issues later on.

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 5, 2023, 1:07pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/5 "2023-11-05T13:07:44Z")

</div>

Thanks. but according to attached pic there is just one name in my data stream tab . On the other hand I have installed filebeat on 3 hosts and now how can find out which server is sending data stream to Elasticsearch ?

 ![ind](https://us1.discourse-cdn.com/elastic/original/3X/e/7/e7d3260e99ec150d027de430d16a94df6e1317b3.jpeg)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 5, 2023, 1:11pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/6 "2023-11-05T13:11:08Z")

</div>

You need to go into Discover and filter your data based on the field `host.name` or any other field that identifies the Filebeat that is sending the data.

Having one index/data stream **per host** is a bad approach that can lead to many issues.

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 5, 2023, 1:31pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/7 "2023-11-05T13:31:25Z")

</div>

Does it your mean we have to create multiple index/data stream ? Does it better to create one index/data stream for each host ?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 5, 2023, 2:15pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/8 "2023-11-05T14:15:40Z")

</div>

> [@baber1223](#):
>
> Does it your mean we have to create multiple index/data stream ? Does it better to create one index/data stream for each host ?

No, it is the opposite, the _per host_ was missing on my previous answer, I just fixed it.

This is what I mean:

> Having one index/data stream **per host** is a bad approach that can lead to many issues.

Imagine that you have 15k host, having one data stream per host would lead to have 15k data streams and having too many indices can impact heavily on performance, also you could end up with too many small indices, which is also bad for performance.

Filebeat adds some fields like `agent.name` and `agent.id` that you can use to filter the filebeat of each host and you can also [add custom fields](https://www.elastic.co/guide/en/beats/filebeat/current/add-fields.html) to filter on if you want.

---

<div class="post-metadata">

**Author:** ![baber1223](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baber1223/32/83533_2.png) [@baber1223](https://discuss.elastic.co/u/baber1223)\
**Post date:** [November 5, 2023, 2:39pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/9 "2023-11-05T14:39:15Z")

</div>

I think maybe that is better create specific index for specific services instead of each host . What is your idea ?

On the other hand how can set in filebeat to create specific index/ data stream with custom name ?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 5, 2023, 2:43pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/10 "2023-11-05T14:43:56Z")

</div>

> [@baber1223](#):
>
> I think maybe that is better create specific index for specific services instead of each host . What is your idea ?

Filebeat only supports one output, if you have multiple services on a host, you would need to run multiple filebeats and this can become pretty hard to manage depending on the number of services/hosts.

> [@baber1223](#):
>
> On the other hand how can set in filebeat to create specific index/ data stream with custom name ?

I do not use filebeat anymore, but you can follow the documentation [here](https://www.elastic.co/guide/en/beats/filebeat/current/change-index-name.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 3, 2023, 2:44pm UTC](https://discuss.elastic.co/t/why-there-are-not-any-index-on-elasticsearch-after-run-filebeat/346422/11 "2023-12-03T14:44:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.