# Filebeat 8.3

**URL:** <https://discuss.elastic.co/t/filebeat-8-3/314630>\
**Category:** Beats\
**Tags:** filebeat, metricbeat\
**Created:** [September 17, 2022, 11:52pm UTC](https://discuss.elastic.co/t/filebeat-8-3/314630 "2022-09-17T23:52:30Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![abaltan](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@abaltan](https://discuss.elastic.co/u/abaltan)\
**Post date:** [September 17, 2022, 11:52pm UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/1 "2022-09-17T23:52:30Z")

</div>

Hello,

Hope you doing well.

we are upgrading Filebeat from 7.12 to 8.3 and we are using the config file below. But for some reason filebeat is not working. Could you please check if there is anything we need to modify on our configuration file.

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    
    - /opt/apigee/var/log/edge-message-processor/messagelogging/*/SF-MessageLogging/*/*/*.log
  json.keys_under_root: true
  json.add_error_key: true
  json.message_key: log

setup.ilm.enabled: false

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0
  index.codec: best_compression

name: "apigee-dev-beat"
tags: ["apigee-dev-beat"]

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "api-log"
setup.ilm.pattern: "{now/d}-000001"

output.elasticsearch:
  hosts: ["http://IP:port"]
  # ssl.verification_mode: none
  username: [username]
  password: [passwrod]
  index: "api-log-%{[agent.version]}-%{+yyyy.MM.dd}"
  pipeline: "geoip"

setup.template.name: "api-log"
setup.template.pattern: "api-log-*"
setup.template.overwrite: true
setup.template.append_fields:
  - name: apitimestamp
    type: date
# format: "strict_date_optional_time||epoch_millis||yyyy-MM-dd'T'HH:mm:ss.SSS||yyyy-MM-dd'T'HH:mm:ssZZ"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

```

Thanks in advance  
Regards,

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 18, 2022, 12:09am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/2 "2022-09-18T00:09:22Z")

</div>

> [@abaltan](#):
>
> Could you please check if there is anything we need to modify on our configuration file.

You going to need to tell us exactly what is not working? then perhaps we can help.

Are no events getting ingested?

Are events getting ingest and they are wrongly parsed?

Did you run

`filebeat test config`

What do you mean by not working?

Have you looked at the filebeat logs? for ERRORs?

---

<div class="post-metadata">

**Author:** ![abaltan](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@abaltan](https://discuss.elastic.co/u/abaltan)\
**Post date:** [September 18, 2022, 12:32am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/3 "2022-09-18T00:32:18Z")

</div>

Hi @stephenb,

Thanks for your reply. The process itself is not running. as shown below

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/0/a0fe5a6562f8be47e89fb1d41ac99715b9ea1b4c.png)

---

<div class="post-metadata">

**Author:** ![abaltan](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@abaltan](https://discuss.elastic.co/u/abaltan)\
**Post date:** [September 18, 2022, 12:38am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/4 "2022-09-18T00:38:48Z")

</div>

Hello,

it looks like there was an issue with  
setup.ilm.enabled: auto, changed auto to true and it worked.

Could you please share a document to show the difference between auto and true/false.

Thanks

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 18, 2022, 12:54am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/5 "2022-09-18T00:54:27Z")

</div>

Pretty easy to search and find 🙂

> **[Configure index lifecycle management | Filebeat Reference \[8.3\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/8.3/ilm.html#_configuration_options_24)**

> setup.ilm.enabled  
> Enables or disables index lifecycle management on any new indices created by Filebeat. \>\>\> Valid values are true and false.

---

<div class="post-metadata">

**Author:** ![abaltan](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@abaltan](https://discuss.elastic.co/u/abaltan)\
**Post date:** [September 18, 2022, 1:21am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/6 "2022-09-18T01:21:41Z")

</div>

I already read that, but I am new to elastic and it is still not clear for me. anyway thx for your time.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 18, 2022, 2:04am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/7 "2022-09-18T02:04:13Z")

</div>

`auto` is not a valid value.

`true` means use the settings defined in the filebeat.yml for ILM

`false` mean you either do not want ILM or you have [set it up manually](https://www.elastic.co/guide/en/elasticsearch/reference/8.3/index-lifecycle-management.html) through index template, ILM policy and bootstrap index.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 19, 2022, 12:54am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/8 "2022-09-19T00:54:03Z")

</div>

@abaltan

Does the above help or do you have a more specific question.

> `auto` is no longer a valid setting it was a valid setting in 7.x ... in the Past 7.x `auto` was (the default) is specified on version 7.0 and later, Filebeat automatically uses index lifecycle management if the feature is enabled in Elasticsearch and has the required license; \> otherwise, Filebeat creates daily indices.
> 
> `true` and `false` is much cleaner now  
> Either `true` you use the setting in the filebeat.yml or `false` you do not.

ILM is a big topic.. basically the way you are doing it is daily indices (rollover)... if you want to use hot / warm etc you will need to setup and ILM policy and map it into the index template .. .by default it will use the filebeat ILM policy... below I showed you how to name your own policy which you would create.

I noticed a few other things in your config above.  
You had an extra `setup.ilm.enabled: false` at the top

Also I cleaned up the names and the template name etc.. etc.. I think this will work better for you

You need to run `filebeat setup -e` to load / create the template at least once.

Here is my cleaned up version with some side notes for you. This is 8.x so it filebeat uses data streams by default perhaps take a glance at that!

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
   - /Users/sbrown/workspace/sample-data/discuss/container-mixed/*.log
  # json.keys_under_root: true
  # json.add_error_key: true
  # json.message_key: log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0
  index.codec: best_compression

name: "apigee-dev-beat"
tags: ["apigee-dev-beat"]

setup.ilm.enabled: true
setup.ilm.rollover_alias: "api-log"
setup.ilm.pattern: "{now/d}-000001"

output.elasticsearch:
  hosts: ["localhost:9200"]
  index: "api-log-%{[agent.version]}" <!---- Don't add -%{+yyyy.MM.dd} its a data strean and getts added to the backing index automatically
  # ssl.verification_mode: none
  # username: [username]
  # password: [passwrod]
  # pipeline: "geoip"

setup.template.enabled: true <!--- You should explicitly set this 
setup.template.name: "api-log-%{[agent.version]}" <!----This needs to match the index name because now it is a data stream
setup.template.pattern: "api-log-%{[agent.version]}" <!-- Better matching Hygiene because it is a data stream 
setup.template.overwrite: false <!--- Careful with this if set to true it will overwrite every time.
setup.ilm.policy_name: api-policy <!---- If you want to use your own policy 
setup.template.append_fields:
  - name: apitimestamp
    type: date
# # format: "strict_date_optional_time||epoch_millis||yyyy-MM-dd'T'HH:mm:ss.SSS||yyyy-MM-dd'T'HH:mm:ssZZ"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

```

Results template

 ![Screen Shot 2022-09-18 at 6.35.13 PM](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9edd5fca909f223d06fc3f830ee2c0e602f21998.png)

Data Stream

 ![Screen Shot 2022-09-18 at 5.57.18 PM](https://us1.discourse-cdn.com/elastic/original/3X/7/8/78cb81cd81a5fef6aebe935c43e8f9e0ad074941.png)

Backing index

 ![Screen Shot 2022-09-18 at 5.57.57 PM](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bbb60442b8cfd222236478977fad1b4b9074b2b8.png)

---

<div class="post-metadata">

**Author:** ![abaltan](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@abaltan](https://discuss.elastic.co/u/abaltan)\
**Post date:** [September 19, 2022, 8:40am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/9 "2022-09-19T08:40:52Z")

</div>

Hi @stephenb,

Thank you so much. Yeah I have read about data streams but I still need to read more tbh.

I appreciate your support.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 17, 2022, 10:41am UTC](https://discuss.elastic.co/t/filebeat-8-3/314630/10 "2022-10-17T10:41:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.