# Custom log file configuration

**URL:** <https://discuss.elastic.co/t/custom-log-file-configuration/159680>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 6, 2018, 8:21am UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680 "2018-12-06T08:21:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vijayakumar\_Kannan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakumar_kannan/32/34873_2.png) [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\
**Post date:** [December 6, 2018, 8:21am UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680/1 "2018-12-06T08:21:52Z")

</div>

i have the custom log file which i want to parse and send the logs to ES from the filebeat configuration. So i had the following

- filebeat configuration

```auto
filebeat.prospectors:
- type: log
  enabled: true
    - /tmp/custom.log
  pipeline: filebeat-custom-pipeline

```

- ES configuration

```auto
1. validated the pipeline with _ingest/pipeline/_simulate
2. posted the ingest pipeline in ES configuration

```

Now i had to change one field " **clientip**" type to " **geopoint**" data type. the article refers in filebeat go for new filebeat module development.

if i you can guide me for alternate way for creating the mapping for fields created out of custom file that would be more helpful.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 6, 2018, 1:38pm UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680/2 "2018-12-06T13:38:15Z")

</div>

> the article refers in filebeat go for new filebeat module development.

Which article?

You need to adapt the template mapping. This is normally done by adjusting fields.yml, but is also possible via json. There is a number of settings for [template setup documented here](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html). The `setup.template.append_fields` setting might fit your needs.

---

<div class="post-metadata">

**Author:** ![Vijayakumar\_Kannan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakumar_kannan/32/34873_2.png) [@Vijayakumar\_Kannan](https://discuss.elastic.co/u/Vijayakumar_Kannan)\
**Post date:** [December 10, 2018, 9:13am UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680/3 "2018-12-10T09:13:11Z")

</div>

thanks steffens, since i am using the filebeat 6.3.2 version so i had to use the fields.yml for field configuration and i am able to get it after the current index deletion and restarted the filebeat configuration.

i have a question

1. When i update the fields.yml does it take effect only on new index creation or any steps to be performed ?
2. if i have 500 servers and i need the custom fields.yml in single server (custom logs) in that server i had to enable the option of template overwrite?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 11, 2018, 1:23pm UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680/4 "2018-12-11T13:23:29Z")

</div>

> When i update the fields.yml does it take effect only on new index creation or any steps to be performed ?

The fields.yml is used to install a template mapping. This is a one time setup step. Afterwards Elasticsearch will use the template mapping to create an index.  
You can use setup or enable template overwriting to force the template to be replaced. This will only affect new indices though.

> if i have 500 servers and i need the custom fields.yml in single server (custom logs) in that server i had to enable the option of template overwrite?

Yes, overwrite might help here. But once you add another server needing to add another field you might have a problem. Then you should consider to have an extra centralized configuration with a centralized fields.yml and run run `filebeat setup` if required.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 8, 2019, 1:23pm UTC](https://discuss.elastic.co/t/custom-log-file-configuration/159680/5 "2019-01-08T13:23:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.