# Beats fields.yml and multi-fields

**URL:** <https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277>\
**Category:** Beats\
**Created:** [May 17, 2018, 8:09am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277 "2018-05-17T08:09:09Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sirk\_Johannsen](https://avatars.discourse-cdn.com/v4/letter/s/46a35a/32.png) [@Sirk\_Johannsen](https://discuss.elastic.co/u/Sirk_Johannsen)\
**Post date:** [May 17, 2018, 8:09am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/1 "2018-05-17T08:09:09Z")

</div>

Hello,

I really appreciate the great work on the beats. But the documentation for the new fields.yml is just awful or plainly missing 😉  
Can someone explain to me how to achieve multi-field mapping as described here:  
[https://www.elastic.co/guide/en/elasticsearch/reference/6.2/multi-fields.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/multi-fields.html)  
within fields.yml ?

Many thanks and best regards,

Sirk

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 17, 2018, 8:39am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/2 "2018-05-17T08:39:23Z")

</div>

Missing is probably the better word for it ☹

Here is an example: [https://github.com/elastic/beats/blob/master/auditbeat/\_meta/fields.common.yml#L26](https://github.com/elastic/beats/blob/master/auditbeat/_meta/fields.common.yml#L26)

Are you using the new `append_fields` feature or modifying the original file?

---

<div class="post-metadata">

**Author:** ![Sirk\_Johannsen](https://avatars.discourse-cdn.com/v4/letter/s/46a35a/32.png) [@Sirk\_Johannsen](https://discuss.elastic.co/u/Sirk_Johannsen)\
**Post date:** [May 17, 2018, 8:56am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/3 "2018-05-17T08:56:41Z")

</div>

Nice. Thanks. `multi_fields` is what I was looking for. I'm modifying the original file. Or better passing a custiom `fields.yml` via `setup.template.fields`

Many thanks !

Sirk

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 22, 2018, 8:37am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/4 "2018-05-22T08:37:45Z")

</div>

The feature I was referring to is not really documented yet and is only in master so far. Here you can see the details: [https://github.com/elastic/beats/pull/6024](https://github.com/elastic/beats/pull/6024)

The tricky part before append\_fields is that each new version of beats contains new fields (normally) so the fields.yml should be overwritten. But then you have to apply your changes every time.

Can you share a bit more background on what kind of fields you added to fields.yml. This should also help me to better understand how people use it / modify it.

---

<div class="post-metadata">

**Author:** ![Sirk\_Johannsen](https://avatars.discourse-cdn.com/v4/letter/s/46a35a/32.png) [@Sirk\_Johannsen](https://discuss.elastic.co/u/Sirk_Johannsen)\
**Post date:** [May 22, 2018, 9:01am UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/5 "2018-05-22T09:01:37Z")

</div>

Sure. The thing is that we use Filebeat -\> ES. No logstash or anything and we log from nginx and go app directly in json to disk.  
So Filebeat picks up the already formatted logs from disk and just passed them on to Elasticsearch. This leads to having a totally custom nginx log with our own field-names but more importantly our other apps have custom log format not matching anything out there 😉  
Therefore we need to define our custom mapping.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 22, 2018, 1:28pm UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/6 "2018-05-22T13:28:39Z")

</div>

Thanks for the details. JSON logs is definitively a big use case here.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 19, 2018, 3:28pm UTC](https://discuss.elastic.co/t/beats-fields-yml-and-multi-fields/132277/7 "2018-06-19T15:28:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
