# Filebeat and Nginx module - wrong data types in ES

**URL:** <https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 31, 2018, 10:51am UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505 "2018-12-31T10:51:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![JohnParson](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@JohnParson](https://discuss.elastic.co/u/JohnParson)\
**Post date:** [December 31, 2018, 10:51am UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505/1 "2018-12-31T10:51:19Z")

</div>

I'm using Nginx module in my Filebeat agent and sending data straight to ES. All data is sent correctly but there are some issues with data types. Almost all are using "text" data type. For example **`nginx.access.user_agent.name`** and **`nginx.access.body_sent.bytes`** which should use "keyword" and "long" data types according to [https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-nginx.html](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-nginx.html).

I'm running Filebeat in Docker, version is 6.5.3.

This is my filebeat.yml configuration.

* * *

filebeat.modules:  
 - module: nginx  
 access:  
 enabled: true  
 var.paths: ["/usr/share/filebeat/logs/nginx/\*access.log"]  
 error:  
 enabled: true  
 var.paths: ["/usr/share/filebeat/logs/nginx/\*error.log"]  
 - module: system  
 syslog:  
 enabled: true  
 var.paths: ["/usr/share/filebeat/logs/system/syslog"]  
 var.convert\_timezone: true  
name: california  
fields:  
 env: production

setup.kibana.host: myserver:5601

output.elasticsearch:  
 hosts: myserver:9200  
 indices:  
 - index: "filebeat-nginx-%{+yyyy.MM.dd}"  
 when.contains:  
 fileset.module: "nginx"  
 - index: "filebeat-system-%{+yyyy.MM.dd}"  
 when.contains:  
 fileset.module: "system"

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 31, 2018, 2:43pm UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505/2 "2018-12-31T14:43:31Z")

</div>

I wonder if the cause may be that the standard index template that is uploaded assumes that you do not change the index names and that you therefore have had dynamic mappings applied instead. If this is the case you should be able to copy the uploaded index template and change the index pattern that determines which indices it matches. This will however only apply to newly created indices.

---

<div class="post-metadata">

**Author:** ![ozmhsh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ozmhsh/32/49129_2.png) [@ozmhsh](https://discuss.elastic.co/u/ozmhsh)\
**Post date:** [December 31, 2018, 8:26pm UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505/3 "2018-12-31T20:26:57Z")

</div>

Can you explain a bit more about how the index template is prepared by filebeat? Based on what I read from the document, the fields.yml determines the field mapping in the template, but I found many fields.yml in the Filebeat directory, the root level, the \_meta directory in each module/fileset etc. I could not find the answer from the document (maybe I missed it).

One issue I am facing (and opened another thread but got no reply) is that @timestamp field is shown as text rather than date. I am sure I did something wrong (as in another environment it is shown as date) but could not figure out how to fix it.

---

<div class="post-metadata">

**Author:** ![JohnParson](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@JohnParson](https://discuss.elastic.co/u/JohnParson)\
**Post date:** [January 2, 2019, 6:23am UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505/4 "2019-01-02T06:23:36Z")

</div>

Thanks Christian, you were on the right track. If you create new index in filebeat.yml you also have to  
add `setup.template.name` and `setup.template.pattern`. This is actually documented in the official documentation but I missed it.

So it works now with this configuration:

> indices:  
> - index: "filebeat-%{[beat.version]}-nginx-%{+yyyy.MM.dd}"  
> setup.template.name: "filebeat-%{[beat.version]}"  
> setup.template.pattern: "filebeat-%{[beat.version]}-\*"  
> when.contains:  
> fileset.module: "nginx"

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 30, 2019, 6:23am UTC](https://discuss.elastic.co/t/filebeat-and-nginx-module-wrong-data-types-in-es/162505/5 "2019-01-30T06:23:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.