# Filebeats creating the incorrect datatype in index

**URL:** https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849
**Category:** Beats
**Tags:** filebeat
**Created:** [August 26, 2019, 11:13pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849 "2019-08-26T23:13:13Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![mbradsher](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@mbradsher](https://discuss.elastic.co/u/mbradsher)
#### Post date: [August 26, 2019, 11:13pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/1 "2019-08-26T23:13:13Z")

</div>

I find this perplexing but it seems as though filebeats has decided to take an input that is always a text object with no numerals in it and index it as a date object. As a result the filebeats data is rejected by ES because of date formatting issues.

How can someone make sure that the index is being properly created by Filebeats?

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 27, 2019, 5:52pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/2 "2019-08-27T17:52:56Z")

</div>

Hi @mbradsher and welcome,

Could you give more details about your problem? What version of filebeat are you using? What field is unexpectedly being converted to date?

---

<div class="post-metadata">

### Author: ![mbradsher](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@mbradsher](https://discuss.elastic.co/u/mbradsher)
#### Post date: [August 27, 2019, 8:42pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/3 "2019-08-27T20:42:16Z")

</div>

The version we are using 7.2.1. It takes two log types from within an ECS container. The one that is failing is posted here:

```
type: log
    enabled: true
    paths:
      - /dir1/dir2/dir3/dir4/%{var1}/%{var2}/%{var3}/*.log
      - /dir1/dir2/dir3/dir4/*/*/*/*.log
    multiline.pattern: ^\[
    multiline.match: after
    multiline.negate: true
    tags: ["index1"]
    fields:
      app_name: "myindex_1"
    processors:
      - dissect:
          tokenizer: "/%{}/%{}/%{}/%{}/%{var1}/%{var2}/%{var3}/%{var4}.log"
          field: "log.file.path"
          target_prefix: "myindex_"

```

Essentially var3 is a database table name but the index in elasticsearch is being created with a 'date' type. Then when a log file is created we get an illegal\_argument\_exception.

```
{"type":"illegal_argument_exception","reason":"failed to parse date field [<table name>] with format [strict_date_optional_time||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"date_time_parse_exception: Failed to parse with all enclosed parsers"}}}

```

Is this a case where we want to create an Elasticsearch index template or is there a simpler solution? I'm still very new to this tech stack.

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [August 28, 2019, 9:45am UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/4 "2019-08-28T09:45:33Z")

</div>

But what is the real name of `var3` and `<table name>`? I am asking to check if there is already a predefined field with this name formatted as date. Send it to me to a private message if you want to avoid making it public.

If there is a predefined field, you may consider to use another field that is not currently used.

If you want to completely ignore the predefined mappings and define your own, you can try to use the [`setup.template.fields` option](https://www.elastic.co/guide/en/beats/filebeat/7.3/configuration-template.html). You can find examples of complete `fields.yml` files by [downloading the beats code](https://github.com/elastic/beats/) and running `make fields` from a beat directory.

---

<div class="post-metadata">

### Author: ![mbradsher](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@mbradsher](https://discuss.elastic.co/u/mbradsher)
#### Post date: [September 5, 2019, 5:40pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/5 "2019-09-05T17:40:07Z")

</div>

Thank you for your help. The real value of var3 is 'stepid' and the issue only happens intermittently so I do not expect this is from a name collision. Instead I took your suggestion to be explicit about the mapping by using the experimental [setup.template.append\_fields](https://www.elastic.co/guide/en/beats/filebeat/7.3/configuration-template.html) property.

I've run into new problems but they are expressed on a new forum [here.](https://discuss.elastic.co/t/does-setup-template-append-fields-work-for-multifields/198125)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 3, 2019, 5:40pm UTC](https://discuss.elastic.co/t/filebeats-creating-the-incorrect-datatype-in-index/196849/6 "2019-10-03T17:40:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
