# Filebeat: failed to parse field \[user\_agent.version\] of type \[date\]

**URL:** https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124
**Category:** Elasticsearch
**Created:** [March 6, 2023, 8:59pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124 "2023-03-06T20:59:49Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 8:59pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/1 "2023-03-06T20:59:49Z")

</div>

This begins as a filebeat issue but I think it's now a matter of elasticsearch index.

I'm seeing repeated messages like this in our logging. I can see this is related to the nginx module but I'm unsure how to go about fixing it (custom mapping, enable dynamic mapping, edit module, ?).

I know it's related to the nginx filebeat module. This entry seems to be key:

```auto
{"type":"mapper_parsing_exception","reason":"failed to parse field [user_agent.version] of type [date] in document

```

full log entry:

```auto
Mar 6 07:22:27 rgo032 filebeat[4914]: 2023-03-06T07:22:27.103-0800#011WARN#011[elasticsearch]#011elasticsearch/client.go:414#011Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2023, time.March, 6, 7, 22, 26, 929462784, time.Local), Meta:{"pipeline":"filebeat-7.17.8-nginx-access-pipeline"}, Fields:{"agent":{"ephemeral_id":"2be6b958-12f3-4e22-ad23-d5a9021edd15","hostname":"rgo032","id":"3b97f44c-65e6-4d28-ab3f-a4e80ff361fb","name":"rgo032","type":"filebeat","version":"7.17.8"},"ecs":{"version":"1.12.0"},"event":{"dataset":"nginx.access","module":"nginx","timezone":"-08:00"},"fileset":{"name":"access"},"host":{"name":"rgo032"},"input":{"type":"log"},"log":{"fi
    le":{"path":"/var/log/nginx/access.log"},"offset":2643238},"message":"192.168.1.15 - stevans [06/Mar/2023:07:22:26 -0800] \"PROPFIND /remote.php/dav/files/stevans/ HTTP/1.1\" 207 331 \"-\" \"Mozilla/5.0 (Macintosh) mirall/2.10.0 (build 6519) (ownCloud, osx-21.1.0 ClientArchitecture: x86_64 OsArchitecture: x86_64)\"","service":{"type":"nginx"}}, Private:file.State{Id:"native::12845627-64769", PrevId:"", Finished:false, Fileinfo:(*os.fileStat)(0xc000a14a90), Source:"/var/log/nginx/access.log", Offset:2643473, Timestamp:time.Date(2023, time.March, 6, 0, 0, 3, 582773674, time.Local), TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0xc4023b, Device:0xfd01}, IdentifierName:"native"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [user_agent.version] of type [date] in document with id 'GmmEt4YBeNusebpST5KS'. Preview of field's value: '2.10.0'","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2.10.0] with format [strict_date_optional_time||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"Failed to parse with all enclosed parsers"}}}, dropping event!

```

/var/log/nginx/access.log:

```auto
    192.168.1.15 - stevans [06/Mar/2023:07:21:24 -0800] "PROPFIND /remote.php/dav/files/stevans/ HTTP/1.1" 207 331 "-" "Mozilla/5.0 (Macintosh) mirall/2.10.0 (build 6519) (ownCloud, osx-21.1.0 ClientArchitecture: x86_64 OsArchitecture: x86_64)"

```

I was able to do some more investigation here and found a difference between our two indices:

```auto
GET filebeat-7.17.8/_mapping/field/user_agent.version
...
  "filebeat-7.17.8" : {
    "mappings" : {
      "user_agent.version" : {
        "full_name" : "user_agent.version",
        "mapping" : {
          "version" : {
            "type" : "date"
...

```

```auto
GET filebeat-7.17.9/_mapping/field/user_agent.version
...
  "filebeat-7.17.9" : {
    "mappings" : {
      "user_agent.version" : {
        "full_name" : "user_agent.version",
        "mapping" : {
          "version" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
...

```

If I understand this correctly, the filebeat-7.17.8 index is incorrect as the `"type" : "date"` doesn't seem right compared to the filebeat-7.17.9 index. Did this happen because of a few funky log entries?

Is the only way to fix this is reindexing or dropping the index?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 6, 2023, 9:47pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/2 "2023-03-06T21:47:57Z")

</div>

Hi @mevan

> [@mevan](#):
>
> Did this happen because of a few funky log entries?

Yes... probably ... turns out `user_agent.version` is not a "defined / mapped" field you could add it to the `fields.yml` file in the filebeat as a `keyword` and then run `setup` to avoid it in the future.

> [@mevan](#):
>
> Is the only way to fix this is reindexing or dropping the index?

Yes... no fixing / changing a mapping on an existing index.

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 10:11pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/3 "2023-03-06T22:11:01Z")

</div>

Hello and thanks.

I found the file `/etc/filebeat/fields.yml` and the contents contain:

```auto
- key: nginx
  title: "Nginx"
  description: >
    Module for parsing the Nginx log files.
  ...
              - name: user_agent
              type: group
              fields:
                - name: device
                  type: alias
                  path: user_agent.device.name
                  migration: true
                - name: name
                  type: alias
                  path: user_agent.name
                  migration: true
                - name: os
                  type: alias
                  path: user_agent.os.full_name
                  migration: true
                - name: os_name
                  type: alias
                  path: user_agent.os.name
                  migration: true
                - name: original
                  type: alias
                  path: user_agent.original
                  migration: true

```

Would I add something like this after `-name: original`:

```
            - name: original
              type: alias
              path: user_agent.original
              migration: true

            - name: version
              type: keyword
              path: user_agent.version
              migration: true

```

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 6, 2023, 10:13pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/4 "2023-03-06T22:13:52Z")

</div>

> [@mevan](#):
>
> ```auto
> - name: version
> type: keyword
> path: user_agent.version
> migration: true
> 
> ```

LGTM! give it a try ... then run `setup` and take a look at the template (you might need to delete the templates to force overwrite

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 10:21pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/5 "2023-03-06T22:21:09Z")

</div>

I ran this and got an error. My guess is I'd need to set `setup.ilm.overwrite: true` in `/etc/elasticsearch/elasticsearch.yml` and restart for this to take?

```auto
cd /etc/filebeat
# filebeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/kibana: failed to import Kibana index pattern: 1 error: error loading index pattern: returned 403 to import file: Unable to bulk_create index-pattern: <nil>. Response: {"statusCode":403,"error":"Forbidden","message":"Unable to bulk_create index-pattern"}

```

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 10:47pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/6 "2023-03-06T22:47:05Z")

</div>

I put `setup.ilm.overwrite: true` in `/etc/filebeat/filebeat.yml` and then ran `filebeat setup -e`. I'm getting a different error now...

```auto

2023-03-06T14:45:05.077-0800	INFO	template/load.go:123	Template with name "filebeat-7.17.8" loaded.
2023-03-06T14:45:05.077-0800	INFO	[index-management]	idxmgmt/std.go:297	Loaded index template.
2023-03-06T14:45:05.077-0800	INFO	[index-management.ilm]	ilm/std.go:108	Index alias is not checked as setup.ilm.check_exists is disabled
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2023-03-06T14:45:05.078-0800	INFO	kibana/client.go:180	Kibana url: http://localhost:5601
2023-03-06T14:45:06.460-0800	INFO	kibana/client.go:180	Kibana url: http://localhost:5601
2023-03-06T14:45:06.528-0800	ERROR	instance/beat.go:1026	Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/kibana: failed to import Kibana index pattern: 1 error: error loading index pattern: returned 403 to import file: Unable to bulk_create index-pattern: <nil>. Response: {"statusCode":403,"error":"Forbidden","message":"Unable to bulk_create index-pattern"}
Exiting: Error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/kibana: failed to import Kibana index pattern: 1 error: error loading index pattern: returned 403 to import file: Unable to bulk_create index-pattern: <nil>. Response: {"statusCode":403,"error":"Forbidden","message":"Unable to bulk_create index-pattern"}

```

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 11:01pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/7 "2023-03-06T23:01:30Z")

</div>

These are the privileges given to the `filebeat_writer` role. I'm assuming this is the user/role invoked with `setup`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e55a211d74f67c821db1a710c8d5b9a08f2c4de9.png)

And the full output from `filebeat setup -e`

> **[2023-03-06T14:56:59.035-0800	INFO	instance/beat.go:697	Home path:...](https://pastebin.com/j5Qn3dKN)**
>
> Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 6, 2023, 11:08pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/8 "2023-03-06T23:08:29Z")

</div>

The setup Role is different... I did not understand that you did not have a setup role, the writer roles does not have the needed to do all the kibana stuff but that is really not needed.

Loading index pattern does not really matter...

```auto
2023-03-06T14:45:05.077-0800	INFO	template/load.go:123	Template with name "filebeat-7.17.8" loaded.
2023-03-06T14:45:05.077-0800	INFO	[index-management]	idxmgmt/std.go:297	Loaded index template.
2023-03-06T14:45:05.077-0800	INFO	[index-management.ilm]	ilm/std.go:108	Index alias is not checked as setup.ilm.check_exists is disabled
Index setup finished. 

```

`Loaded index template.` \<\<\< This is what is important.

That should be enough did you check the loaded template and see if the new field is there?

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 6, 2023, 11:12pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/9 "2023-03-06T23:12:49Z")

</div>

> [@stephenb](#):
>
> check the loaded template

Do I do that the same way I showed above? Like this:

```auto
GET filebeat-7.17.8/_mapping/field/user_agent.version
GET filebeat-7.17.9/_mapping/field/user_agent.version

```

They are still showing the same thing.

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 6, 2023, 11:40pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/10 "2023-03-06T23:40:06Z")

</div>

Ohh darn I did not look at the fields closely I do not think that will work ... let me look and get back with a better answer.

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 7, 2023, 12:13am UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/11 "2023-03-07T00:13:00Z")

</div>

Ok `user_agent.version` is already a defined field / mapping so no need to edit the fields.

What this tells me is that you have not been running `filebeat setup -e` which sets up the templates and that is most likely your issue.

To see the current mapping look at  
`curl localhost:9200/_index_template/filebeat-7.17.3`

It easier is to use `jq`

```auto
curl localhost:9200/_index_template/filebeat-7.17.3 | jq '[.index_templates[].index_template.template.mappings.properties.user_agent]'
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 343k 100 343k 0 0 10.7M 0 --:--:-- --:--:-- --:--:-- 13.9M
[
  {
    "properties": {
      "original": {
        "ignore_above": 1024,
        "type": "keyword",
        "fields": {
          "text": {
            "type": "match_only_text"
          }
        }
      },
      "os": {
        "properties": {
          "full_name": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "kernel": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "name": {
            "ignore_above": 1024,
            "type": "keyword",
            "fields": {
              "text": {
                "type": "match_only_text"
              }
            }
          },
          "type": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "family": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "version": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "platform": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "full": {
            "ignore_above": 1024,
            "type": "keyword",
            "fields": {
              "text": {
                "type": "match_only_text"
              }
            }
          }
        }
      },
      "name": {
        "ignore_above": 1024,
        "type": "keyword"
      },
      "device": {
        "properties": {
          "name": {
            "ignore_above": 1024,
            "type": "keyword"
          }
        }
      },
      "version": {
        "ignore_above": 1024,
        "type": "keyword"
      }
    }
  }
]

```

I think you are not running setup, did you run setup before?

so now to get a new index with the right mapping _ **after** _ you see the right mapping is

`POST filebeat-7.17.9/_rollover`

Then try

```auto
$ curl localhost:9200/filebeat-7.17.9/_mapping/field/user_agent.version | jq % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 169 100 169 0 0 18049 0 --:--:-- --:--:-- --:--:-- 84500
{
  "filebeat-7.17.9-2023.03.06-000001": {
    "mappings": {
      "user_agent.version": {
        "full_name": "user_agent.version",
        "mapping": {
          "version": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![mevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mevan/32/101086_2.png) [@mevan](https://discuss.elastic.co/u/mevan)
#### Post date: [March 7, 2023, 2:01pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/12 "2023-03-07T14:01:19Z")

</div>

Thank you for all the help here. I think we have come full circle to what I noted when I opened this issue (see above). That is, I have two indices `filebeat-7.17.8` and `filebeat-7.17.9`. Below is what I'm seeing for the mappings. I have not applied any `POST` or `PUT` against either index. In fact, the only potentially invasive thing I've done since working on this is running `filebeat setup` as noted earlier in this discussion.

You asked:

> did you run setup before?

...and I'm unsure. I'm looking at some notes from a previous engineer and I see this was installed via Ansible and it's noted "Filebeat can create the index." Until I have time to look at the Ansible code I'll assume `filebeat setup` may not have occurred when this elasticsearch became operational.

Does this issue reflect the lack of loading the "[recommended index template](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html#setup-assets)?"

As this has been very time consuming and complex, all for remedying what may have been an improper installation, I'm considering dropping the indices and starting over. I can't find a document that guides this but I'm assuming it could be accomplished by just deleting each index. Do you have anything to recommend for this course? Would you advise against it?

* * *

```auto
# curl -s --user elastic:foobar https://elasticsearch.vvv.io:9200/filebeat-7.17.8/_mapping/field/user_agent.version | jq
{
  "filebeat-7.17.8": {
    "mappings": {
      "user_agent.version": {
        "full_name": "user_agent.version",
        "mapping": {
          "version": {
            "type": "date"
          }
        }
      }
    }
  }
}

```

```auto
# curl -s --user elastic:foobar https://elasticsearch.vvv.io:9200/filebeat-7.17.9/_mapping/field/user_agent.version | jq
{
  "filebeat-7.17.9": {
    "mappings": {
      "user_agent.version": {
        "full_name": "user_agent.version",
        "mapping": {
          "version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }

```

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [March 7, 2023, 2:43pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/13 "2023-03-07T14:43:29Z")

</div>

Hi @mevan

Apologies, but I do not know how you got in this configuration in the first place.

So yes, if the data is not important I would delete both of those indices and start over.

Neither of those mappings are correct.

Also, are you using the nginx module? Hopefully you are.

I would clean up those indices and then I would follow the [quick start guide](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html) and follow those steps.

That probably would have helped you get started in the correct way in the first place.

That quick start guide example even uses the nginx module

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 4, 2023, 2:43pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-parse-field-user-agent-version-of-type-date/327124/14 "2023-04-04T14:43:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
