# Filebeat and fields with dots in name
**URL:** https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620
**Category:** Beats
**Tags:** filebeat
**Created:** [September 9, 2019, 7:37am UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620 "2019-09-09T07:37:44Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@marcomusso](https://discuss.elastic.co/u/marcomusso)
#### Post date: [September 9, 2019, 7:37am UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/1 "2019-09-09T07:37:44Z")
Hi all,
I found that the issue discussed in:
> [@Filebeat can't work with a dot in field name](https://discuss.elastic.co/t/filebeat-cant-work-with-a-dot-in-field-name/84758):
>
> Hi all, I am trying to insert a log in ElasticSearch whose structure is JSON. Some fields have a dot in their name, and because of this Filebeat can't insert the data into Elasticsearch. The fail is: WARN Can not index event (status=400): {"type":"mapper\_parsing\_exception","reason":"failed to parse","caused\_by":{"type":"class\_cast\_exception","reason":null}} The data which I want to insert in Elasticsearch is: {"fields":{"data.1":1,"data.2":0},"name":"hostname","tags":{"host":"hostname"},"tim…
is still relevant in 2019 using the default audit output in JSON (ref: log4j2.properties for Elasticsearch 7.x) which is:
```auto
appender.audit_rolling.layout.pattern = {\
"@timestamp":"%d{ISO8601}"\
%varsNotEmpty{, "node.name":"%enc{%map{node.name}}{JSON}"}\
%varsNotEmpty{, "node.id":"%enc{%map{node.id}}{JSON}"}\
%varsNotEmpty{, "host.name":"%enc{%map{host.name}}{JSON}"}\
%varsNotEmpty{, "host.ip":"%enc{%map{host.ip}}{JSON}"}\
%varsNotEmpty{, "event.type":"%enc{%map{event.type}}{JSON}"}\
%varsNotEmpty{, "event.action":"%enc{%map{event.action}}{JSON}"}\
%varsNotEmpty{, "user.name":"%enc{%map{user.name}}{JSON}"}\
%varsNotEmpty{, "user.run_by.name":"%enc{%map{user.run_by.name}}{JSON}"}\
%varsNotEmpty{, "user.run_as.name":"%enc{%map{user.run_as.name}}{JSON}"}\
%varsNotEmpty{, "user.realm":"%enc{%map{user.realm}}{JSON}"}\
%varsNotEmpty{, "user.run_by.realm":"%enc{%map{user.run_by.realm}}{JSON}"}\
%varsNotEmpty{, "user.run_as.realm":"%enc{%map{user.run_as.realm}}{JSON}"}\
%varsNotEmpty{, "user.roles":%map{user.roles}}\
%varsNotEmpty{, "origin.type":"%enc{%map{origin.type}}{JSON}"}\
%varsNotEmpty{, "origin.address":"%enc{%map{origin.address}}{JSON}"}\
%varsNotEmpty{, "realm":"%enc{%map{realm}}{JSON}"}\
%varsNotEmpty{, "url.path":"%enc{%map{url.path}}{JSON}"}\
%varsNotEmpty{, "url.query":"%enc{%map{url.query}}{JSON}"}\
%varsNotEmpty{, "request.method":"%enc{%map{request.method}}{JSON}"}\
%varsNotEmpty{, "request.body":"%enc{%map{request.body}}{JSON}"}\
%varsNotEmpty{, "request.id":"%enc{%map{request.id}}{JSON}"}\
%varsNotEmpty{, "action":"%enc{%map{action}}{JSON}"}\
%varsNotEmpty{, "request.name":"%enc{%map{request.name}}{JSON}"}\
%varsNotEmpty{, "indices":%map{indices}}\
%varsNotEmpty{, "opaque_id":"%enc{%map{opaque_id}}{JSON}"}\
%varsNotEmpty{, "x_forwarded_for":"%enc{%map{x_forwarded_for}}{JSON}"}\
%varsNotEmpty{, "transport.profile":"%enc{%map{transport.profile}}{JSON}"}\
%varsNotEmpty{, "rule":"%enc{%map{rule}}{JSON}"}\
%varsNotEmpty{, "event.category":"%enc{%map{event.category}}{JSON}"}\
}%n
```
will generate a document like this:
`{"@timestamp":"2019-08-28T07:20:26,164", "node.name":"dev-data-4-instance-1", "node.id":"ZJl7IaozQNSe3bvvfQk2iw", "event.type":"transport", " **event.action**":"access_granted", " user.name ":"logstash", "user.realm":"native", "user.roles":["logstash"], " origin.type ":"local_node", "origin.address":"10.66.41.223:9301", "request.id":"0OHMnG9GSRWzcAwrCgFDNg", "action":"indices:data/write/bulk[s][r]", "request.name":"BulkShardRequest", "indices":["myindex-7d-2019.08.28"]}`
Which is impossible to filter with a valid processor config (tested on 5.x and 6.x) like this:
```auto
processors:
- drop_event:
when:
- equals:
user.name: logstash
```
or
```auto
processors:
- drop_event.when.equals.user.name: 'logstash'
```
In the end I filtered at the source by putting in elasticsearch.yml:
```auto
xpack.security.audit.logfile.events.ignore_filters:
filter_unwanted_users:
users = [...]
```
But that of course will need a rolling restart and it's kind of less nice than just use filebeat (of course it has the advantage of not even writing to the disk).
---
### Author:  [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [September 17, 2019, 9:52pm UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/2 "2019-09-17T21:52:18Z")
Hi,
This syntax seemed to work for me in a quick test:
```auto
- drop_event:
when.equals:
"user.name": logstash
```
The quotes make the YAML parser pass that as a single key. Let me know if that doesn't work for you.
---
### Author:  [@marcomusso](https://discuss.elastic.co/u/marcomusso)
#### Post date: [September 18, 2019, 7:39am UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/3 "2019-09-18T07:39:16Z")
Thanks for the reply, I'm pretty sure I tried but it's worth retrying again, maybe I used single quotes? Can't remember right now. I'll let you know asap.
---
### Author:  [@marcomusso](https://discuss.elastic.co/u/marcomusso)
#### Post date: [September 18, 2019, 8:19am UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/4 "2019-09-18T08:19:22Z")
Well, this doesn't work for me (filebeat-5.6.11):
```auto
---
filebeat:
prospectors:
- input_type: log
paths:
- /var/log/elasticsearch/instance-1/*_audit.json
encoding: plain
fields_under_root: false
document_type: elastic-audit
scan_frequency: 10s
harvester_buffer_size: 16384
max_bytes: 10485760
processors:
- drop_event:
when.equals:
"user.name": kibana
```
---
### Author:  [@marcomusso](https://discuss.elastic.co/u/marcomusso)
#### Post date: [September 18, 2019, 9:26am UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/5 "2019-09-18T09:26:04Z")
It might also be something related to the puppet module and/or the yaml syntax.
The important thing is that the syntax with double quote is supported and working for you (and possibly for everyone getting here for the same problem).
I'm not sure right now that this is a widespread problem.
---
### Author:  [@marcomusso](https://discuss.elastic.co/u/marcomusso)
#### Post date: [September 18, 2019, 2:26pm UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/6 "2019-09-18T14:26:15Z")
We are following up this behavior on a support ticket, will update the discussion with our findings.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 16, 2019, 2:26pm UTC](https://discuss.elastic.co/t/filebeat-and-fields-with-dots-in-name/198620/7 "2019-10-16T14:26:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.