marcmga
(marc)
February 11, 2016, 4:14pm
1
Hello there,
I'm configuring filebeat. before i used logstashforwarder.
In logstash i'm getting logs BUT, if i set filebeat to add a new fields, i receive it like field.nameIchoose
If i use mutate filter to rename that field it doesn't work. Moreover if i try to delete it i can't.
It means that fields, created in filebeat can't be deleted?
Thank you all
You're not giving your configuration so we can only guess, and my guess is that you're not using the current field reference syntax for nested fields. Read more at https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references .
To get your fields at the top level from the start, use the fields_under_root
configuration option in Filebeat.
marcmga
(marc)
February 12, 2016, 9:56am
3
Here is my config
There's something i'm doing wrong, because it avoids some configurations, such as document_type..
filebeat:
List of prospectors to fetch data.
prospectors:
paths:
/opt/tomcat/logs/tomcat.*.log
input_type: log
document_type: filebeat
scan_frequency: 60s
fields:
host: xxxx
fields_under_root: true
multiline:
#pattern: "(^%{TIMESTAMP_ISO8601})|(^%{IPV4})"
pattern: "()|(^(?:[0-9]{1,3}.){3}[0-9]{1,3}$)"
negate: true
match: after
max_lines: 200
timeout: 4s
tail_files: true
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["x.x.x.x:5044"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB
Please format your config as code so the indentation isn't stripped away. There's a toolbar button for that.
marcmga
(marc)
February 12, 2016, 12:46pm
5
filebeat:
# List of prospectors to fetch data.
prospectors:
-
paths:
- /opt/tomcat/logs/tomcat.*.log
input_type: log
document_type: filebeat
scan_frequency: 60s
fields:
host: xxxx
fields_under_root: true
multiline:
pattern: (^([0-9]{4})(-?)(1[0-2]|0[1-9])\2(3[01]|0[1-9]|[12][0-9])\s(2[0-3]|[01][0-9]):?([0-5][0-9]):?([0-5][0-9])$)|(^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$)
negate: true
match: after
max_lines: 200
timeout: 4s
tail_files: true
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["x.x.x.x:5044"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB
The indentation is off. input_type
and document_type
should be on the same level as paths
. See the example in the docs .
filebeat:
# List of prospectors to fetch data.
prospectors:
-
paths:
- /opt/tomcat/logs/tomcat.*.log
input_type: log
document_type: filebeat
scan_frequency: 60s
fields:
host: aleadigital
fields_under_root: true
marcmga
(marc)
February 15, 2016, 11:29am
7
Hi Magnus,
right, that was the problem. Thanks