Problem deleting fields or renaming (from filebeat)

Hello there,

I'm configuring filebeat. before i used logstashforwarder.
In logstash i'm getting logs BUT, if i set filebeat to add a new fields, i receive it like field.nameIchoose
If i use mutate filter to rename that field it doesn't work. Moreover if i try to delete it i can't.

It means that fields, created in filebeat can't be deleted?

Thank you all

You're not giving your configuration so we can only guess, and my guess is that you're not using the current field reference syntax for nested fields. Read more at https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references.

To get your fields at the top level from the start, use the fields_under_root configuration option in Filebeat.

Here is my config
There's something i'm doing wrong, because it avoids some configurations, such as document_type..

filebeat:

List of prospectors to fetch data.

prospectors:

paths:

  • /opt/tomcat/logs/tomcat.*.log
    input_type: log
    document_type: filebeat
    scan_frequency: 60s
    fields:
    host: xxxx
    fields_under_root: true
    multiline:
    #pattern: "(^%{TIMESTAMP_ISO8601})|(^%{IPV4})"
    pattern: "()|(^(?:[0-9]{1,3}.){3}[0-9]{1,3}$)"
    negate: true
    match: after
    max_lines: 200
    timeout: 4s
    tail_files: true
    registry_file: /var/lib/filebeat/registry

output:
logstash:
hosts: ["x.x.x.x:5044"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB

Please format your config as code so the indentation isn't stripped away. There's a toolbar button for that.

filebeat:
 # List of prospectors to fetch data.
 prospectors:
 -
  paths:
  - /opt/tomcat/logs/tomcat.*.log
 input_type: log
 document_type: filebeat
 scan_frequency: 60s
 fields:
  host: xxxx
 fields_under_root: true
 multiline:
    pattern: (^([0-9]{4})(-?)(1[0-2]|0[1-9])\2(3[01]|0[1-9]|[12][0-9])\s(2[0-3]|[01][0-9]):?([0-5][0-9]):?([0-5][0-9])$)|(^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$)
    negate: true
    match: after
    max_lines: 200
    timeout: 4s

 tail_files: true
 registry_file: /var/lib/filebeat/registry

output:
   logstash:
    hosts: ["x.x.x.x:5044"]
shipper:
 logging:
   files:
     rotateeverybytes: 10485760 # = 10MB

The indentation is off. input_type and document_type should be on the same level as paths. See the example in the docs.

filebeat:
  # List of prospectors to fetch data.
  prospectors:
  -
    paths:
      - /opt/tomcat/logs/tomcat.*.log
    input_type: log
    document_type: filebeat
    scan_frequency: 60s
    fields:
      host: aleadigital
    fields_under_root: true

Hi Magnus,

right, that was the problem. Thanks