The processors only filter part of message

Elastic Stack Beats
1

Hi, i want to use filebeat send message to elasticsearch directly. And i use processors as a filter to drop some useless fields. But when i check message from kibana only half messages drop fields. And i only run one filebeat. so is there have some solutions to let all messages drop fields.

here is my filebeat.yml

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - h:\log\*
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
output.elasticsearch:
  hosts: ["localhost:9200"]
processors:
- drop_fields:
    fields: ["offset", "_score", "_version","beat"]

And the pic of my kibana:

捕获
捕获.PNG1431×718 39.6 KB

what's more, can i decode the json in the "message"?
i have try- decode_json_fields: but it didn't work

2

_score and _version are internal Elasticsearch fields, so do not exist in Filebeat and can not be dropped.

3

but what about offset and beat it is in the _source and why in some message i could drop it but the other not

4

How many filebeat instances have you running? Can you not exclude the beat field?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.