Add_process_metadata cannot parse pid in integer

Elastic Stack Beats
1

Hi all,

I got some weird error when using add_process_metadata in filebeat 7.10.2. It doesn't seem to like integer value.

Given following json input:

{ "level": "info", "pid": 1, "message": "test1", "tags": ["tag1", "tag2"] }

and filebeat config:

logging.level: debug
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
filebeat.inputs:
- type: log
  paths:
  - /tmp/test.log
  json.keys_under_root: false
  processors:
  - add_process_metadata:
      match_pids:
      - json.pid
      ignore_missing: false
output.console:
  pretty: true

I expected it to grab process information for pid 1. But filebeat actually threw following error:

2021-02-05T11:51:57.708Z DEBUG [processors] processing/processors.go:128 Fail to apply processor client{add_process_metadata=[match_pids=[json.pid process.ppid process.parent.pid process.parent.ppid], mappings={"container.id":"container.id","process.args":"process.args","process.executable":"process.executable","process.name":"process.name","process.pid":"process.pid","process.ppid":"process.ppid","process.start_time":"process.start_time","process.title":"process.title"}, ignore_missing=false, overwrite_fields=false, restricted_fields=false, host_path=/, cgroup_prefixes=[/kubepods /docker]]}: error applying add_process_metadata processor: cannot parse field 'json.pid' (not an integer or string)
2021-02-05T11:51:57.710Z DEBUG [processors] processing/processors.go:203

If I add a convert processor to convert json.pid to string and then looks up by the new field, it actually works....

- convert:
    fields:
    - {from: "json.pid", to: "pr.pid", type: "string"}

Not sure whether this is a bug.

Thanks

2

The example log you have shared misses the last closing }. When I tried running Filebeat with that line, I got an error. But after I fixed the log line, it was able to parse the event correctly.

Are you sure the event you are testing with is a valid new line delimited JSON?

3

Hi Noémi,

That is a copy and paste error. The log parsed fine, but it doesn't contain process metadata unless I changed the type of PID to string

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

6

Hi @Yin_Zhang, thanks for reporting!

This is indeed a bug. I've created an issue for it: add_process_metadata fails to parse pid from valid integer fields · Issue #26830 · elastic/beats · GitHub

And a fix is underway in add_process_metadata: Support different integer types for pid field by adriansr · Pull Request #26829 · elastic/beats · GitHub