IP field do not add every time

[leslie]

when config filebeat input , i use host metadata to add server ip as below

- add_host_metadata:
    netinfo.enabled: true
- rename:
    fields:
      - from: "host.ip"
        to: "ip"
    ignore_missing: true
    fail_on_error: false
- include_fields:
    fields: ["ip", "appid", "message","category"]

but the ip is missed sometimes, what should i do to promise the field? it happens on both centos and windows server(filebeat version 6.4.0)

[jsoriano (Jaime Soriano)]

Hi @leslie and welcome

When there is some problem collecting this network information it gets logged in filebeat logs, could you check these logs for something related?

Other thing that can be happening here is that filebeat keeps an internal cache for this host information, as this is something that doesn't change so frequently. It is updated every 5 minutes, could it be that filebeat started before network was configured in these hosts? In this case the information should appear at some moment.

[leslie]

hi @jsoriano ,
i set log level as 'DEBUG', and restart filebeat, the network infomation is still not got, and no error log found. here is part of log when restarting and publishing a message.

this problem appearance on my centos and windows servers. Is there other tips to figure out the problem?

2018-09-29T14:00:02.309+0800    INFO    [beat]  instance/beat.go:784    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-08-17T17:11:31+08:00","containerized":true,"hostname":"fat-××××××","ips":["127.0.0.1/8","::1/128","172.16.×××.×××/24","fe80::21be:××××:××××:××××/64","fe80::a9fe:××××:××××:××××/64"],"kernel_version":"3.10.0-693.el7.x86_64","mac_addresses":["00:50:××:××:××:××"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":4,"patch":1708,"codename":"Core"},"timezone":"CST","timezone_offset_sec":28800,"id":"ecdb7002c6df42f191××××××××××××××"}}}
...
2018-09-29T14:00:22.604+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.604+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.604+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 0, After: 0, Pending: 0
2018-09-29T14:00:22.604+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.604+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.604+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 0, After: 0, Pending: 0
2018-09-29T14:00:22.605+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10161/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10161/access.log, offset: 32632
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:548        File didn't change: /opt/logs/10161/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10166/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10166/access.log, offset: 64651
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:548        File didn't change: /opt/logs/10166/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10167/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10167/access.log, offset: 13361
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:548        File didn't change: /opt/logs/10167/access.log
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 3, After: 3, Pending: 0
2018-09-29T14:00:22.605+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.605+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.606+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10161/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10161/catalina.out, offset: 1839969
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:546        Harvester for file is still running: /opt/logs/10161/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10166/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10166/catalina.out, offset: 0
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:548        File didn't change: /opt/logs/10166/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /opt/logs/10167/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /opt/logs/10167/catalina.out, offset: 0
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:548        File didn't change: /opt/logs/10167/catalina.out
2018-09-29T14:00:22.606+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 3, After: 3, Pending: 0
2018-09-29T14:00:22.606+0800    DEBUG   [harvester]     log/log.go:102  End of file reached: /opt/logs/10161/catalina.out; Backoff now.
2018-09-29T14:00:22.607+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 0, After: 0, Pending: 0
2018-09-29T14:00:22.632+0800    DEBUG   [input] input/input.go:152      Run input
2018-09-29T14:00:22.632+0800    DEBUG   [input] log/input.go:174        Start next scan
2018-09-29T14:00:22.632+0800    DEBUG   [input] log/input.go:404        Check file for harvesting: /var/log/messages
2018-09-29T14:00:22.632+0800    DEBUG   [input] log/input.go:494        Update existing file for harvesting: /var/log/messages, offset: 41105
2018-09-29T14:00:22.632+0800    DEBUG   [input] log/input.go:546        Harvester for file is still running: /var/log/messages
2018-09-29T14:00:22.632+0800    DEBUG   [input] log/input.go:195        input states cleaned up. Before: 1, After: 1, Pending: 0
2018-09-29T14:00:24.606+0800    DEBUG   [multiline]     multiline/multiline.go:174      Multiline event flushed because timeout reached.
2018-09-29T14:00:24.606+0800    DEBUG   [publish]       pipeline/processor.go:308       Publish event: {
  "@timestamp": "2018-09-29T06:00:19.606Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "doc",
    "version": "6.4.0",
    "pipeline": "filebeat-6.4.0-customize-tomcat-log-"
  },
  "category": "tomcat-log",
  "beat": {
    "version": "6.4.0",
    "name": "fat-×××××",
    "hostname": "fat-×××××"
  },
  "host": {
    "name": "fat-×××××"
  },
  "appid": "10161",
  "message": "2018-09-29 14:00:19.086 ×××××××××××××××××××××××××× "
}

[leslie]

the problem may be caused by the rename process. If remove the process and change ip to host.ip in include fields, it works well.

Further more, if set rename process as:

- rename:
    fields:
      - from: "host.ip"
        to: "ip"
    ignore_missing: false
    fail_on_error: true

as a result, both of ip and host.ip may not in final result, and not error in filebeat log.