Metricbeat6.4.0 panic when add add_host_metadata in configfile

Elastic Stack Beats
1 
processors:
   - add_host_metadata:
     netinfo.enabled: false

metricbeat6.4.0 sometimes panic(random time:1hour or 1day), when add add_host_metadata in configfile, error like:

fatal error: concurrent map writes
/root/goproject/src/github.com/elastic/beats/metricbeat/main.go:34 +0x2f
/root/goproject/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xda
/root/goproject/src/github.com/elastic/beats/libbeat/processors/add_host_metadata/add_host_metadata.go:72
goroutine 90 [running]:
runtime.throw(0x27e3a8f, 0x15)
        /usr/local/go/src/runtime/panic.go:616 +0x81 fp=0xc4208359c8 sp=0xc4208359a8 pc=0x10f99a1
runtime.mapassign_faststr(0x26f5cc0, 0xc420aae0f0, 0x27c59c1, 0x2, 0x27c59c1)
        /usr/local/go/src/runtime/hashmap_fast.go:703 +0x3e9 fp=0xc420835a38 sp=0xc4208359c8 pc=0x10da7d9
github.com/elastic/beats/libbeat/common.MapStr.Put(0xc420aae120, 0x27c59bc, 0x7, 0x234d420, 0xc4208d3200, 0xc4204b323e, 0
xc4204b3270, 0x8, 0xc42011da80)
        /root/goproject/src/github.com/elastic/beats/libbeat/common/mapstr.go:174 +0xa6 fp=0xc420835ab8 sp=0xc420835a38 p
c=0x144fda6
github.com/elastic/beats/libbeat/processors/add_host_metadata.(*addHostMetadata).loadData(0xc4204882a0)
        /root/goproject/src/github.com/elastic/beats/libbeat/processors/add_host_metadata/add_host_metadata.go:91 +0x408
fp=0xc420835c78 sp=0xc420835ab8 pc=0x176d548
github.com/elastic/beats/libbeat/processors/add_host_metadata.(*addHostMetadata).Run(0xc4204882a0, 0xc4209f45c0, 0x0, 0xf
fffffffffffffff, 0xc420835cd8)
        /root/goproject/src/github.com/elastic/beats/libbeat/processors/add_host_metadata/add_host_metadata.go:72 +0x2b f
p=0xc420835c98 sp=0xc420835c78 pc=0x176d0eb
github.com/elastic/beats/libbeat/publisher/pipeline.(*program).Run(0xc420590720, 0xc4209f45c0, 0xc4209f45c0, 0x0, 0x0)
        /root/goproject/src/github.com/elastic/beats/libbeat/publisher/pipeline/processor.go:167 +0xad fp=0xc420835d30 sp
=0xc420835c98 pc=0x15ecd5d
github.com/elastic/beats/libbeat/publisher/pipeline.(*program).Run(0xc42030e000, 0xc4209f45c0, 0xc420dd54d0, 0x28746b0, 0
x0)
        /root/goproject/src/github.com/elastic/beats/libbeat/publisher/pipeline/processor.go:167 +0xad fp=0xc420835dc8 sp
=0xc420835d30 pc=0x15ecd5d
github.com/elastic/beats/libbeat/publisher/pipeline.(*client).publish(0xc42056a0c0, 0xbede12304b114f3a, 0x24abc43e5ab8, 0
x38a1740, 0x0, 0xc420c69e60, 0x0, 0x0)
        /root/goproject/src/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:84 +0x474 fp=0xc420835eb0 sp=0x
c420835dc8 pc=0x15e4644
github.com/elastic/beats/libbeat/publisher/pipeline.(*client).Publish(0xc42056a0c0, 0xbede12304b114f3a, 0x24abc43e5ab8, 0
x38a1740, 0x0, 0xc420c69e60, 0x0, 0x0)
        /root/goproject/src/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:63 +0x95 fp=0xc420835f08 sp=0xc
420835eb0 pc=0x15e41a5
github.com/elastic/beats/metricbeat/mb/module.PublishChannels.func1(0xc42056a180)
        /root/goproject/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:41 +0x125 fp=0xc420835fd8 sp=0xc4208
35f08 pc=0x1abbb35
        /root/goproject/src/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:63 +0x95 fp=0xc420835f08 sp=0xc
420835eb0 pc=0x15e41a5
github.com/elastic/beats/metricbeat/mb/module.PublishChannels.func1(0xc42056a180)
        /root/goproject/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:41 +0x125 fp=0xc420835fd8 sp=0xc4208
35f08 pc=0x1abbb35
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420835fe0 sp=0xc420835fd8 pc=0x1129b41
created by github.com/elastic/beats/metricbeat/mb/module.PublishChannels
        /root/goproject/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xda
2

Hi @zyshn :slight_smile:

It seems that you have found a race condition that isn't easily triggered, according to the frequency you were metioning. Will you like to file a new issue in https://github.com/elastic/beats/issues so that we can discuss there about the fix?

Thanks you for the thumbs up!

3

thank you.
the bug has fixed a week ago,i just found it in https://github.com/elastic/beats/commit/8fed0a9c013b26126888104024a572db63c4c81b.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.