Runtime: out of memory Multiline filebeats

Elastic Stack Beats
1

Hi folks,

I need help to solve an Out of memory filebeat multiline trouble, following my info about configuration and error. Thanks you for your availability.

Version: 7.4.0
Server: Windows Server 2008
Memory: 64 GB
System type: 32-bit

Filebeat.yml:

filebeat.inputs:

        - paths:
            - E:\sba80\siebsrvr\log\WfProcMgr*.log
          fields_under_root: true
          fields:
            type: cdi_ws_outbound
            exclude_files: [".zip"]
            ignore_older: 8h
            multiline.pattern: '^EAITransport(.*?)O conversor de transporte EAI foi inicializado(.*?)$'
            multiline.negate: true
            multiline.match: after
            multiline.flush_pattern: '^(.*?)EAITransportPerf(.*?)Sent Request(.*?)$'
            multiline.timeout: 5m

        filebeat.config.modules:
            path: ${path.config}/modules.d/*.yml
            reload.enabled: false
            
        setup.template.settings:
          index.number_of_shards: 1
           
        setup.kibana:
                          
        output.logstash:
            hosts: ["wllpx70a:5040", "wllpx70b:5040", "wllpx70c:5040"]
          loadbalance: true
          worker: 2

        processors:
          - add_host_metadata: ~
          - add_cloud_metadata: ~

Error log:

runtime: out of memory: cannot allocate 5382144-byte block (1193017344 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x1e84675, 0xd)
	/usr/local/go/src/runtime/panic.go:617 +0x64
runtime.largeAlloc(0x521ebe, 0x101, 0x40aae1f0)
	/usr/local/go/src/runtime/malloc.go:1057 +0x10f
runtime.mallocgc.func1()
	/usr/local/go/src/runtime/malloc.go:950 +0x39
runtime.systemstack(0x272e70)
	/usr/local/go/src/runtime/asm_386.s:396 +0x53
runtime.mstart()
	/usr/local/go/src/runtime/proc.go:1153

goroutine 80 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_386.s:357 fp=0x64c31bbc sp=0x64c31bb8 pc=0x454100
runtime.mallocgc(0x521ebe, 0x1bae6e0, 0x88ba2201, 0x64c31c28)
	/usr/local/go/src/runtime/malloc.go:949 +0x65b fp=0x64c31c10 sp=0x64c31bbc pc=0x40b89b
runtime.makeslice(0x1bae6e0, 0x521ebe, 0x521ebe, 0x40)
	/usr/local/go/src/runtime/slice.go:49 +0x4f fp=0x64c31c24 sp=0x64c31c10 pc=0x43f69f
bytes.makeSlice(0x521ebe, 0x0, 0x0, 0x0)
	/usr/local/go/src/bytes/buffer.go:232 +0x61 fp=0x64c31c38 sp=0x64c31c24 pc=0x4e0b61
bytes.(*Buffer).grow(0x191d1da0, 0x2, 0x0)
	/usr/local/go/src/bytes/buffer.go:145 +0x12a fp=0x64c31c60 sp=0x64c31c38 pc=0x4e05aa
bytes.(*Buffer).WriteString(0x191d1da0, 0x5cc46400, 0x2, 0x0, 0x64c31ca8, 0x64c31cac)
	/usr/local/go/src/bytes/buffer.go:187 +0xbc fp=0x64c31c74 sp=0x64c31c60 pc=0x4e090c
encoding/json.(*encodeState).string(0x191d1da0, 0x5cc46400, 0x31, 0x5cc46401)
	/usr/local/go/src/encoding/json/encode.go:897 +0xce fp=0x64c31ca4 sp=0x64c31c74 pc=0x57ce1e
encoding/json.stringEncoder(0x191d1da0, 0x1bae420, 0x7f35143c, 0x198, 0x100)
	/usr/local/go/src/encoding/json/encode.go:611 +0x93 fp=0x64c31d04 sp=0x64c31ca4 pc=0x57ad53
encoding/json.structEncoder.encode(0x146c6b40, 0x7, 0x8, 0x191d1da0, 0x1d830a0, 0x7f351428, 0x199, 0x100)
	/usr/local/go/src/encoding/json/encode.go:660 +0x27a fp=0x64c31d58 sp=0x64c31d04 pc=0x57b4ba
encoding/json.structEncoder.encode-fm(0x191d1da0, 0x1d830a0, 0x7f351428, 0x199, 0x1d80100)
	/usr/local/go/src/encoding/json/encode.go:631 +0x60 fp=0x64c31d7c sp=0x64c31d58 pc=0x585690
encoding/json.arrayEncoder.encode(0x1451efc0, 0x191d1da0, 0x1b10de0, 0x26177260, 0x97, 0x35b0100)
	/usr/local/go/src/encoding/json/encode.go:787 +0xad fp=0x64c31da4 sp=0x64c31d7c pc=0x57c36d
encoding/json.arrayEncoder.encode-fm(0x191d1da0, 0x1b10de0, 0x26177260, 0x97, 0x100)
	/usr/local/go/src/encoding/json/encode.go:780 +0x52 fp=0x64c31dc0 sp=0x64c31da4 pc=0x5857b2
encoding/json.sliceEncoder.encode(0x1474f448, 0x191d1da0, 0x1b10de0, 0x26177260, 0x97, 0x1b10100)
	/usr/local/go/src/encoding/json/encode.go:761 +0x7a fp=0x64c31de0 sp=0x64c31dc0 pc=0x57c06a
encoding/json.sliceEncoder.encode-fm(0x191d1da0, 0x1b10de0, 0x26177260, 0x97, 0x64c30100)
	/usr/local/go/src/encoding/json/encode.go:756 +0x52 fp=0x64c31dfc sp=0x64c31de0 pc=0x585752
encoding/json.(*encodeState).reflectValue(0x191d1da0, 0x1b10de0, 0x26177260, 0x97, 0x1c70100)
	/usr/local/go/src/encoding/json/encode.go:334 +0x6a fp=0x64c31e14 sp=0x64c31dfc pc=0x57938a
encoding/json.(*encodeState).marshal(0x191d1da0, 0x1b10de0, 0x26177260, 0x14870100, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:306 +0xd7 fp=0x64c31e34 sp=0x64c31e14 pc=0x578eb7
encoding/json.(*Encoder).Encode(0x64c31ecc, 0x1b10de0, 0x26177260, 0x26177260, 0x2591ee68)
	/usr/local/go/src/encoding/json/stream.go:200 +0x5d fp=0x64c31e78 sp=0x64c31e34 pc=0x5836cd
github.com/elastic/beats/filebeat/registrar.writeTmpFile(0x1461c910, 0x42, 0x180, 0x7f22e000, 0x3e20, 0x3e20, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:424 +0x2da fp=0x64c31ef8 sp=0x64c31e78 pc=0x1301a1a
github.com/elastic/beats/filebeat/registrar.(*Registrar).writeRegistry(0x147db5c0, 0x2c, 0x64c31f54)
	/go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:392 +0xc1 fp=0x64c31f40 sp=0x64c31ef8 pc=0x13015d1
github.com/elastic/beats/filebeat/registrar.(*Registrar).flushRegistry(0x147db5c0)
	/go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:373 +0x25 fp=0x64c31f60 sp=0x64c31f40 pc=0x1301475
github.com/elastic/beats/filebeat/registrar.(*Registrar).Run(0x147db5c0)
	/go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:302 +0x1f5 fp=0x64c31fe8 sp=0x64c31f60 pc=0x1300f15
runtime.goexit()
	/usr/local/go/src/runtime/asm_386.s:1321 +0x1 fp=0x64c31fec sp=0x64c31fe8 pc=0x455a71
created by github.com/elastic/beats/filebeat/registrar.(*Registrar).Start
	/go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:272 +0x10f
2

When number of worker was 1, I had error too.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.