# Filebeat throwing exception with increased value of max\_bytes

**URL:** https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400
**Category:** Beats
**Tags:** filebeat
**Created:** [May 29, 2017, 6:43am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400 "2017-05-29T06:43:23Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![sumit\_monga](https://avatars.discourse-cdn.com/v4/letter/s/a698b9/32.png) [@sumit\_monga](https://discuss.elastic.co/u/sumit_monga)
#### Post date: [May 29, 2017, 6:43am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/1 "2017-05-29T06:43:23Z")

</div>

I have a problem where I need to read a big xml file (say about 50 MB) through Filebeat and then send this whole content as a single event to logstash which will then use some filters such as xml , mutate etc and finally dump the json in elasticsearch. To do this I am using multiline with filebeat with increased values for multiline.max\_lines and also since the file is bigger than 10 MB, i m using about 100 MB for max\_bytes. However when i run filebeat i get the following exceptions :

```
       runtime: VirtualAlloc of 84803584 bytes failed with errno=487  
      fatal error: runtime: cannot map pages in arena address space  

      runtime stack:  
      runtime.throw(0x97485e, 0x30)  
              /usr/local/go/src/runtime/panic.go:566 +0x7f  
      runtime.sysMap(0x48d90000, 0x50e0000, 0x26fd01, 0xc617d8)  
              /usr/local/go/src/runtime/mem_windows.go:116 +0xf9  
      runtime.( mheap).sysAlloc(0xc50d40, 0x50e0000, 0x1)  
              /usr/local/go/src/runtime/malloc.go:407 +0x123  
      runtime.( mheap).grow(0xc50d40, 0x2870, 0x0)  
              /usr/local/go/src/runtime/mheap.go:726 +0x52  
      runtime.( mheap).allocSpanLocked(0xc50d40, 0x286d, 0x0)  
              /usr/local/go/src/runtime/mheap.go:630 +0x4d5  
      runtime.( mheap).alloc_m(0xc50d40, 0x286d, 0x0, 0x1, 0xc546000)  
              /usr/local/go/src/runtime/mheap.go:515 +0x116  
      runtime.( mheap).alloc.func1()  
              /usr/local/go/src/runtime/mheap.go:579 +0x3d  
      runtime.systemstack(0x26ff10)  
              /usr/local/go/src/runtime/asm_386.s:339 +0x88  
      runtime.( mheap).alloc(0xc50d40, 0x286d, 0x0, 0x3a90001, 0x3a9dac0)  
              /usr/local/go/src/runtime/mheap.go:580 +0x5a  
      runtime.largeAlloc(0x50da000, 0x451000, 0x44cd0000)  
              /usr/local/go/src/runtime/malloc.go:774 +0x94  
      runtime.mallocgc.func1()  
              /usr/local/go/src/runtime/malloc.go:669 +0x31  
      runtime.systemstack(0x1d4b4a00)  
              /usr/local/go/src/runtime/asm_386.s:323 +0x5e  
      runtime.mstart()  
              /usr/local/go/src/runtime/proc.go:1079  

```

below is the filebeat.yml

filebeat.yml

```
filebeat.prospectors:

- input_type: log
  paths:
    - D:\sdn_advanced\sdn_advanced.xml

  close_eof: true

  multiline.pattern: '^<?xml'

  multiline.negate: true

  multiline.match: after
  multiline.max_lines: 1000000
  
  #something around 100 MB
  max_bytes: 104857600  

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

logging:
  to_files: true
  files:
    path: D:/Softwares/Filebeat/logs
  level: debug
  selectors: ["*"]
```

---

<div class="post-metadata">

### Author: ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [May 29, 2017, 7:55am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/2 "2017-05-29T07:55:40Z")

</div>

That looks like you are getting out of memory. How much free memory do you have on the system and what is the total size of the files you are ingesting? Filebeat is going to potentially buffer thousands of events in memory for batching, so for extremely large events you probably need to drastically reduce the size of all queues.

---

<div class="post-metadata">

### Author: ![sumit\_monga](https://avatars.discourse-cdn.com/v4/letter/s/a698b9/32.png) [@sumit\_monga](https://discuss.elastic.co/u/sumit_monga)
#### Post date: [May 29, 2017, 8:39am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/3 "2017-05-29T08:39:11Z")

</div>

Yes this looks like an OOM issue. However I think there is sufficient memory available on the system . below is the output of systeminfo command .

Total Physical Memory: 8,098 MB  
Available Physical Memory: 1,995 MB  
Virtual Memory: Max Size: 16,195 MB  
Virtual Memory: Available: 8,074 MB

Also as per filebeat.yml shared above, I am having only a single file and I want this big xml into a single event. As per your comment, @tudor which configuration needs to be changed for reducing queue sizes

---

<div class="post-metadata">

### Author: ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [May 29, 2017, 9:15am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/4 "2017-05-29T09:15:42Z")

</div>

If you expect a single event, then it's maybe not the queue sizes. Can you run filebeat with `-d "*"` and post the log here? I'd be curious to see if it gets past multiline or not.

---

<div class="post-metadata">

### Author: ![sumit\_monga](https://avatars.discourse-cdn.com/v4/letter/s/a698b9/32.png) [@sumit\_monga](https://discuss.elastic.co/u/sumit_monga)
#### Post date: [May 29, 2017, 9:41am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/5 "2017-05-29T09:41:44Z")

</div>

@tudor attaching the log containing the error. This error occurs when the max\_bytes configured is around 100 MB . However it runs fine when the value for max\_bytes is default (10 MB).

runtime: VirtualAlloc of 165609472 bytes failed with errno=487  
fatal error: runtime: cannot map pages in arena address space

runtime stack:  
runtime.throw(0x97485e, 0x30)  
/usr/local/go/src/runtime/panic.go:566 +0x7f  
runtime.sysMap(0x50ee0000, 0x9df0000, 0xcfd01, 0xc617d8)  
/usr/local/go/src/runtime/mem\_windows.go:116 +0xf9  
runtime.(\*mheap).sysAlloc(0xc50d40, 0x9df0000, 0x1)  
/usr/local/go/src/runtime/malloc.go:407 +0x123  
runtime.(\*mheap).grow(0xc50d40, 0x4ef8, 0x0)  
/usr/local/go/src/runtime/mheap.go:726 +0x52  
runtime.(\*mheap).allocSpanLocked(0xc50d40, 0x4ef7, 0x40e054)  
/usr/local/go/src/runtime/mheap.go:630 +0x4d5  
runtime.(\*mheap).alloc\_m(0xc50d40, 0x4ef7, 0x0, 0x1, 0x18158000)  
/usr/local/go/src/runtime/mheap.go:515 +0x116  
runtime.(\*mheap).alloc.func1()  
/usr/local/go/src/runtime/mheap.go:579 +0x3d  
runtime.systemstack(0xcfe8c)  
/usr/local/go/src/runtime/asm\_386.s:339 +0x88  
runtime.(\*mheap).alloc(0xc50d40, 0x4ef7, 0x0, 0x33bf0001, 0x33bf9680)  
/usr/local/go/src/runtime/mheap.go:580 +0x5a  
runtime.largeAlloc(0x9dee000, 0x451000, 0x4907a000)  
/usr/local/go/src/runtime/malloc.go:774 +0x94  
runtime.mallocgc.func1()  
/usr/local/go/src/runtime/malloc.go:669 +0x31  
runtime.systemstack(0xc4bd40)  
/usr/local/go/src/runtime/asm\_386.s:323 +0x5e  
runtime.mstart()  
/usr/local/go/src/runtime/proc.go:1079

goroutine 42 [running]:  
runtime.systemstack\_switch()  
/usr/local/go/src/runtime/asm\_386.s:277 fp=0x227cf5e0 sp=0x227cf5dc  
runtime.mallocgc(0x9dee000, 0x0, 0x6512000, 0x4907a000)  
/usr/local/go/src/runtime/malloc.go:670 +0x7b2 fp=0x227cf648 sp=0x227cf5e0  
runtime.growslice(0x8ca960, 0x4907a000, 0x1f96000, 0x1f96000, 0x1f96001, 0x0, 0x0, 0x0)  
/usr/local/go/src/runtime/slice.go:126 +0x183 fp=0x227cf688 sp=0x227cf648  
os.(\*File).writeConsole(0x12e520d0, 0x2c19c0a9, 0xe08783, 0xe09f57, 0x2d9e82c, 0x0, 0x0)  
/usr/local/go/src/os/file\_windows.go:361 +0x5a8 fp=0x227cfaf8 sp=0x227cf688  
os.(\*File).write(0x12e520d0, 0x2a206000, 0x2d9e82c, 0x2da0000, 0x0, 0x0, 0x0)  
/usr/local/go/src/os/file\_windows.go:398 +0xc0 fp=0x227cfb30 sp=0x227cfaf8  
os.(\*File).Write(0x12e520d0, 0x2a206000, 0x2d9e82c, 0x2da0000, 0x2d9e82b, 0x0, 0x0)  
/usr/local/go/src/os/file.go:142 +0x6f fp=0x227cfb64 sp=0x227cfb30  
log.(\*Logger).Output(0x12e37b60, 0x4, 0x3ae00000, 0x2d9e801, 0x0, 0x0)  
/usr/local/go/src/log/log.go:166 +0x2c5 fp=0x227cfbdc sp=0x227cfb64  
[github.com/elastic/beats/libbeat/logp.send](http://github.com/elastic/beats/libbeat/logp.send)(0x4, 0x7, 0x955f41, 0x5, 0x95bf0d, 0xb, 0x227cfd40, 0x1, 0x1)  
/go/src/github.com/elastic/beats/libbeat/logp/log.go:85 +0x172 fp=0x227cfca8 sp=0x227cfbdc  
[github.com/elastic/beats/libbeat/logp.debugMessage](http://github.com/elastic/beats/libbeat/logp.debugMessage)(0x3, 0x958e76, 0x7, 0x95bf0d, 0xb, 0x227cfd40, 0x1, 0x1)  
/go/src/github.com/elastic/beats/libbeat/logp/log.go:76 +0xd5 fp=0x227cfce8 sp=0x227cfca8  
[github.com/elastic/beats/libbeat/logp.Debug](http://github.com/elastic/beats/libbeat/logp.Debug)(0x958e76, 0x7, 0x95bf0d, 0xb, 0x227cfd40, 0x1, 0x1)  
/go/src/github.com/elastic/beats/libbeat/logp/log.go:95 +0x59 fp=0x227cfd0c sp=0x227cfce8  
[github.com/elastic/beats/libbeat/publisher.(\*client](http://github.com/elastic/beats/libbeat/publisher.(*client)).filterEvent(0x12eebd60, 0x12fc60e0, 0x0)  
/go/src/github.com/elastic/beats/libbeat/publisher/client.go:214 +0x282 fp=0x227cfd68 sp=0x227cfd0c  
[github.com/elastic/beats/libbeat/publisher.(\*client](http://github.com/elastic/beats/libbeat/publisher.(*client)).PublishEvents(0x12eebd60, 0x1ce377d0, 0x1, 0x3, 0x1ce37810, 0x3, 0  
3, 0x1)  
/go/src/github.com/elastic/beats/libbeat/publisher/client.go:143 +0x1ef fp=0x227cfeac sp=0x227cfd68  
[github.com/elastic/beats/filebeat/publisher.(\*syncLogPublisher](http://github.com/elastic/beats/filebeat/publisher.(*syncLogPublisher)).Publish(0x12ee7260, 0x0, 0x0)  
/go/src/github.com/elastic/beats/filebeat/publisher/sync.go:62 +0x23b fp=0x227cff98 sp=0x227cfeac  
[github.com/elastic/beats/filebeat/publisher.(\*syncLogPublisher](http://github.com/elastic/beats/filebeat/publisher.(*syncLogPublisher)).Start.func1(0x12ee7260)  
/go/src/github.com/elastic/beats/filebeat/publisher/sync.go:45 +0xc8 fp=0x227cffc8 sp=0x227cff98  
runtime.goexit()  
/usr/local/go/src/runtime/asm\_386.s:1612 +0x1 fp=0x227cffcc sp=0x227cffc8  
created by [github.com/elastic/beats/filebeat/publisher.(\*syncLogPublisher](http://github.com/elastic/beats/filebeat/publisher.(*syncLogPublisher)).Start  
/go/src/github.com/elastic/beats/filebeat/publisher/sync.go:50 +0x9e

goroutine 1 [chan receive]:  
[github.com/elastic/beats/filebeat/beater.(\*Filebeat](http://github.com/elastic/beats/filebeat/beater.(*Filebeat)).Run(0x12ee8fa0, 0x12ed4180, 0x0, 0x0)  
/go/src/github.com/elastic/beats/filebeat/beater/filebeat.go:200 +0xe46  
[github.com/elastic/beats/libbeat/beat.(\*Beat](http://github.com/elastic/beats/libbeat/beat.(*Beat)).launch(0x12ed4180, 0x9b59b8, 0x0, 0x0)  
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:225 +0x900  
[github.com/elastic/beats/libbeat/beat.Run](http://github.com/elastic/beats/libbeat/beat.Run)(0x959ac5, 0x8, 0x0, 0x0, 0x9b59b8, 0x0, 0x0)  
/go/src/github.com/elastic/beats/libbeat/beat/beat.go:141 +0x57  
main.main()  
/go/src/github.com/elastic/beats/filebeat/main.go:22 +0x40

goroutine 35 [syscall]:  
os/signal.signal\_recv(0x0)  
/usr/local/go/src/runtime/sigqueue.go:116 +0x11e  
os/signal.loop()  
/usr/local/go/src/os/signal/signal\_unix.go:22 +0x1a  
created by os/signal.init.1  
/usr/local/go/src/os/signal/signal\_unix.go:28 +0x36

goroutine 19 [select]:  
[github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug.Run.func1](http://github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug.Run.func1)(0x12ee2240, 0x12ee21c0, 0x12ee2200)  
/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug/service.go:32 +0x14c  
created by [github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug.Run](http://github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug.Run)  
/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/windows/svc/debug/service.go:38 +0xd2

goroutine 14 [chan receive]:  
[github.com/elastic/beats/libbeat/logp.logMetrics(0x12ed4200)](http://github.com/elastic/beats/libbeat/logp.logMetrics(0x12ed4200))  
/go/src/github.com/elastic/beats/libbeat/logp/metrics.go:28 +0x2b4  
created by [github.com/elastic/beats/libbeat/logp.Init](http://github.com/elastic/beats/libbeat/logp.Init)  
/go/src/github.com/elastic/beats/libbeat/logp/logp.go:161 +0x381

goroutine 37 [select]:  
[github.com/elastic/beats/libbeat/publisher.(\*messageWorker](http://github.com/elastic/beats/libbeat/publisher.(*messageWorker)).run(0x12ee6960)  
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:63 +0x1c3  
created by [github.com/elastic/beats/libbeat/publisher.(\*messageWorker](http://github.com/elastic/beats/libbeat/publisher.(*messageWorker)).init  
/go/src/github.com/elastic/beats/libbeat/publisher/worker.go:57 +0xff

goroutine 38 [select]:  
[github.com/elastic/beats/libbeat/publisher.(\*bulkWorker](http://github.com/elastic/beats/libbeat/publisher.(*bulkWorker)).run(0x12eef640)  
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:54 +0x284  
created by [github.com/elastic/beats/libbeat/publisher.newBulkWorker](http://github.com/elastic/beats/libbeat/publisher.newBulkWorker)  
/go/src/github.com/elastic/beats/libbeat/publisher/bulk.go:42 +0x1da

goroutine 39 [chan receive]:  
[github.com/elastic/beats/libbeat/service.HandleSignals.func1](http://github.com/elastic/beats/libbeat/service.HandleSignals.func1)(0x12eefac0, 0x12ee8f94, 0x12ee8fb0)  
/go/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x31  
created by [github.com/elastic/beats/libbeat/service.HandleSignals](http://github.com/elastic/beats/libbeat/service.HandleSignals)  
/go/src/github.com/elastic/beats/libbeat/service/service.go:35 +0x158

---

<div class="post-metadata">

### Author: ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [May 29, 2017, 10:57am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/6 "2017-05-29T10:57:35Z")

</div>

Interesting, now it seems to run out of memory when logging the event (for debugging). I tried reproducing, but on macOS things seem to be working fine. Perhaps the OS has troubles finding a continuous memory buffer that large.

Going back to the big picture, perhaps filebeat is not the right tool for this case. If it's a single file and event, maybe you want to just index it with a simple curl command instead?

---

<div class="post-metadata">

### Author: ![sumit\_monga](https://avatars.discourse-cdn.com/v4/letter/s/a698b9/32.png) [@sumit\_monga](https://discuss.elastic.co/u/sumit_monga)
#### Post date: [May 31, 2017, 6:01am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/7 "2017-05-31T06:01:22Z")

</div>

Thanks @tudor. Feels same that Filebeat is not the right tool for this. I have tried similar thing using logstash using multiline codec with input plugin using combined configuration of max\_lines and max\_bytes. This seems to be working fine till the control goes to logstash output plugin where it is failing with the below response

```
 {:code=>400, :response_body=>"{\"error\":{\"root_cause\":[{\"type\":\"parse_exception\",\"reason\":\"Failed to derive x
content\"}],\"type\":\"parse_exception\",\"reason\":\"Failed to derive xcontent\"},\"status\":400}", :request_body=>""} 

```

I will raise a separate question for that.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 28, 2017, 6:01am UTC](https://discuss.elastic.co/t/filebeat-throwing-exception-with-increased-value-of-max-bytes/87400/8 "2017-06-28T06:01:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.