Marvel 2 HTTP content length exceeded 104857600 bytes

Elastic Stack Elasticsearch
1

Hello team

I am using ES 2.0.0 + Marvel 2.0.0.
Probably after some heavy indexing I am getting the periodical error in ES-log:

[2015-11-20 10:35:45,199][WARN ][shield.transport.netty ] [by0kmh] Caught exception while handling client http traffic, closing connection [id: 0xe75b04f8, /10.205.114.251:17825 => /10.205.114.251:9200]
org.jboss.netty.handler.codec.frame.TooLongFrameException: HTTP content length exceeded 104857600 bytes.
....
[2015-11-20 10:35:45,200][ERROR][marvel.agent.exporter.http] failed sending data to [http://10.205.114.251:9200/_bulk]: IOException[Error writing to server]

and cannot recover from that even after reconfiguration and reinstalling marvel-agent

My questions:

  1. is there a parameter for marvel.agent.exporter like flush_size to limit the es_bulk content?
  2. which internal cache/counter/dataset I have to clear to throw away the existing buffer and continue to work?

Thanks in advanced!

2

Thanks for reporting Kaem.

There's a bug in marvel where you can fall in under certain circumstances... and I think you hit this one. It will be fixed in the next releases.

How many indices & shards do you have?

3

Hello,
I have 2 nodes, 100 indices and 423 shards

4

Side note, you should reduce your shard count. You are probably wasting resources :slight_smile:

5

Hi

after upgrading to marvel 2.1 the issues was fixed.

thx