Could not pack/validate JSON response

[rock1013 (rock)]

Elasticsearch Version

Version: 8.13.1, Build: deb/9287f29bba5e270bd51d557b8daccb7d118ba247/2024-03-29T10:05:29.787251984Z, JVM: 22

Installed Plugins

none

Java Version

bundled

OS Version

Linux server102 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Problem Description

After update to 8.12.2 and 8.13.1,ES response of fluentbit cant over 64KB. Json will be truncated. So it cant be parsed.

Steps to Reproduce

With update to 8.12.2 and 8.13.1, the fluentbit logs errors.

[Christian_Dahlqvist]

Sounds like this is a limitation/problem with fluentbit, which is not supported here. As a potential workaround, have you tried reducing the bulk request size to see if that limits the size of the response?

[rock1013 (rock)]

Hi Christian_Dahlqvist,
Thanks for help.
I checked fluentbit doc, there's no chuck size configure at all. And its set to around 2MB.
What we found now is respose tcp window full. I will upload picture. But without upgrade to 8.12.2 from 8.12.1 all things go well.

200 is es. 240 is fluentbit.

[Christian_Dahlqvist]

Then it may be an OS level issue. I still do not see how it is an Elasticsearch issue.

[rock1013 (rock)]

More work we’ve done here is ...
1.the reponse json is cut off

[rock1013 (rock)]

Due to new user limit
2.TCP Window Full found

[rock1013 (rock)]

it looks the upgrade made this happen.

[rock1013 (rock)]

[rock1013 (rock)]

My apology, I bypassed the openresty nginx. Things go well again.

[DavidTurner]

As mentioned here, this is happening because you're disabling support for chunked encoding. Don't do that, chunked encoding is a mandatory part of HTTP/1.1.