Check Kibana max-old-space-size

Elastic Stack Kibana
1

Hi,
I tried adding --max-old-space-size=2048 into the bin/kibana file under NODE_OPTIONS. How do I check that this came in effect?

Thanks!

2

Hi there!

You should only need to set this if you are experiencing problems with exhausting memory, which is only common when changing the enabled plugins (either disabling any of the built-ins or installing custom ones). After setting this, are you still seeing issues? If so what is the error message and what version of Kibana are you using?

3

Josh,
Yes, we were getting the Java Heap error ("FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory"). I was researching for possible solution that's why I added the "max-old-space-size=2048" into the NODE_OPTIONS.

I just want to know if there's a way to check the max-old-space-size. I have not seen the error but the night is young.

We are using Kibana 7.6.

Thanks!
Vee

4

Hi,
I just saw the error again:

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 0x769a75878a1]

Security context: 0x068ee341e6e9
1: byteLength [0x1d8e5c8f96a9] [buffer.js:~509] [pc=0x769a8eb23f7](this=0x1d8e5c8d31f1 <JSFunction Buffer (sfi = 0x3bd8d6d96039)>,string=0x23a6fdfd3c71 <Very long string[384638914]>,encoding=0x068ee343ddd1 <String[4]: utf8>)
2: size [0x1ab23cd1c1e1] [/kibana/kibana-7.6.0-linux-x86_64/node_modules/hapi/lib/response.js:682] [bytecode=0x188f...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

......
......

This occurred after adding the max-old-space-size=2048 to bin/kibana.

Anyone has another suggestion?

Thanks!
Vee

5

I'm not aware of any way to verify the setting is being picked up. This setting is passed from Node.js into V8 (the JavaScript engine) directly, so there may be a way get this information out via V8's debugging API.

That said, I would try increasing this limit until you see the problem go away. If that doesn't work then it's not being set correctly. You can also set it when you execute Kibana:

NODE_OPTIONS="--max-old-space-size=4096" ./bin/kibana

Is there anything that seems to trigger this failure? For example, does this always happen on startup or when using a particular feature? It's possible you've encountered a bug in a feature that is creating very large amounts of memory usage that cannot be fixed by increasing this limit.

6

Josh,
Thanks. I will try adding it on the command line..

It seems like this failure is triggered when a query is run that returns a big result. It's hard to troubleshoot given the limited error log we see.

-Vee

7

If a specific query is triggering the problem, you may want to try upgrading the Stack. We have fixed issues like this in the past and it's possible there is a bug fix for your specific issue. Do you know where these queries are originating from (Discover, Dashboard, Canvas, etc.)?

8

Josh,
I'm still trying to figure out where the query is being executed from. Is there a way to tell where it's coming from?

Thanks!
Vee

9

Your best be is turn on query logging by adding these to your kibana.yml file and restarting Kibana:

elasticsearch.logQueries: true
logging.verbose: true

You then should be able to see which query happens just before Kibana crashes. I hope that is helpful!

10

Will try this. Thanks Josh!

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.