Getting error 'Payload Too Large' in Kibana UI

Elastic Stack Kibana
1

Hello! We are currently running the ELK on 7.17.26 version along with ReadonlyREST plugin in both Kibana an Elasticsearch in order to use Keycloak authentication in Kibana. This past weeks some users were experiencing this error in the Kibana UI when they log in:

image
image480×116 7.85 KB

We've had this issue before and we know it's because of the amount of permissions that are attatched to some users when they log in. For now the workaround is to make the users remove some unused permissions and they are able to enter Kibana again. We recently implemented some filtering of unnecessary permissions in Keycloak side to help with this issue, but this is not a permanent solution because the amount of permissions might keep growing (we handle a big amount of logs and access must be restricted so permissions are needed).

We already asked about this in the ReadonlyREST forum and they assured that this error screen does not come from their side. We also checked our NGINX running in our Kibana server and it doesn't seem to be the problem either. Our only thing left to confirm is if this error comes from Kibana because we have no other place to look for.

If you by any chance can help with this we'd aprecciate it since it's only a matter of time before we face this issue again. Let me know if I can provide any other information.

Thanks in advance!

2

Hi @Natalia_Mellino,

I assume you are using your own on-prem install? Have you tried increasing the server.maxPayload setting.

I would still be careful not to set it too high for performance reasons, but you could see if that helps.

Let us know!

3

Hi! Thank you for your help. We saw that setting in the documentation and we were not sure about it, because we couldn't test it properly in that moment, so we went with the permission filtering instead. We have a test environment now where we are able to replicate this issue and I already asked my team to try this so I'll get back here when we do.

About the error itself, can you confirm if this message error comes from Kibana? We were investigating a lot trying to find out where does it come from (to understand better) and we found nothing, it does not seem to come from Elasticsearch, ReadonlyREST, Keycloak nor NGINX from what we saw.

4

I do see related issues with the same error in Kibana, which is why I'm guessing it's coming from Kibana. Have you checked the Kibana logs to see if the error is in there?

5

Yes, we've checked but nothing specific appears in the logs when an user fails to login with this message, no errors, no warnings, anything. (Logging is enabled on 'INFO' level).