Error fetching fields for index pattern file*

[_bugc4t]

Helo, bit new to elastic but I'm trying to get data into my ECK deployment. I'm using Filebeat to do this. I've got one option on the index patterns page with my File name but and I selected file*. I'm worried that not all of my data was imported, but not sure how to fix this on the k8s cluster side? Is it assigning more memory? Or is there a hard limit?

Error: Payload Too Large
    at fetchResponse$ (https://localhost:5601/bundles/commons.bundle.js:3:3009775)
    at s (https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774550)
    at Generator._invoke (https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774303)
    at Generator.forEach.e.<computed> [as next] (https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774907)
    at s (https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774550)
    at t (https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:775045)
    at https://localhost:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:775195

[stephenb (Stephen Brown)]

Hi @_bugc4t Welcome to the community

What Version of the Elastic Stack are on you on?

You most likely need to set the following in your kibana.yml. Increasing this allows larger index patterns to be updated. Perhaps update it to 2MB , 2097152 (but not too large)

server.maxPayload: The maximum payload size in bytes for incoming server requests.
Default: 1048576

try
server.maxPayload: 2097152

You can read about Kibana configuration settings here :

Configure Kibana | Kibana Guide [7.14] | Elastic

I asked about the version because in new versions of the Elastic Stack index patterns are more efficient / dynamic so they tend to be much smaller.

Also if older you may need to use the older setting

server.maxPayloadBytes: [7.13.0] Deprecated in 7.13.0. In 8.0 and later, this setting will no longer be supported.This setting has been renamed to server.maxPayload .

[_bugc4t]

Thanks for this, changing the limit didn't work so I switched to using ECK 1.7 with the Google Cloud Module (I think the lack of it was always the issue) but now it all works!