Maximum allowed string Issue

Elastic Stack Elasticsearch
1

I get this error:
The content length (732630494) is bigger than the maximum allowed string (536870888)

I added to kibana.yml:
server.maxPayloadBytes: 888888888
savedObjects.maxImportPayloadBytes: 50485760

I added to elasticsearch.yml: (i also got an error about Async)
search.max_async_search_response_size: 100mb

Still receiving the same error:
The content length (732630494) is bigger than the maximum allowed string (536870888)

What am i doing wrong???

2

Why are you looking to send such immense payloads to Elasticsearch? What kind of data is it?

1 Like
3

Well some component changed to debug mode and since then we receive this error.
The content length (732630494) is bigger than the maximum allowed string (536870888) does that mean that there is a single log that is bigger then 73mb?

4

Are you ingesting a whole log as a single document?

Elasticsearch is not designed or optimized to handle extremely large documents, so the default limits that are in place should not be modified (which I asume you have done). The standard way to deal with log data is to ingest files with each line as a single document (multiline log entries can be merged into a single document).

5

Thank you,
I am not sure what I am experiancing. I checked and the longest log line did not exceed 10mb. But the file that filebeat read from was 700mb.

So not sure where this error is coming from.
this is my filebeat settings

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - C:\ddd_logs\*\*.json
      - C:\ddd_logs\errors\*.json
    json.keys_under_root: true
    json.add_error_key: true
    json.message_key: message

setup.template.settings:
  index.number_of_shards: 1
 
output.logstash:
  
  hosts: ["xxxxxx:5044"]
  index: "testxxxx"


processors:
  - drop_fields:
      fields: ["ecs.version", "agent.ersion", "agent.type", "agent.id", "agent.hostname", "input.type", "tags", "log.flags", "agent.name", "agent.ephemeral_id"]
6

Also, confused about general terms
when I get this error The content length (732630494) is bigger than the maximum allowed string (536870888) in a search
What does it mean?
that the a log in my search is bigger then 732630494?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.