Big-ish line missing n Elasticsearch 7.9.0

Elastic Stack Elasticsearch
1

We are using a standalone Elasticsearch on EC2 and the stack looks like Application (JSON) --> Fluent Bit --> Elasticsearch (7.9.0) on EC2 --> Kibana on EC2

We have a scenario wherein if the application generated JSON log line is really big-ish like 1000+ lines (braces, fields) , then this particular line alone is missing in Kibana.

To narrow down the issue,

• Have enabled Fluent Bit OUTPUT to write to a file and this big-ish line is present. So, have eliminated Fluentbit
• Ran a GET API call from Kibana Dev Tools, the big-ish line is missing. So, have eliminated Kibana as it is just a log viewer

Need some help to fix this issue

Thanks

2

Hi,

Do you use an ingest pipeline or do you only store the data?

Can you show us the Index Mapping for this particular field? At first glance I think it could be stored as keyword with ignore_above set: ignore_above | Elasticsearch Guide [7.15] | Elastic

Best regards
Wolfram

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.