Hi,
Will Elasticsearch reject or drop documents if document size is too large? I am ingesting documents using logstash but I don’t see any error logs in Logstash or Elasticsearch.
How do I know if Elasticsearch rejects any documents?
Hey Amruth, simple answer -- if documents are being rejected you should get messages in the logs.
Root cause - if you have "missing" documents in elasticsearch its generally cause of mapping conflicts. E.g. you are POSTing in a document that has text in a date field.
If you do have super large fields you can set the ignore above parameter for your text fields - so it will not fail when the field is to large - as explained here --> https://www.elastic.co/guide/en/elasticsearch/reference/current/ignore-above.html
The best thing to do is to just try to manually post the missing reports into elastic using the dev_tools tab in Kibana or via curl. If it failed you will get a pretty good error message that will lead you down the right track.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.