I noticed that some of my files are not being ingested properly with the "_jsonparsefailure" tag.
When I take a look at the message it received, it is indeed receiving only part of the JSON file. I suspect that when Logstash sees a lot of new files at once, part of the JSON file gets cut off, and Logstash will parse the JSON file incorrectly.
I think the issue is that I am using a VM. and I am ingesting files through a Shared Folder. The issue was resolved when I ingested files that is local to the VM.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.