I am Ingesting data as Nested Objects using type=nested and include_in_parent=true. I am ingesting event objects where each event object has 3000 nested objects i.e. 3000 nested documents. I insert several of these objects in ElasticSearch. Ingestion happens almost instantly and is not an issue.
However, when I am trying to search this data using Kibana or the JAVA API from Elasticsearch, it takes around 10-11 seconds to read each nested object. I was wondering is there a size limit when creating/inserting nested objects ?
Any pointers on this would be highly appreciated. The reason I used nested type is because I have to insert 3000 objects/message and have an ingestion rate of 40 messages/sec to be Ingested. When I use non-nested objects, i.e. ingest each of the 3000 objects as seperate objects, the Ingestion rate is very slow and allows me to ingest only 10 messages/sec where Ingesting it as Nested allows me to Ingest 20 messages/sec.
System : Ubuntu, 15GB dedicated to ElasticSearch, 64GB disk, Virtual Machine having 10 CPU