I am having a monitoring stack, in which, I am sending beats data directly to elasticsearch without logstash in between.
Data sent by beats is not get stored in elasticsearch immediately. It lags ~4-5 Min, and lag increases further after certain period.
ElasticSearch Version: 6.7
I am not using for prod environment, hence, I have kept
only 1 master node, 1 data node and 1 client node.
Master and Data Node have 2-2 Gi of RAM and 1 Gi of heap space.
Client Node has 4 Gi RAM and 2 Gi heap space.
*I know this isn't a recommended architecture, but, I am using it just for dev environment at non-critical level, and I have to use this in limited resources.
What changes I should do in architecture/configuration like flush_time, queue_size, etc.?