I want to do a SUM Aggregation of bytes per IP from my traffic logs.
There are 80 million documents in one index with 3 shards and 1 replica distributed to three data-nodes.
Only the top ten results are shown in the visualization. The bytes field is mapped as "long" - the ip field mapped as "ip".
Often the visualization exits with a timeout.
I/O wait seems to be ok - CPU is only at about 50%.
What could cause this problem?
When looking at iotop and top I can see reads from the disks and a high CPU load only for the first seconds of the query - shouldn't there be a constant load until the query is finished?
If you run the query from your example above directly against ES, are you experiencing timeout issues? Or just from within Kibana when rendering the visualization?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.