Bulk import performance

Hi,

I'm importing massive documents into Elasticsearch 1.7.1.

At the beginning everything looked fine but after sometime(roughly 1b data have been imported). The bulk indexing performance get worse.

Then I changed the refresh interval to -1, set the indices throttle policy to none and increased the bulk queue size, but these do not help too much.

One thing I do notice is that compared with the disk IO write, the disk IO read is quite high:

ID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
5143 be/4 elastics 9.53 M/s 43.38 K/s 0.00 % 90.59 % java -Xms15g -Xmx15g -Djava.awt.headless=true -XX:+UseParN~etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch
5145 be/4 elastics 12.39 M/s 39.43 K/s 0.00 % 90.35 % java -Xms15g -Xmx15g -Djava.awt.headless=true -XX:+UseParN~etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch
5144 be/4 elastics 12.08 M/s 35.49 K/s 0.00 % 90.26 % java -Xms15g -Xmx15g -Djava.awt.headless=true -XX:+UseParN~etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch
5146 be/4 elastics 14.60 M/s 55.21 K/s 0.00 % 90.04 % java -Xms15g -Xmx15g -Djava.awt.headless=true -XX:+UseParN~etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch

Is there anything else I can do to improve the situation?

Thanks,

Tim