Hi, David Pilato.,
On a single node? With standard hard disks?
More than 7.5k per sec is not bad at all.
Actually We Have two Nodes[both are master nodes only] Running in our
Server and our hard disks is SSD Only.
we have 8 shards and 0 replicas. and with bulk size is: 8k
[we have tried for 1k, 4k ,10k, 12k also but at 8k we we have good speed
with minimum resource<here while indexing we are calling Thread.sleep() for
pause the indexing operation>]
For the above Configuration also is our Indexing speed good?
Now i am sharing a peace of code of our Application
.................................................................................................................................
XContentBuilder builder = XContentFactory.jsonBuilder();
builder.startObject();
int position = 0;
for (Object fieldName : csvFields) {
builder.field((String) fieldName,
nextLine[position++]); }
builder.endObject();
bulkRequest.add(Requests.indexRequest("cdrdata").type("cdr")
.id(nextLine[0]).create(true).source(builder)); //here we have one
method like " .setRefresh(true) " but we are not using refresh here
//here we have taken sleep method for pause the index operation
for few seconds like.,
try { Thread.sleep(1000); //
because if we not pause the indexing operation the ES node Heap size is
Incrementing up to its max size[5gb] and
} catch (Exception e) { } //
causing NoNodeException or Some thing Like., TimeOut Exception
bulkRequest.execute(new ActionListener<BulkResponse>() {
@Override
public void onResponse(BulkResponse bulkResponse) {
//some print stmts }
@Override
public void onFailure(Throwable e) {// error print
stmts }
});
.....................................................................................................................
here., Is there any thing to change for boost the speed of Indexing
Operation..
You can probably increase it using more boxes (one shard per box),
disabling replicas, disable refresh, use SSD drives…
have we 8 shards and 0 replicas. and we have not use any refresh while
doing bulk Indexing..
But I don't know is the refreshing operation enable default or not? if the
refresh operation is default then how to disable it?
You can probably increase it using more boxes (one shard per box)
And You have mentioned here use One Shard per Box ., i clearly not able
to understand what is box here, is it server or node?
Also give me any suggestions to speed up my indexing operation.
Once again thanks for replying., David Pilato
Best Regards
Mohammad Rafi.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.