Bulk API: different values for the same parameter at batch and operation level?


(yawei) #1

Hi there,

I noticed that in Java bulk API, some parameters can be set can on both
per-batch-request level and per-operation level, e.g. the consistency level
parameter: BulkRequestBuilder#setConsistencyLevel v.s IndexRequestBuilder.
setConsistencyLevel.

What if the parameter has different values between these two level? Will
the per-operation one override the per-batch-request one?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1962c6cd-8829-4b58-8b8e-ef3350f6ea98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

When using bulk indexing, the parameters in the single index request are
not used.

In order to stay consistent in the API, the bulk request provides some
methods of the same name as the single indexing action again. But this is
now for controlling the internal bulk execution that spans a number of
indexing operation. It is not implemented as a mere sequence of indexing
actions, but as a new distributed asynchronous action, with dedicated "bulk
write consistency".

Regarding consistency level, the execution for a single indexing operation
is simple: first, the index is looked up in the cluster state, then, the
replica level of the index is checked, then indexing on primary shard is
executed, and that primary shard passes on the execution to nodes that hold
replica shards. All straight forward.

For bulk indexing, the consistency level setting might reflect multiple
indices over multiple shards and different replica levels, and will as such
executed: the sequence of single indexing actions is ordered and divided
into groups of members that belong to the same shard, and sent off to the
involved nodes. The nodes holding the involved shards execute the
primary/replica operation in concurrency, and follow the bulk request
consistency level parameter.

Jörg

On Thu, Jul 3, 2014 at 3:06 AM, Yawei yawei.li@gmail.com wrote:

Hi there,

I noticed that in Java bulk API, some parameters can be set can on both
per-batch-request level and per-operation level, e.g. the consistency level
parameter: BulkRequestBuilder#setConsistencyLevel v.s IndexRequestBuilder.
setConsistencyLevel.

What if the parameter has different values between these two level? Will
the per-operation one override the per-batch-request one?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1962c6cd-8829-4b58-8b8e-ef3350f6ea98%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1962c6cd-8829-4b58-8b8e-ef3350f6ea98%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEr5LO_HWb%2BFjOqFvWPeGApF2y27qMAfQhdficpDGUaUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3