Sending index requests via the rabbitmq river


(benny.sadeh) #1

hi group,

I have a problem understanding how to transform an IndexRequest to the Bulk
API format.
I would expect to find a method on IndexRequest or IndexRequestBuilder to
perform such transformation for me, but haven't found such.

alas, looking at the explanation, it says it expects the following JSON
structure:

action_and_meta_data\n
optional_source\n
....

and the example given is:

{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }

which I understand to correspond to:

[index] action_and_meta_data
source
[delete] action_and_meta_data
[create] action_and_meta_data
source

if that's so, than I kind of get it, but kind of not knowing how
to programmatically convert an IndexRequest to such form.

any hints at how to go about would be much appreciated ...
thanks in advance,
Benny


(Drew Raines) #2

Benny Sadeh wrote:

hi group,

I have a problem understanding how to transform an IndexRequest to the
Bulk API format.
I would expect to find a method on IndexRequest or IndexRequestBuilder
to perform such transformation for me, but haven't found such.

[...]

if that's so, than I kind of get it, but kind of not knowing how
to programmatically convert an IndexRequest to such form.

any hints at how to go about would be much appreciated ...

Logically a bulk request is a collection of other types of requests.
There are add() methods on BulkRequest that reflect this.

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/action/bulk/BulkRequest.java#L63

-Drew


(Drew Raines) #3

Drew Raines wrote:

Benny Sadeh wrote:

hi group,

I have a problem understanding how to transform an IndexRequest to the
Bulk API format.
I would expect to find a method on IndexRequest or IndexRequestBuilder
to perform such transformation for me, but haven't found such.

[...]

if that's so, than I kind of get it, but kind of not knowing how
to programmatically convert an IndexRequest to such form.

any hints at how to go about would be much appreciated ...

Logically a bulk request is a collection of other types of requests.
There are add() methods on BulkRequest that reflect this.

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/action/bulk/BulkRequest.java#L63

I think I read your message too quickly. IIUC, you're asking if
IndexRequest can output a string of text suitable for injection into
a rabbit queue. It cannot. You should create lines of json using
whatever mechanism you like and then use those to form the payload of
your rabbit msg. If you'd like to use the ES lib to do that, take a
look at XContentFactory.jsonBuilder().

-Drew


(system) #4