High level rest client with data stream

[prithvirajsivarajan (Prithviraj S)]

I'm using high level rest client to bulk insert data. In the BulkRequest object, we are inserting the data with the 'index' operation. like this:

{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }

But in data stream, we have to use 'create' operation. like this,

{"create":{ }}
{ "@timestamp": "2099-03-08T11:04:05.000Z", "user": { "id": "vlb44hny" }, "message": "Login attempt failed" }

Is there an option where I could tell my rest client to use 'create' instead of 'index'?

Or is there a work around to insert the docs with 'index' op?

How exactly 'create' and 'index' operations differ from each other and why data stream need to use 'create' opration

[joecwu (Joe Wu)]

I'm not sure what kind of rest client are you using.

In Java rest client, you can set the OpType to OpType.CREATE in IndexRequest object and BulkRequest should generate create operation instead of index.

Because the DataStream has its own logic to maintain the index rollover and that is why you should not specify the index name during the document indexing on DataStream.