Problem bulk indexing

Elastic Stack Elasticsearch
1

More newbie questions...

I am trying out bulk indexing for the first time, running one line of the example from the manual:

curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }'

The result is an error:

{"error":"ActionRequestValidationException[Validation Failed: 1: no requests added]"}

It seems a mystery to me. I've read through this thread and other posts from Lukáš Vlček, but I'm not even able to get the most basic bulk operation going.

2

curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" : { "_index" :
"test", "_type" : "type1", "_id" : "1" } }'

You're missing the final newline. Try this instead:

curl -XPUT 'http://localhost:9200/_bulk' -d '
{ "index" : { "_index" :"test", "_type" : "type1", "_id" : "1" } }
'

clint

3

There should also be the actual json document to index, in another new line,
and then a new line to end it all.

On Wed, Jan 5, 2011 at 1:44 PM, Clinton Gormley clinton@iannounce.co.ukwrote:

curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" : { "_index" :
"test", "_type" : "type1", "_id" : "1" } }'

You're missing the final newline. Try this instead:

curl -XPUT 'http://localhost:9200/_bulk' -d '
{ "index" : { "_index" :"test", "_type" : "type1", "_id" : "1" } }
'

clint

4

On Wed, 2011-01-05 at 14:31 +0200, Shay Banon wrote:

There should also be the actual json document to index, in another new
line, and then a new line to end it all.

oh yes :slight_smile:

On Wed, Jan 5, 2011 at 1:44 PM, Clinton Gormley
clinton@iannounce.co.uk wrote:

    > curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" :
    { "_index" :
    > "test", "_type" : "type1", "_id" : "1" } }'
    
    
    
    You're missing the final newline. Try this instead:
    
    curl -XPUT 'http://localhost:9200/_bulk' -d '
    { "index" : { "_index" :"test", "_type" : "type1", "_id" :
    "1" } }
    '
    
    
    clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.

5

Those two suggestions did the trick! I was leaving out the document contents to simplify the test, but obviously that was an over-simplification... Thanks Shay, Clint.

At the risk of outlasting my welcome, I did have a separate bulk index question: What is the limit on # of documents I can ship in one API call to _bulk? I'm sure the answer depends on size of docs and specs of my server, but do you have any kind of guidelines on what would constitute a ridiculously huge payload?

Steve

On Wed, 2011-01-05 at 14:31 +0200, Shay Banon wrote: > There should also be the actual json document to index, in another new > line, and then a new line to end it all.

oh yes :slight_smile:

On Wed, Jan 5, 2011 at 1:44 PM, Clinton Gormley
clinton@iannounce.co.uk wrote:

    > curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" :
    { "_index" :
    > "test", "_type" : "type1", "_id" : "1" } }'
    
    
    
    You're missing the final newline. Try this instead:
    
    curl -XPUT 'http://localhost:9200/_bulk' -d '
    { "index" : { "_index" :"test", "_type" : "type1", "_id" :
    "1" } }
    '
    
    
    clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.

6

The bulk request ends up being represented in memory on the receiving node,
and broken down in request per shard. So its really up to the memory
allocation and the number of concurrent bulk requests.

On Wed, Jan 5, 2011 at 7:19 PM, searchersteve stevesuo@gmail.com wrote:

Those two suggestions did the trick! I was leaving out the document
contents
to simplify the test, but obviously that was an over-simplification...
Thanks Shay, Clint.

At the risk of outlasting my welcome, I did have a separate bulk index
question: What is the limit on # of documents I can ship in one API call to
_bulk? I'm sure the answer depends on size of docs and specs of my server,
but do you have any kind of guidelines on what would constitute a
ridiculously huge payload?

Steve

Clinton Gormley wrote:

On Wed, 2011-01-05 at 14:31 +0200, Shay Banon wrote:

There should also be the actual json document to index, in another new
line, and then a new line to end it all.

oh yes :slight_smile:

On Wed, Jan 5, 2011 at 1:44 PM, Clinton Gormley
clinton@iannounce.co.uk wrote:

    > curl -XPUT 'http://localhost:9200/_bulk' -d '{ "index" :
    { "_index" :
    > "test", "_type" : "type1", "_id" : "1" } }'



    You're missing the final newline. Try this instead:

    curl -XPUT 'http://localhost:9200/_bulk' -d '
    { "index" : { "_index" :"test", "_type" : "type1", "_id" :
    "1" } }
    '


    clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Problem-bulk-indexing-tp2194866p2199824.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

7

Can you please tell me what's wrong with this one:
curl -X PUT "http://localhost:9200/_bulk" -d '{ "index" : { "_index" : "profusion_db", "_type" : "service_index", "_id" : "1" } }
{ "field1" : "value1" }
'