Bulk Indexing via REST

Elastic Stack Elasticsearch
1

All,

I'm hitting this url "http://localhost:9200/_bulk" on my local elasticSearch. Thru http PUT METHOD, I'm sending following JSON:
{ "index" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1142" } }
{"name":"Exhibitor2","type":"Silver Sponsor","description":"exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description.","nameSort":"Exhibitor2","typeSort":"SilverSponsor","website":null,"boothNameNum":"","published":"exhibitorPublished","206":"","205":"","cv":""}
{ "index" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1143" } }
{"name":"Exhibitor3","type":"Gold Sponsor","description":"exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description. exhibitor description.","nameSort":"Exhibitor3","typeSort":"GoldSponsor","website":null,"boothNameNum":"","published":"exhibitorPublished","206":"","205":"","cv":""}

But, it seems like it is indexing ONLY first element of id = 1142. I know this for sure because when I hit this URL "http://localhost:9200/dev1_m1_conference/en_exhibitor/_search", it returns me the first element as a result.

Am I missing something while doing REST bulk indexing?

Thanks,

2

Hey,

can you put a gist, so that we can take your data and reconstruct the
problem properly (the sample above might contain unwanted line breaks etc)?
Make sure your indexing data does not contain any new lines...

--Alex

On Wed, Jun 5, 2013 at 7:16 PM, samCougars sbaniya@gmail.com wrote:

All,

I'm hitting this url "http://localhost:9200/_bulk" on my local
elasticSearch. Thru http PUT METHOD, I'm sending following JSON:
{ "index" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor",
"_id" : "1142" } }
{"name":"Exhibitor2","type":"Silver Sponsor","description":"exhibitor
description. exhibitor description. exhibitor description. exhibitor
description. exhibitor description. exhibitor description. exhibitor
description. exhibitor description. exhibitor

description.","nameSort":"Exhibitor2","typeSort":"SilverSponsor","website":null,"boothNameNum":"","published":"exhibitorPublished","206":"","205":"","cv":""}
{ "index" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor",
"_id" : "1143" } }
{"name":"Exhibitor3","type":"Gold Sponsor","description":"exhibitor
description. exhibitor description. exhibitor description. exhibitor
description. exhibitor description. exhibitor description. exhibitor
description. exhibitor description. exhibitor

description.","nameSort":"Exhibitor3","typeSort":"GoldSponsor","website":null,"boothNameNum":"","published":"exhibitorPublished","206":"","205":"","cv":""}

But, it seems like it is indexing ONLY first element of id = 1142. I know
this for sure because when I hit this URL
"http://localhost:9200/dev1_m1_conference/en_exhibitor/_search", it
returns
me the first element as a result.

Am I missing something while doing REST bulk indexing?

Thanks,

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-via-REST-tp4036078.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

I tried a simple example w/ less string, but still the same problem. Here is a link to gist:

4

I know my JSON string is correct because I tried this today:
{ "delete" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1142" } }
{ "create" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1142" } }
{"name":"Exhibitor2","type":"Silver Sponsor"}
{ "delete" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1143" } }
{ "create" : { "_index" : "dev1_m1_conference", "_type" : "en_exhibitor", "_id" : "1143" } }
{"name":"Exhibitor3","type":"Gold Sponsor"}

and it indexed both 1143 and 1143. Deleting an index before creating can be costly when I'm indexing 100000 + records at a time. Any help is much appreciated...
Thanks,
Sam

5

You need a final line feed character, after the last JSON line.

Jörg

Am 06.06.2013 22:09, schrieb samCougars:

Any help is much
appreciated...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

I'm using \n (return character) after every index line. Can you explain more on "final line feed character"? Sample may be?

7

Then it should do fine. For clarity, I marked the line feeds in a
comment gist:190bdf58531252a59629 · GitHub
The final line feed can not be left out.

Jörg

On Tuesday, June 11, 2013 1:31:08 AM UTC+2, Sam wrote:

I'm using \n (return character) after every index line. Can you explain
more
on "final line feed character"? Sample may be?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Bulk-Indexing-via-REST-tp4036078p4036330.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.