# Batch bulk operation - wikidata large json

**URL:** <https://discuss.elastic.co/t/batch-bulk-operation-wikidata-large-json/140527>\
**Category:** Elasticsearch\
**Created:** [July 18, 2018, 11:27am UTC](https://discuss.elastic.co/t/batch-bulk-operation-wikidata-large-json/140527 "2018-07-18T11:27:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![geppo](https://avatars.discourse-cdn.com/v4/letter/g/3be4f8/32.png) [@geppo](https://discuss.elastic.co/u/geppo)\
**Post date:** [July 18, 2018, 11:27am UTC](https://discuss.elastic.co/t/batch-bulk-operation-wikidata-large-json/140527/1 "2018-07-18T11:27:09Z")

</div>

I'm trying to indicize the wikidata entities dump - taken from [here](https://dumps.wikimedia.org/wikidatawiki/entities/) - creating a bulk json.

I have manipulate the initial json in this way:

```
{"index": {"_type": "page", "_id": 1}}
 {"type":"item","id":"Q26","labels":{"en-gb":{"language":"en-gb","value":"Northern Ireland"},"en": {"language":"en","value":"Northern Ireland"},"it":{"language":"it","value":"Irlanda del Nord"},"fr":{"language":"fr","value":"Irlande du Nord"},"eo":{"language":"eo","value":"Nord-Irlando"},"pl":{"language":"pl","value":"Irlandia P\u00f3\u0142nocna"}, [..]`

```

and I have tried to indexed with this command:

```
`cat nuovowikidata.json | parallel --pipe -L 2 -N 2000 -j3 'curl -H "Content-Type: application/x-ndjson" -s http://localhost:9200/wikidata_entities/_bulk --data-binary @- > /dev/null'`

```

but the operation runs very slow. in 36h i have indicized only 216448 docs . How could I improve that speed?  
Thanks in advance

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 15, 2018, 11:27am UTC](https://discuss.elastic.co/t/batch-bulk-operation-wikidata-large-json/140527/2 "2018-08-15T11:27:15Z")

</div>

Closed on August 15, 2018, 11:27am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.