Fast populate index using elastic search

I am a beginner in symfony2 elastic search integration.

Currently my set up is simple with 5 shards and 1 node ( just one simple ubuntu node)
with symfony and elastic search installed and running.
How can I make indexing faster .I have millions of pictures and I am
using "caption" type to index it.
I run " php app/console fos:elastica:populate --no-reset
--index=search --type=picture --no-debug --batch-size=1000"
and it is taking too much time.(Around 2 hours)

I am also using bulk apis and I want to index million items but the
document suggests using "doc" , "source" ,"id" etc.

  1. I am using perl code
    use Search::Elasticsearch;

my $es = Search::Elasticsearch-new;
my $bulk = $es->bulk_helper(
index = 'search',
type = 'picture'

Index docs:

Helpful link:

  1. Also Using curl request I use

curl -s -XPOST localhost:9200/search/picture/_bulk --data-binary @requests; echo

where reuests contain
{ "index" : { "_index" : "search", "_type" : "picture" } }
{ "mappings" : "caption" }

How could I populate all the indexes very quickly?
Can somebody please help
The above two methods dont work . I am sure I might be making mistakes using perl bulk library or using _bulk curl requests
Please advise


Try setting refresh rate to -1 for the index while you bulk, also increase your bulk sizing.

The speed of indexing will also depend on your node setup and sizing.

I assume you meant doing in elasticsearch.yml file

index.refresh_interval: -1
indices.memory.index_buffer_size: 30%

elasticsearch.yml changes require a restart which isn't practical, and setting index.refresh_interval there would (I imagine) only change the default setting for new indexes. The refresh interval setting can be changed on the fly for any index. Not sure about indices.memory.index_buffer_size.

Using the curl bulk apis, I get response as this which seems that it is not indexing it.

curl -s -XPOST localhost:9200/search/picture/_bulk --data-binary @requests; echo


which does not make sense as I dont think any of the indexes are populated

My "requests" file contains:
{ "index" : { "_index" : "search", "_type" : "picture" } }
{ "mappings" : "caption" }

Also in symfony2 this is my config.yml file configuration:
default: { host: localhost, port: 9200 }
caption: ~
deletedAt: ~

And this is the cluster health report
"cluster_name" : "elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 10,
"active_shards" : 10,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 10,
"number_of_pending_tasks" : 0

Please advise