# Fast populate index using elastic search **URL:** \ **Category:** Elasticsearch\ **Created:** [June 9, 2015, 5:35pm UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244 "2015-06-09T17:35:44Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![jimmy2017](https://avatars.discourse-cdn.com/v4/letter/j/e274bd/32.png) [@jimmy2017](https://discuss.elastic.co/u/jimmy2017)\ **Post date:** [June 9, 2015, 5:35pm UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/1 "2015-06-09T17:35:44Z")
I am a beginner in symfony2 elastic search integration. Currently my set up is simple with 5 shards and 1 node ( just one simple ubuntu node) with symfony and elastic search installed and running. How can I make indexing faster .I have millions of pictures and I am using "caption" type to index it. I run " php app/console fos:elastica:populate --no-reset --index=search --type=picture --no-debug --batch-size=1000" and it is taking too much time.(Around 2 hours) I am also using bulk apis and I want to index million items but the document suggests using "doc" , "source" ,"id" etc. 1. I am using perl code use Search::Elasticsearch; my $es = Search::Elasticsearch-new; my $bulk = $es-\>bulk\_helper( index = 'search', type = 'picture' ); # Index docs: $bulk-\>index(); Helpful link: [https://metacpan.org/pod/Search::Elasticsearch::Bulk#flush](https://metacpan.org/pod/Search::Elasticsearch::Bulk#flush) 1. Also Using curl request I use curl -s -XPOST localhost:9200/search/picture/\_bulk --data-binary @requests; echo where reuests contain { "index" : { "\_index" : "search", "\_type" : "picture" } } { "mappings" : "caption" } How could I populate all the indexes very quickly? Can somebody please help The above two methods dont work . I am sure I might be making mistakes using perl bulk library or using \_bulk curl requests Please advise Thanks ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 9, 2015, 9:31pm UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/2 "2015-06-09T21:31:59Z")
Try setting refresh rate to -1 for the index while you bulk, also increase your bulk sizing. The speed of indexing will also depend on your node setup and sizing. ---
**Author:** ![jimmy2017](https://avatars.discourse-cdn.com/v4/letter/j/e274bd/32.png) [@jimmy2017](https://discuss.elastic.co/u/jimmy2017)\ **Post date:** [June 10, 2015, 6:42am UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/3 "2015-06-10T06:42:08Z")
> [@warkolm](#): > > Try setting refresh rate to -1 for the index while you bulk, also increase your bulk sizing. I assume you meant doing in elasticsearch.yml file index.refresh\_interval: -1 indices.memory.index\_buffer\_size: 30% ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 10, 2015, 6:44am UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/4 "2015-06-10T06:44:50Z")
elasticsearch.yml changes require a restart which isn't practical, and setting index.refresh\_interval there would (I imagine) only change the default setting for new indexes. The refresh interval setting can be changed on the fly for any index. Not sure about indices.memory.index\_buffer\_size. ---
**Author:** ![jimmy2017](https://avatars.discourse-cdn.com/v4/letter/j/e274bd/32.png) [@jimmy2017](https://discuss.elastic.co/u/jimmy2017)\ **Post date:** [June 10, 2015, 6:58am UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/5 "2015-06-10T06:58:02Z")
Using the curl bulk apis, I get response as this which seems that it is not indexing it. curl -s -XPOST localhost:9200/search/picture/\_bulk --data-binary @requests; echo Response: {"took":2,"errors":false,"items":[{"create":{"\_index":"search","\_type":"picture","\_id":"AU3cN\_GH8glW2H0tbkIz","\_version":1,"status":201}}]} which does not make sense as I dont think any of the indexes are populated My "requests" file contains: { "index" : { "\_index" : "search", "\_type" : "picture" } } { "mappings" : "caption" } Also in symfony2 this is my config.yml file configuration: fos\_elastica: clients: default: { host: localhost, port: 9200 } indexes: search: types: picture: mappings: caption: ~ deletedAt: ~ tags: And this is the cluster health report { "cluster\_name" : "elasticsearch", "status" : "yellow", "timed\_out" : false, "number\_of\_nodes" : 1, "number\_of\_data\_nodes" : 1, "active\_primary\_shards" : 10, "active\_shards" : 10, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 10, "number\_of\_pending\_tasks" : 0 } Please advise Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:08am UTC](https://discuss.elastic.co/t/fast-populate-index-using-elastic-search/2244/6 "2017-07-06T00:08:40Z")