# How to speed up indexing by using Python API **URL:** \ **Category:** Elasticsearch\ **Created:** [May 22, 2014, 9:09am UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-by-using-python-api/17670 "2014-05-22T09:09:17Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![11149](https://avatars.discourse-cdn.com/v4/letter/1/da6949/32.png) [@11149](https://discuss.elastic.co/u/11149)\ **Post date:** [May 22, 2014, 9:09am UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-by-using-python-api/17670/1 "2014-05-22T09:09:17Z")
Hi all: Now , I am trying to index my logs by using the elasticsearch Python API, but I only get about 600 records/s indexing speed. but, on the same ES cluster, with the same data, logstash(redis -\> logstash -\> elasticsearch) can index data at the speed about 3000records/s. any advice on how to speed up indexing speed by using the Python API? thanks very much. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Honza\_Kral](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honza_kral/32/876_2.png) [@Honza\_Kral](https://discuss.elastic.co/u/Honza_Kral)\ **Post date:** [May 22, 2014, 12:49pm UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-by-using-python-api/17670/2 "2014-05-22T12:49:33Z")
Hi, what method are you using in your python script? Have you looked at the bulk and streaming\_bulk helpers in ealsticsearch-py? [http://elasticsearch-py.readthedocs.org/en/master/helpers.html](http://elasticsearch-py.readthedocs.org/en/master/helpers.html) Hope this helps, Honza On Thu, May 22, 2014 at 11:09 AM, 潘飞 [cnweike@gmail.com](mailto:cnweike@gmail.com) wrote: > Hi all: > > Now , I am trying to index my logs by using the elasticsearch Python API, > but I only get about 600 records/s indexing speed. > > but, on the same ES cluster, with the same data, logstash(redis -\> logstash > -\> elasticsearch) can index data at the speed about 3000records/s. > > any advice on how to speed up indexing speed by using the Python API? > > thanks very much. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CABfdDipXuL01-WNM0DbDE2Y%2BqBTs5G3wdRofMGN6C2eY2uwWrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CABfdDipXuL01-WNM0DbDE2Y%2BqBTs5G3wdRofMGN6C2eY2uwWrA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![emeschitc](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@emeschitc](https://discuss.elastic.co/u/emeschitc)\ **Post date:** [May 22, 2014, 1:35pm UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-by-using-python-api/17670/3 "2014-05-22T13:35:26Z")
Hi, use bulk indexing ! This will speed you up by at least an order of magnitude em On Thu, May 22, 2014 at 11:09 AM, 潘飞 [via Elasticsearch Users] \< [ml-node+s115913n4056261h83@n3.nabble.com](mailto:ml-node+s115913n4056261h83@n3.nabble.com)\> wrote: > Hi all: > > Now , I am trying to index my logs by using the elasticsearch Python API, > but I only get about 600 records/s indexing speed. > > but, on the same ES cluster, with the same data, logstash(redis -\> > logstash -\> elasticsearch) can index data at the speed about 3000records/s. > > any advice on how to speed up indexing speed by using the Python API? > > thanks very much. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [hidden email][http://user/SendEmail.jtp?type=node&node=4056261&i=0](http://user/SendEmail.jtp?type=node&node=4056261&i=0) > . > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1b957d36-b7ad-4671-999a-de06aaa74407%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > * * * > > If you reply to this email, your message will be added to the discussion > below: > > [http://elasticsearch-users.115913.n3.nabble.com/How-to-speed-up-indexing-by-using-Python-API-tp4056261.html](http://elasticsearch-users.115913.n3.nabble.com/How-to-speed-up-indexing-by-using-Python-API-tp4056261.html) > To unsubscribe from Elasticsearch Users, click here[http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe\_by\_code&node=115913&code=ZW1lc2NoaXRjQGdtYWlsLmNvbXwxMTU5MTN8LTExODcwOTk0NDI=](http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=115913&code=ZW1lc2NoaXRjQGdtYWlsLmNvbXwxMTU5MTN8LTExODcwOTk0NDI=) > . > NAML[http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro\_viewer&id=instant\_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify\_subscribers!nabble%3Aemail.naml-instant\_emails!nabble%3Aemail.naml-send\_instant\_email!nabble%3Aemail.naml](http://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:27am UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-by-using-python-api/17670/4 "2017-07-06T01:27:39Z")