# ES - 1.4.2 - Uploading index data seems low and slow
**URL:** https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648
**Category:** Elasticsearch
**Created:** [January 14, 2015, 9:03pm UTC](https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648 "2015-01-14T21:03:21Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Bhumir\_Jhaveri](https://discuss.elastic.co/u/Bhumir_Jhaveri)
#### Post date: [January 14, 2015, 9:03pm UTC](https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648/1 "2015-01-14T21:03:21Z")
I have just migrated to ES 1.4.2 - I have 5 data nodes and 1 master node -
all these ES instances are having 32 gigs of heap (machine has 120 gigs) -
2tb of storage each - 16 cpus
I am trying to upload 3 indexes simultaneously - each of them are of size
12 gigs - i am trying to upload this via pig job and noticed that uploading
to ES is running really low and slow
Earlier I was on ES 1.2.2 and on that I always have got better upload speed
as compared 1.4.2 - Is there a setting which I should enable to boost up
the upload?
Previously (Everything was running on single node - i.e. data, master,
client etc) + currently (little different architecture as I mentioned in my
first line) for each index I have created 5 shards and 1 replica - Don't
know what has been messed up in 1.4.2 version - anyone has any idea on it?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 14, 2015, 11:13pm UTC](https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648/2 "2015-01-14T23:13:27Z")
FYI once you get to 32GB heap you lose some efficiency, try to keep heap
under 32GB, so 31GB or less.
Are you using the bulk API?
On 15 January 2015 at 10:03, Bhumir Jhaveri [bhumir81@gmail.com](mailto:bhumir81@gmail.com) wrote:
> I have just migrated to ES 1.4.2 - I have 5 data nodes and 1 master node -
> all these ES instances are having 32 gigs of heap (machine has 120 gigs) -
> 2tb of storage each - 16 cpus
> I am trying to upload 3 indexes simultaneously - each of them are of size
> 12 gigs - i am trying to upload this via pig job and noticed that uploading
> to ES is running really low and slow
>
> Earlier I was on ES 1.2.2 and on that I always have got better upload
> speed as compared 1.4.2 - Is there a setting which I should enable to boost
> up the upload?
>
> Previously (Everything was running on single node - i.e. data, master,
> client etc) + currently (little different architecture as I mentioned in my
> first line) for each index I have created 5 shards and 1 replica - Don't
> know what has been messed up in 1.4.2 version - anyone has any idea on it?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9QdaFWy5ySS%2B5iPM2KR1EA%3D568THOaT1-t5r2hw6fe1w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9QdaFWy5ySS%2B5iPM2KR1EA%3D568THOaT1-t5r2hw6fe1w%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Bhumir\_Jhaveri](https://discuss.elastic.co/u/Bhumir_Jhaveri)
#### Post date: [January 14, 2015, 11:43pm UTC](https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648/3 "2015-01-14T23:43:09Z")
Oh yeah - I reset that 31gigs.
I am not sure if I am using pig bulk api - I am uploading it via pig script
since the data to be uploaded is huge
-\>STORE big\_data INTO '$INDEX\_NAME/$DOCUMENT' USING
org.elasticsearch.hadoop.pig.EsStorage('es.input.json=true');
On Wednesday, January 14, 2015 at 3:13:55 PM UTC-8, Mark Walkom wrote:
> FYI once you get to 32GB heap you lose some efficiency, try to keep heap
> under 32GB, so 31GB or less.
>
> Are you using the bulk API?
>
> On 15 January 2015 at 10:03, Bhumir Jhaveri \<[bhum...@gmail.com](mailto:bhum...@gmail.com)
> \\> wrote:
>
> > I have just migrated to ES 1.4.2 - I have 5 data nodes and 1 master node
> >
> > - all these ES instances are having 32 gigs of heap (machine has 120 gigs)
> > - 2tb of storage each - 16 cpus
> > I am trying to upload 3 indexes simultaneously - each of them are of
> > size 12 gigs - i am trying to upload this via pig job and noticed that
> > uploading to ES is running really low and slow
> >
> > Earlier I was on ES 1.2.2 and on that I always have got better upload
> > speed as compared 1.4.2 - Is there a setting which I should enable to boost
> > up the upload?
> >
> > Previously (Everything was running on single node - i.e. data, master,
> > client etc) + currently (little different architecture as I mentioned in my
> > first line) for each index I have created 5 shards and 1 replica - Don't
> > know what has been messed up in 1.4.2 version - anyone has any idea on it?
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4709416-6d96-4a52-b2b3-e246f6bdc0a0%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/348a80e1-7cac-40ee-a700-f88c15fae409%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/348a80e1-7cac-40ee-a700-f88c15fae409%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:38am UTC](https://discuss.elastic.co/t/es-1-4-2-uploading-index-data-seems-low-and-slow/21648/4 "2017-07-06T00:38:51Z")