# ES5 vs ES2 index size increase **URL:** \ **Category:** Elasticsearch\ **Created:** [March 30, 2017, 9:05pm UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749 "2017-03-30T21:05:57Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Kazama](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@Kazama](https://discuss.elastic.co/u/Kazama)\ **Post date:** [March 30, 2017, 9:05pm UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/1 "2017-03-30T21:05:57Z")
The only changes I've made to my indexes migrating from ES 2.4.4 to ES 5.3.0 are such mappings upgrade (_rest of the fields are compatible with 5.x_): ``` { x: string, index: not_analyzed } => { x: keyword } { x: string, term_vector: yes } => { x: text, term_vector: yes } ``` Reindexing from scratch I've got such stats: dataset1=560.392 docs. ES 2.4.4 index size=99.7M ES 5.3.0 index size=104M dataset2=2.583.604 docs ES 2.4.4 index size=623M ES 5.3.0 index size=662M Is it a general rule: 5.x index size is larger than 2.x one for the same docs? **Mby it matters** : 2.4.4 comes from the official deb repository, 5.3.0 comes from the official docker image (I mean how they were configured etc). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 31, 2017, 5:10am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/2 "2017-03-31T05:10:25Z")
A first guess is doc values. They are generated when you have keyword type. It's not identical to not\_analyzed actually. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 31, 2017, 5:17am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/3 "2017-03-31T05:17:12Z")
I recall more data has been migrated to doc\_values in 5.x compared to 2.x, which means that the index size in 5.x, depending on your mappings, may take up a bit more space. When I tested it on a sample data set a while back I think it was in the range of 3-5%, but your milage may vary. ---
**Author:** ![Kazama](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@Kazama](https://discuss.elastic.co/u/Kazama)\ **Post date:** [March 31, 2017, 9:49am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/4 "2017-03-31T09:49:22Z")
> [@dadoonet](#): > > A first guess is doc values.They are generated when you have keyword type. It's not identical to not\_analyzed actually. All relevant fields have doc\_values explicitly disabled or enabled, no changes were made to this during migration. > [@Christian\_Dahlqvist](#): > > I recall more data has been migrated to doc\_values in 5.x compared to 2.x, which means that the index size in 5.x, depending on your mappings, may take up a bit more space. When I tested it on a sample data set a while back I think it was in the range of 3-5%, but your milage may vary. Thanks, I guess that's the reason. I have some fields with doc\_values enabled indeed. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 31, 2017, 9:56am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/5 "2017-03-31T09:56:56Z")
Have you run a force merge on these indices to ensure they have the same number of segments? ---
**Author:** ![Kazama](https://avatars.discourse-cdn.com/v4/letter/k/d07c76/32.png) [@Kazama](https://discuss.elastic.co/u/Kazama)\ **Post date:** [March 31, 2017, 10:49am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/6 "2017-03-31T10:49:57Z")
I've just figured out indexing on 2.4.4 was done into 1 shard and it was done into 6 shards on 5.3.0. After 5.3.0 reindexing into 1 shard **and** doing \_forcemerge here are the updated stats: dataset1=560.392 docs. ES 2.4.4 index size=99.7M ES 5.3.0 index size=100M dataset2=2.583.604 docs ES 2.4.4 index size=623M ES 5.3.0 index size=630M 5.3.0 is a bit higher still but the difference is really small. Thanks for your suggestions! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 28, 2017, 10:50am UTC](https://discuss.elastic.co/t/es5-vs-es2-index-size-increase/80749/7 "2017-04-28T10:50:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.