# Should there be disk savings for storing fields as IP rather than String? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 16, 2017, 5:05am UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674 "2017-01-16T05:05:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![aeio](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@aeio](https://discuss.elastic.co/u/aeio)\ **Post date:** [January 16, 2017, 5:05am UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/1 "2017-01-16T05:05:27Z")
Hello, I hypothesized that the 4 octets of an IP address would be smaller to store than its 15 UTF8 (?) characters. I seem to be wrong. I tested this by populating 2 indexes with the same, randomly generated, ip address document. Index 1 has a mapping with the appropriate "ip" type. Index 2 has no explicit mapping. ❯ python -c "import json,random; print '\n'.join([json.dumps({'index': {}}) + '\n' + json.dumps({'ip': '.'.join('%s' % random.randint(0,255) for i in range(4))}) for x in range(10000000/4)])" \> ips-for-bulk-insert.json ❯ ES\_HEAP\_SIZE=5g /opt/elasticsearch-2.4.2/bin/elasticsearch --node.master=true --node.data=true --path.data=./delete-me & ❯ curl -s -XPOST localhost:9200/myindex1/ --data '{ "mappings": { "mytype": { "properties": { "ip": { "type": "ip" } } } } }' ❯ curl -s -XPOST localhost:9200/myindex1/mytype/\_bulk --data-binary @ips-for-bulk-insert.json \> /dev/null 2\>&1 ❯ curl -s -XPOST localhost:9200/myindex2/mytype/\_bulk --data-binary @ips-for-bulk-insert.json \> /dev/null 2\>&1 ❯ curl -s -XPOST "localhost:9200/\_forcemerge?max\_num\_segments=1" ❯ du -h ips-for-bulk-insert.json _91M ips-for-bulk-insert.json_ ❯ du -sh ./delete-me/elasticsearch/nodes/0/indices/myindex1 _151M ./delete-me/elasticsearch/nodes/0/indices/myindex1_ ❯ du -sh ./delete-me/elasticsearch/nodes/0/indices/myindex2 _152M ./delete-me/elasticsearch/nodes/0/indices/myindex2_ ❯ curl localhost:9200/\_mapping _{"myindex2":{"mappings":{"mytype":{"properties":{"ip":{"type":"string"}}}}},"myindex1":{"mappings":{"mytype":{"properties":{"ip":{"type":"ip"}}}}}}_ So, I'm a little confused by this result. Is there some kind of overhead I'm not accounting for? Some mistakes in my experiment? Is string compression just that good? One thing I noticed was, before I forced merging, the IP index was actually bigger than the String index. Any insights would be appreciated. Thanks. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [January 16, 2017, 11:44am UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/2 "2017-01-16T11:44:06Z")
Hi, > [@aeio](#): > > ❯ du -sh ./delete-me/elasticsearch/nodes/0/indices/myindex1 > 151M ./delete-me/elasticsearch/nodes/0/indices/myindex1 > ❯ du -sh ./delete-me/elasticsearch/nodes/0/indices/myindex2 > 152M ./delete-me/elasticsearch/nodes/0/indices/myindex2 This look very similar... can you please explain what confused you in your results? Note that it's perfectly fine that index files on the filesystem have different size before the forced merge; the ip field is indexed differently using the "ip" type (indexed as a numeric to allow fast range queries) and the string type. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [January 16, 2017, 11:46am UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/3 "2017-01-16T11:46:18Z")
Also, you could be interested in this blog article : > **[Support for IPv6 in Elasticsearch](https://www.elastic.co/blog/indexing-ipv6-addresses-in-elasticsearch)** > > Thanks to the new BKD structure in Elasticsearch 5.0 (part of the Elastic Stack, formerly ELK Stack), there's support for IPv6. ---
**Author:** ![aeio](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@aeio](https://discuss.elastic.co/u/aeio)\ **Post date:** [January 16, 2017, 5:50pm UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/4 "2017-01-16T17:50:05Z")
> [@tanguy](#): > > This look very similar... can you please explain what confused you in your results? I was hoping that myindex2, which has the proper IP mapping, would be much smaller than myindex2, which has no mapping and defaults to String. We're planning to scale our system up and I was hoping for some space savings by using proper types. Thank you for the article link. It was a very interesting read. Just to be clear: The experiment was done with ES 2.4.2, and all the IPs were IPv4. So the increased overhead of storing everything as IPv6 shouldn't have affected us, if I read correctly. ---
**Author:** ![aeio](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@aeio](https://discuss.elastic.co/u/aeio)\ **Post date:** [January 16, 2017, 7:00pm UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/5 "2017-01-16T19:00:15Z")
I just did a follow up test using ES 5.1.2, the same data set, and the following mapping: ❯ curl localhost:9200/\_mapping _{"ip-index":{"mappings":{"mytype":{"properties":{"ip":{"type":"ip"}}}}},"string-index":{"mappings":{"mytype":{"properties":{"ip":{"type":"text","fields":{"keyword":{"type":"keyword","ignore\_above":256}}}}}}}}_ Note that I only explicitly set ip-index's mapping, string-index's mapping was auto generated. The results were a size of 143MB for the ip-index and 217MB for the string-index. So, it looks like there are savings I just need to upgrade to ES5. Thanks for pointing me in the right direction. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 13, 2017, 7:00pm UTC](https://discuss.elastic.co/t/should-there-be-disk-savings-for-storing-fields-as-ip-rather-than-string/71674/6 "2017-02-13T19:00:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.