Does elasticsearch reduce bloating?


(Jason Brooks) #1

Hello,

Does elasticsearch deduplicate data? I mean if there is a name/value combination in common for many events, does elasticsearch do any storage optimization?

For example,I have a large ( >500,000) number of events. in each event is the "EventCode" field, and there are perhaps 20 unique Eventcodes. If this were sql, I would break these eventcodes into their own table and use a foreign key to significantly reduce disk storage, and complexity.

If elasticsearch doesn't do this out of the box, are there ways to provide hints to do this?

Thank you for your information and time...

--jason


(Jason Brooks) #2

OOps: Did I say >500,000 events? more like > 8,305,270 events...

:slight_smile:


(Mark Walkom) #3

It does not deduplicate.
It does compress, and there are two levels you can use - https://www.elastic.co/guide/en/elasticsearch/reference/6.1/index-modules.html#index-codec


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.