Does ElasticSearch compress the logs? Suppose I supply a few hundred gigabytes of logs, how much space will the elastic search take in total, including the indices and the original documents?
Which logs?
It does compress the indices, see https://www.elastic.co/guide/en/elasticsearch/reference/2.2/index-modules.html#_static_index_settings
Suppose I want to use elasticsearch for log indexing and the size of my logs is 11TB roughly. How much server space will I require for elasticsearch, for both, the indices and the original documents?
About 11TB. The logs are compressed but the indexes that it uses to make
searching them quick tend to take up roughly the same amount of space as
you get from compression. Roughly. You are better off indexing a
representative sample and seeing what size you see.
I am currently using the default settings if elasticsearch. What are the ways in which I can reduce the size taken by elasticsearch without much compromising with the speed with which it searches the logs?
And what type of servers are preferred? the one with fixed storage or variable storage with I/O costs?