Hi,
This is a super simple question, but googling has not helped me yet.
Is it ok to store a large chunk of data ES?
For example, I want to store a whole HTML page under the field name of
"original_text".
(I'm planning to store a news article in its HTML format, which is
about 100KB.)
So in the future I can access the HTML document whenever I want.
Is this going to cause any problem for ES?
Increased latency? Or too much cache usage? Or memory constantly being
swapped?
If I choose never to cache the result when I access the HTML document,
is it going to solve any of the issues I mentioned?
Any tip would be appreciated.
Thanks.
I don't see why that would be an issue. Maybe some actually uses ES in
this kind of setup and might give you some tips, but it should work.
Swapping should only occur if you allocate more memory to ES than you
have RAM. And that's not recommended - typically half your RAM should
go to ES. OS caching should be as busy with large documents as it
would be with more smaller ones (or even less busy).
As for ES caching, AFAIK this applies only to filters, not queries. So
it depends on how you search.
Hi,
This is a super simple question, but googling has not helped me yet.
Is it ok to store a large chunk of data ES?
For example, I want to store a whole HTML page under the field name of
"original_text".
(I'm planning to store a news article in its HTML format, which is
about 100KB.)
So in the future I can access the HTML document whenever I want.
Is this going to cause any problem for ES?
Increased latency? Or too much cache usage? Or memory constantly being
swapped?
If I choose never to cache the result when I access the HTML document,
is it going to solve any of the issues I mentioned?
Any tip would be appreciated.
Thanks.
I don't see why that would be an issue. Maybe some actually uses ES in
this kind of setup and might give you some tips, but it should work.
Swapping should only occur if you allocate more memory to ES than you
have RAM. And that's not recommended - typically half your RAM should
go to ES. OS caching should be as busy with large documents as it
would be with more smaller ones (or even less busy).
As for ES caching, AFAIK this applies only to filters, not queries. So
it depends on how you search.
Hi,
This is a super simple question, but googling has not helped me yet.
Is it ok to store a large chunk of data ES?
For example, I want to store a whole HTML page under the field name of
"original_text".
(I'm planning to store a news article in its HTML format, which is
about 100KB.)
So in the future I can access the HTML document whenever I want.
Is this going to cause any problem for ES?
Increased latency? Or too much cache usage? Or memory constantly being
swapped?
If I choose never to cache the result when I access the HTML document,
is it going to solve any of the issues I mentioned?
Any tip would be appreciated.
Thanks.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.