I went through the group and came across various replies by David Pilato,
Kimchy and other people which stated to use EBS volumes or local disks for
storing ES indices and S3 for periodic snapshots. Wanted to know the reason
why using S3 will be a bad choice for creating and storing indices on S3
(even if I mount S3 to EC2 instance).
My main points for storing and creating indices on S3 are:
I want to avoid cost of using both EBS and S3. (EBS for storing indices
data and S3 for snapshots). If I only use S3 then due to replication
feature I do not have to worry about my data loss.
Using S3 I will not have to worry about failure of nodes as all data is
in centralized placed being replicated.
I can use AWS auto scaling feature to much advantage as I can spawn a
new EC2 machine when demand rises and then all the indexing and querying
can be done through a central data source on S3.
If somebody can suggest how I can achieve creating and storing of indices
on S3 it will be a great help.
Thanking You
--
This message contains confidential information and is intended only for the
individual to whom it is addressed. If you are not the intended recipient,
you should not disseminate, distribute or copy this e-mail. Please notify
the sender immediately by e-mail if you have received this e-mail by
mistake and permanently delete this e-mail from your system. E-mail
transmission cannot be guaranteed to be secure or error-free as information
could be intercepted, corrupted, lost, destroyed, late or incomplete, or
could contain viruses. The sender therefore does not accept liability for
any errors or omissions in the contents of this message, which arise as a
result of e-mail transmission. If verification is required, please request
a hard-copy version from the sender. Druva, www.druva.com
I went through the group and came across various replies by David Pilato,
Kimchy and other people which stated to use EBS volumes or local disks for
storing ES indices and S3 for periodic snapshots. Wanted to know the reason
why using S3 will be a bad choice for creating and storing indices on S3
(even if I mount S3 to EC2 instance).
My main points for storing and creating indices on S3 are:
I want to avoid cost of using both EBS and S3. (EBS for storing indices
data and S3 for snapshots). If I only use S3 then due to replication
feature I do not have to worry about my data loss.
Using S3 I will not have to worry about failure of nodes as all data is
in centralized placed being replicated.
I can use AWS auto scaling feature to much advantage as I can spawn a
new EC2 machine when demand rises and then all the indexing and querying
can be done through a central data source on S3.
If somebody can suggest how I can achieve creating and storing of indices
on S3 it will be a great help.
Thanking You
This message contains confidential information and is intended only for
the individual to whom it is addressed. If you are not the intended
recipient, you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately by e-mail if you have received this
e-mail by mistake and permanently delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or error-free as
information could be intercepted, corrupted, lost, destroyed, late or
incomplete, or could contain viruses. The sender therefore does not accept
liability for any errors or omissions in the contents of this message,
which arise as a result of e-mail transmission. If verification is
required, please request a hard-copy version from the sender. Druva, www.druva.com
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.