Elasticsearch on read-only filesystem?

Elastic Stack Elasticsearch
1

Hi list,

I'm new to Elasticsearch. I was wondering if there was a way for the index
and documents to live on a read-only filesystem; so far I've read about
setting index.blocks.read_only to true, but it seems that Elasticsearch
still needs write access.

I tested that by naively doing
sudo chmod -R a-w /var/lib/elasticsearch
sudo service elasticsearch restart
after setting index.blocks.read_only to true, but the service failed to
start, as I feared but also expected :wink:

Or what about splitting the index over two filesystems, where the volatile
part lives on a writable filesystem and the static part on a read-only
filesystem? I don't know much about Elasticsearch's internals yet, but I'm
assuming that if you are not redistributing your shards over multiple
running instances, and not having to reindex anything because you're not
modifying any documents, then most of the index and all the documents
should be able to be static?

I read this message [1] on this list, but it seems that is more about
restricting access to Elasticsearch rather than keeping the index on a
read-only filesystem.

(I'm running Elasticsearch on an Ubuntu 13.04 system, if that makes any
difference.)

Regards,
Philip

[1] https://groups.google.com/d/topic/elasticsearch/QvRwx2RljMw/discussion

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Why do you want to do that?

In addition to this question, you need to know that Elasticsearch stores also it's own files in addition to Lucene files. Cluster state needs to be updated when a Node leave or join the cluster for example.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 19 oct. 2013 à 01:46, Philip Chimento philip.chimento@gmail.com a écrit :

Hi list,

I'm new to Elasticsearch. I was wondering if there was a way for the index and documents to live on a read-only filesystem; so far I've read about setting index.blocks.read_only to true, but it seems that Elasticsearch still needs write access.

I tested that by naively doing
sudo chmod -R a-w /var/lib/elasticsearch
sudo service elasticsearch restart
after setting index.blocks.read_only to true, but the service failed to start, as I feared but also expected :wink:

Or what about splitting the index over two filesystems, where the volatile part lives on a writable filesystem and the static part on a read-only filesystem? I don't know much about Elasticsearch's internals yet, but I'm assuming that if you are not redistributing your shards over multiple running instances, and not having to reindex anything because you're not modifying any documents, then most of the index and all the documents should be able to be static?

I read this message [1] on this list, but it seems that is more about restricting access to Elasticsearch rather than keeping the index on a read-only filesystem.

(I'm running Elasticsearch on an Ubuntu 13.04 system, if that makes any difference.)

Regards,
Philip

[1] https://groups.google.com/d/topic/elasticsearch/QvRwx2RljMw/discussion

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Read-only search could be useful for kiosk systems

While ES itself could run in volatile memory, an immutable index/search
storage may prevent that malevolent accesses can tamper with the data in
the index. The search storage could be located on field replaceale units.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Yep, I'm trying to have an immutable database in order to be able to do
things such as update it with the OSTree updater [1] or have the database
and index on a DVD. When it needs updating, the update will happen all at
once, overwriting the old with an entirely new database and index. For that
reason, I also don't need or want more than one node per cluster.

[1] Projects/OSTree - GNOME Wiki!

--
Philip

Op zaterdag 19 oktober 2013 12:43:15 UTC-7 schreef Jörg Prante:

Read-only search could be useful for kiosk systems
Interactive kiosk - Wikipedia

While ES itself could run in volatile memory, an immutable index/search
storage may prevent that malevolent accesses can tamper with the data in
the index. The search storage could be located on field replaceale units.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

On Sunday, October 20, 2013 8:05:48 PM UTC-7, Philip Chimento wrote:

Yep, I'm trying to have an immutable database in order to be able to do
things such as update it with the OSTree updater [1] or have the database
and index on a DVD. When it needs updating, the update will happen all at
once, overwriting the old with an entirely new database and index. For that
reason, I also don't need or want more than one node per cluster.

[1] Projects/OSTree - GNOME Wiki!

If anyone else ends up in the same situation, I found unionfs-fuse to give
good results so far: mount the immutable database readonly, then mount an
empty readwrite directory on top of it with copy-on-write semantics. Then,
when updating, wipe the readwrite directory. UnionFS has several inherent
problems but for a simple use like this, it seems sufficient.

--
Philip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.