Write-only cluster / index? Auditing porpouse

Elastic Stack Elasticsearch
1

Hi

I have a goal of creating audit-prof solution, where logs (indices) can not be modified nor deleted (like write-only).
The indices would have configurable TTL (like 365 days), and they would remove the date itself, but no user would be able to override, change, remove existing documents.

For the moment I've created daily snapshots to "write-only" s3 bucket:

It is close enough, but still - the last 24 hours (or the time since last snapshot), could be removed. In case of attack or some aware user action, someone have enough time to clean the ES cluster.

Would You recommend better solution for 'persistent-secure' log storage?
I may be missing some setting?

2

You should secure your cluster like any other datastore/database.
Have access control, have audit logging on that, take backups (as you are).

3

Possible solution?: Ban the: XDELETE, URI: */_update,
But what about 'whole document update' where _version number changes?
Any way to disable 'in-place' document update'?

4

Not unless you set the entire index to read only.

5

Hm, tu continue the topic

In Shield, you have different privileges, namely the create one which only gives the ability to create documents, but neither to update them nor to delete them. That's probably what you're looking for

6

Actually I've been thinking wrt 'document in-place update' - since we can look for the documents with _version > 1 (that would indicate that they were modified), and additionally we can set the 'previous version gc delete' to a day
index.gc_deletes = 82800
we should be able to store the previous versions of documents, and/or at least, with a simple query, have a knowledge of the data modification (that is important too).

What You think about that?

1 Like
7

I cannot find any recent reference to gc_deletes. I'd discourage use of it.

8


9

Any recent reference; That blog post and SO link are both from 2013. There doesn't appear to be any such setting in 2.X.