Limiting data integrity risks from compromised client

Elastic Stack Elasticsearch
1

Let's say I want to monitor the system logs of a cluster of servers with filebeat. Because using one data stream per host doesn't scale well and the cluster is logically part of the same application, they all write to the same data stream with minimal write-only privileges.

I'm struggling to figure out how to avoid two potential obstacles during post mortem analysis in the case of a compromised host:

  • The lack of a server side timestamp (a "processed_at" in addition to the normal timestamp) allows a malicious client to date logs to an arbitrary time. While this doesn't allow overwriting existing documents, it makes it difficult to simply discard all events after a known-bad date.
  • A malicious client could impersonate other clients by adding documents with spoofed identifier fields, making it more difficult to reconstruct the actual course of events.

The ingest pipeline includes a set_security_user processor that would solve the attribution problem, but to my understanding, the pipeline application cannot be enforced.

What are my options?

Thanks in advance

2

You mean a timestamp on Elasticsearch side? This is possible, you need to add a set processor.

3

Hi @nf4ray Welcome to the community! Good questions and good concerns...

That is not exactly accurate either as a default or final pipeline can be set on the server side in an index template to enforce behavior without the client side settings

default pipeline

final pipeline

So you can you absolutely enforce server side. most the security modules store an event.ingested just for that purpose which is the time the event was ingested into elasticsearch

4

Thank you @leandrojmp for the hint and

I missed the final pipeline in the documentation. A quick test confirmed that it fulfills my requirements.

I appreciate the prompt assistance from both of you.