# Limiting data integrity risks from compromised client

**URL:** <https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086>\
**Category:** Elasticsearch\
**Created:** [April 28, 2023, 8:20pm UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086 "2023-04-28T20:20:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nf4ray](https://avatars.discourse-cdn.com/v4/letter/n/f475e1/32.png) [@nf4ray](https://discuss.elastic.co/u/nf4ray)\
**Post date:** [April 28, 2023, 8:20pm UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086/1 "2023-04-28T20:20:40Z")

</div>

Let's say I want to monitor the system logs of a cluster of servers with filebeat. Because using one data stream per host doesn't scale well and the cluster is logically part of the same application, they all write to the same data stream with minimal write-only privileges.

I'm struggling to figure out how to avoid two potential obstacles during post mortem analysis in the case of a compromised host:

- The lack of a server side timestamp (a "processed\_at" in addition to the normal timestamp) allows a malicious client to date logs to an arbitrary time. While this doesn't allow overwriting existing documents, it makes it difficult to simply discard all events after a known-bad date.
- A malicious client could impersonate other clients by adding documents with spoofed identifier fields, making it more difficult to reconstruct the actual course of events.

The ingest pipeline includes a [set\_security\_user processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-node-set-security-user-processor.html) that would solve the attribution problem, but to my understanding, the pipeline application cannot be enforced.

What are my options?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 29, 2023, 12:22am UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086/2 "2023-04-29T00:22:40Z")

</div>

> [@nf4ray](#):
>
> The lack of a server side timestamp (a "processed\_at" in addition to the normal timestamp) allows a malicious client to date logs to an arbitrary time. While this doesn't allow overwriting existing documents, it makes it difficult to simply discard all events after a known-bad date.

You mean a timestamp on Elasticsearch side? This is possible, you need to add a [set processor](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/ingest.html#access-ingest-metadata).

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 29, 2023, 2:28am UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086/3 "2023-04-29T02:28:30Z")

</div>

Hi @nf4ray Welcome to the community! Good questions and good concerns...

> [@nf4ray](#):
>
> The ingest pipeline includes a [set\_security\_user processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-node-set-security-user-processor.html) that would solve the attribution problem, but to my understanding, the pipeline application cannot be enforced.

That is not exactly accurate either as a default or final pipeline can be set on the server side in an index template to enforce behavior without the client side settings

[default pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#set-default-pipeline)

[final pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#pipelines-for-beats)

So you can you absolutely enforce server side. most the security modules store an `event.ingested` just for that purpose which is the time the event was ingested into elasticsearch

---

<div class="post-metadata">

**Author:** ![nf4ray](https://avatars.discourse-cdn.com/v4/letter/n/f475e1/32.png) [@nf4ray](https://discuss.elastic.co/u/nf4ray)\
**Post date:** [April 29, 2023, 7:49am UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086/4 "2023-04-29T07:49:16Z")

</div>

Thank you @leandrojmp for the hint and

> [@stephenb](#):
>
> final pipeline

I missed the final pipeline in the documentation. A quick test confirmed that it fulfills my requirements.

I appreciate the prompt assistance from both of you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 27, 2023, 7:49am UTC](https://discuss.elastic.co/t/limiting-data-integrity-risks-from-compromised-client/331086/5 "2023-05-27T07:49:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
