Alerting on field value change

Elastic Observability Logs
1

I’m currently logging on-change data in ES, and I’m running a latest transform to store the most updated data into a separate index.

I was advised that an ingest pipeline will help me compare any value coming in with the value that is stored in the latest transforms index, and then I can alert when the value has changed.

I have created my ingest pipeline and used the grok processor to pick a field and capture its value, but I don’t know how to compare that value with what is inside the most recent transform index.

Is this possible to do, or should I be taking a different approach?

The goal is to monitor a Boolean field and send an alert every time it changes from true to false.

Thank you.

2

Maybe it’s me, but I find your problem description confusing. Can you share some sample documents, which index is doing what, the transform you are doing, the index pipeline, … ?

3

You want to monitor when it changes from true to false only or every time it changes state, like from false to true as well?

Where this advice to use ingest pipeline came from? It does not make much sense.

If you already have the most recent data on an index created by a transform, you could simple create a Security rule looking into this index.

4

Sorry about the confusing, I’m trying to monitor on-change data when the value for a specific field changes from true to false and when it changes from false to true. The ‘empty’ field is what I want to monitor

Sample data looks like this:

{
  "_index": "latest-card-state",
  "_id": "d0zkoUWm0Csk8ESoqS3yeBZHAAAAAAAA",
  "_version": 16,
  "_source": {
    "parent": "SLOT-1",
    "interface_name": "LM-1",
    "description": "C-Band Enhanced 16xQSFP28",
    "type": "LINECARD",
    "hardware_version": "002",
    "oper_status": "INACTIVE",
    "empty": "true",
    "mfg_name": "abcd",
    "part_no": "abc-def-901",
    "cribl_pipe": "server-gnmi",
    "@timestamp": "2025-11-14T17:09:47.059Z",
    "telegraf_agent": "ws-deployment-6cd88cdd4c-tkd4h-telegraf-agent",
    "name": "vendor",
    "serial_no": "123456",
    "device": "dev1lab.mgt.net",
    "timestamp": 1763140187
  },
  "fields": {
    "@timestamp": [
      "2025-11-14T17:09:47.059Z"
    ]
  }
}
5

The ingest pipeline advice came from one of our Elastic reps :slightly_smiling_face:

So are you saying I should just set up two alerts on the index where the latest data is being saved? One for when the ‘empty’ field reports back false, and the other when it reports back ‘true’.

6

Ok, now I understood, you want to alert when there is a status change, this is a little more complicated than it seems, the security rules would not be enough they would alert on every event when it is true or false.

Not sure where the ingest pipeline fits here, is this ingest pipeline being used in the transform ? What it is doing? It is not clear what it is doing.

7

If I’m understanding correctly, you want to know when a Boolean field’s value changes from version N to version N+1, in the doc’s _version sense, and this should essentially get detected at ingest time ?