Watcher Actions

[mord]

I work with Watcher and I want to create alerts. my want is to check if the "_id" parameter exists in index-2 from index-1.
If it does not exist in, It should copy the log to that index (from 1 to 2).

For example, at the table below we check every "_id" in index-1 and want to copy "333" because it does not exist in index-2.
on the other hand, "222" parameter from index-1 already exists in index-2, therefore nothing should be done.

index-1(_id field) | index-2 (_id field)
111 | 111
222 | 222
222 | 333
333 | 444
222 |
444 |

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "from": "now-1m"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "index_payload": {
      "index": {
        "index": "check",
        "doc_id": "{{ctx.payload._id}}"
      }
    }
  }
}

[spinscale (Alexander Reelsen)]

this requires you to scroll through your data using a scroll search. That however is not supported by any watcher action or input.

Can you maybe take a step back and explain the full use-case, maybe there is another solution we can find. Why do you need the same data in two indices to start with?

--Alex

[mord]

Hi,

I'm using in Symantec WSS to getting logs to my logstash,
I got logs, but for safety, those logs are duplicates from there service.

Downloading the logs and replacing the current logs with the new one