Index multiple payload hits to hits array in new index

Hello,

I am developing a watch that needs access to previous watch payload data. Since the watch scope is that of itself, I was thinking that an index action to index the desired payload for future reference was the way to go. However, I am running into an issue where the indexed payload gets indexed in the "_docs" array. This makes it difficult to access this array using mustache templating in watcher. Also, I am not sure if this will affect index management as I will want to remove older hits in the index.

It makes sense why my code is indexing into the docs array but I was wondering if I can index directly into the hits array similar to how a query output structure looks.

Here's the index action:

"index_payload": {
      "condition": {
        "compare": {
          "ctx.vars.secondQuery.hits.total": {
            "gt": 0
          }
        }
      },
      "transform": {
        "script": {
          "source": "return [' _doc' : ctx.vars.highLoad ]",
          "lang": "painless"
        }
      },
      "index": {
        "index": "cf-server-load-watch-logs"
      }
    }

Where ctx.vars.highLoad is a payload object containing multiple hits.

The output of the index is:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "cf-server-load-watch-logs",
        "_type" : "_doc",
        "_id" : "ycHqzXYBPx5Zh6tJB_TX",
        "_score" : 1.0,
        "_source" : {
          " _doc" : [
            {
              "_index" : "gpg-metricbeat-2021.01",
              "_type" : "_doc",
              "_source" : {
                "@timestamp" : "2021-01-04T14:59:36.679Z",
                "system" : {
                  "load" : {
                    "1" : 137.15,
                    "cores" : 96,
                    "15" : 108.77,
                    "5" : 124.51,
                    "norm" : {
                      "1" : 1.4286,
                      "15" : 1.133,
                      "5" : 1.297
                    }
                  }
                },
              },
              "_id" : "7ObpzXYBKOxPS2foJiyR",
              "_score" : 2.0
            },
            {
              "_index" : "gpg-metricbeat-2021.01",
              "_type" : "_doc",
              "_source" : {
                "@timestamp" : "2021-01-04T15:00:11.520Z",
                "system" : {
                  "load" : {
                    "1" : 143.33,
                    "cores" : 96,
                    "15" : 124.06,
                    "5" : 135.19,
                    "norm" : {
                      "1" : 1.493,
                      "15" : 1.2923,
                      "5" : 1.4082
                    }
                  }
                },
              },
              "_id" : "LSPpzXYBH3imBevIrv2l",
              "_score" : 2.0
            }
          ]
        }
      }
    ]
  }
}

So, ideally I would like to have those 2 hits as separate hits in the hits array not in the docs array.

Thanks in advance.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.