Elastic's Tenable Vulnerability Management Integration - Re-injesting Lost Data

Elastic Stack Elasticsearch
1

TDLR: is there a way to force elastic to injest all existing data for the past month from my tenable source?

For those that prefer an in-depth explaination:

I was tasked with coming out with a Tenable Dashboard that shows asset and vulnerability info. Since both are in seperate data streams, I decided to create a custom pipeline that enriches my vulnerability datastream with asset information. This custom pipeline is executed in the last time of tenable's vulnerability pipeline.

logs-tenable_io.vulnerability@custom (processors for my custom pipeline)

  {
    "enrich": {
      "field": "host.id",
      "policy_name": "enrich_tenable_vulnerability",
      "target_field": "asset-data",
      "ignore_missing": true
    }
  },
  {
    "script": {
      "lang": "painless",
      "source": "for (def tag : ctx['asset-data']['tenable_io']['asset']['tags']) {\n              if(tag[\"key\"] == \"Location\") {\n                ctx[\"Region\"] =  tag['value']\n              }\n              ctx[tag[\"key\"]] = tag['value'];\n            }"
    }
  }
]

The issue here was that initially, I reindexed my existing asset index and created my enrich policy based on it.

PUT /_enrich/policy/enrich_tenable_vulnerability
{
  "match": {
    "indices": "ds-logs-tenable_io.asset-tenable",
    "match_field": "host.id",
    "enrich_fields": ["tenable_io.asset.tags.key","tenable_io.asset.tags.value"]
  }
}

I only found out today that the reindexed index doesn't injest data like the original one does. Therefore my enrich preprocessor has only been enriching my vulnerability dataset based on old asset data, resulting in a loss in data.

I've already changed my match index to the correct asset index

PUT /_enrich/policy/enrich_tenable_vulnerability
{
  "match": {
    "indices": ".ds-logs-tenable_io.asset-tenable-*",
    "match_field": "host.id",
    "enrich_fields": ["tenable_io.asset.tags.key","tenable_io.asset.tags.value"]
  }
}

My question here is, since I lost so much data due my flawed enrich policy logic, is there a way i could re-injest all existing data from my tenable source for the past month?

Would really appreciate some guidance and support since I'm still relatively new to elastic!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.