Recorded Future Integration Package Setup Stack Traces

[codewriterguy (Patrick Kinsella)]

Setting up the Recorded Future integration package. Package installed smoothly and I was setting up feeds for domain, hash, url, ip.

When setting up the url feed, I got an error when clicking Save. Believe it was from choosing the same Integration name as a previously configured feed.

I canceled setup for the new feed. Since that step, when I navigate to the integration and choose Integration Policies, I get a stack trace.

I am able to run configuration for a new feed by choosing "+ Add Recorded Future", but after saving, I also see a stack trace.

Here is a copy of the stack trace:

Error: Cannot read properties of undefined (reading 'agents')
    at render (http://172.30.129.161:5601/50668/bundles/plugin/fleet/kibana/fleet.chunk.0.js:18:10692)
    at n.value (http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:197657)
    at http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:195107
    at Array.forEach (<anonymous>)
    at n.value (http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:194931)
    at http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:193806
    at Array.map (<anonymous>)
    at n.value (http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:193699)
    at n.value (http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:189176)
    at n.value (http://172.30.129.161:5601/50668/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js:400:188630)

Here is a screenshot of the problematic tab:

[nchaulet (Nicolas Chaulet)]

Hi @codewriterguy

I think your configuration got corrupted somehow, we use two different document (saved objects) to store agent policies and package policy and I think you probably have a package policy referring an agent policy that do not exists anymore.

I created a bug ticket in Kibana so we can have a better experience for that [Fleet] Do not crash when a package policy refer a non existing agent policy · Issue #131928 · elastic/kibana · GitHub

In the mean time you could probably delete the problematic saved object from the .kibana index (be carefull of what you delete in that index) and use those queries to find the problematic package policy

GET .kibana/_search?q=type:ingest-agent-policies
{
  "fields": [],
  "_source": false
}

GET .kibana/_search?q=type:ingest-package-policies
{
  "fields": [
    "ingest-package-policies.policy_id"
  ],
  "_source": false
}

Let me know if it work for you

[codewriterguy (Patrick Kinsella)]

Thank you for filing a bug ticket on this! I will take a look with these queries and see what I can do, and report back here

[codewriterguy (Patrick Kinsella)]

It turned out to be OK for me to delete all existing agent policies as we're still standing this cluster up. I deleted all agent policies. Your first query shows no hits:

{
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

I still can't delete the integration package though. "Uninstall Recorded Future" is grayed out in Integrations -> Recorded Future -> Settings

The second query shows 1 hit with no policy ID:

  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 7.779048,
    "hits" : [
      {
        "_index" : ".kibana_8.1.2_001",
        "_id" : "ingest-package-policies:c80f1a4a-7b5b-4f20-971c-8d9c350563a5",
        "_score" : 7.779048,
        "fields" : {
          "ingest-package-policies.policy_id" : [
            ""
          ]
        }
      }
    ]
  }

[codewriterguy (Patrick Kinsella)]

I did also restart ES + Kibana services after removing all agent policies

[codewriterguy (Patrick Kinsella)]

This is interesting, Management -> Fleet shows no agent policies but Management -> Integrations -> Recorded Future shows 1 agent policy

[nchaulet (Nicolas Chaulet)]

Hi @codewriterguy it seems that there is an orphaned package policy that will need to be deleted

You could probably do so with the following query

DELETE .kibana/_doc/ingest-package-policies:c80f1a4a-7b5b-4f20-971c-8d9c350563a5

Let me know if it helped

[codewriterguy (Patrick Kinsella)]

Thank you @nchaulet ! I was able to delete the non-existent agent policy with this command. Very helpful. For reference for anyone visiting this thread I had to delete it with a user with the following role:

POST /_security/role/kibana_removal_danger
{ 
  "indices": [ 
{
  "names": [ ".kibana*" ], 
  "privileges": [ "create_doc","create","delete","index","write","all" ],
  "allow_restricted_indices": true 
}
] 
}