Recorded Future v1.0.1 Not Populating Intel Properly

[codewriterguy (Patrick Kinsella)]

Hi,

The Recorded Future integration package v1.0.1 appears not to be properly pulling risklists from Recorded Future.

Intel is streaming into the instance (screenshot at 22:15UTC):

Elastic agent is healthy:

I've tagged each entity type to make queries for entity types in logs-ti_* easier:

Seeing incorrect quantities coming in. No hashes, only 1895 IP events, 4385 domain events, and > 250,000 URL events. Last screenshot is from the _count endpoint for the data stream.

[codewriterguy (Patrick Kinsella)]

Updated the integration to v1.1.0

Intel is still incomplete. No hash intel and limited for other IOC types.

Ran the following query for tags ip, domain, url, hash:

GET /logs-ti_recordedfuture.threat-default/_search
{
  "query": {
    "bool": {
      "must": [
        {"range": {"@timestamp": {"gte": "now-1d/d"}}},
        {"term": {"tags": {"value": "hash"}}}
      ]
    }
  }
}

Yields document counts:
ip - 2654
domain - 9975
url - 8819
hash - 0

These lists from Recorded Future currently have this many indicators:
ip - 54677
domain - 100000
url - 100000
hash - 99985