Bug: No Misp event data send to Kibana when Threat intel module used

Elastic Stack Beats
,
1

Hello,

I'm trying to integrate IOCs from MISP to Elastic stack (ELK) using the Filebeat Threat intel module.
I'm receiving event in Analytics Discover panel of Kibana with filebeat-* toggle on:
(see below image)

image
image2731×1648 445 KB

But what i receive is not populated with any intelligence from MISP. See below the extract from one hit in analytics dashboard (all the hit collected are similar to this one):

 "event": {
      "ingested": "2023-03-18T06:38:45.326642703Z",
      "original": "{\"response\":[]}",
      "created": "2023-03-18T06:38:44.324Z",
      "timezone": "+01:00",
      "kind": "enrichment",
      "module": "threatintel",
      "category": "threat",
      "type": "indicator",
      "dataset": "threatintel.misp"
    },
    "fileset": {
      "name": "misp"
    },
    "error": {
      "message": "cannot access method/field [size] from a null def reference"
    },
...

I follow the filebeat configuration in the elastic documentation: Threat Intel module | Filebeat Reference [8.6] | Elastic.
Here is my configuration of threat-intel module /etc/filebeat/modules.d/threatintel.yml:

   var.url: https://mb-api.abuse.ch/api/v1/

    # The interval to poll the API for updates.
    var.interval: 10m

  misp:
    enabled: true

    # Input used for ingesting threat intel data, defaults to JSON.
    var.input: httpjson

    # The URL of the MISP instance, should end with "/events/restSearch".  
    var.url: https://localhost/events/restSearch/      

    # The authentication token used to contact the MISP API. Found when looking at user account in the MISP UI.                                       
    var.api_token: BEpdSXuPb2lRyhVjNy9nHiA7EApYdDXXXXXX
    
    # Configures the type of SSL verification done, if MISP is running on self signed certificates                                                    
    # then the certificate would either need to be trusted, or verification_mode set to none.                                                         
    var.ssl.verification_mode: none
   
    # Optional filters that can be applied to the API for filtering out results. This should support the majority of fields in a MISP context.        
    # For examples please reference the filebeat module documentation.
    #var.filters:
    #  - threat_level: [4, 5]
    #  - to_ids: true
   
    # How far back to look once the beat starts up for the first time, the value has to be in hours. Each request afterwards will filter on any event newer
    # than the last event that was already ingested.
    var.first_interval: 300h
   
    # The interval to poll the API for updates.
    var.interval: 5m

In the configuration file i let var.first_interval: 300h.
Maybe I've got no data because of this field.

How this field var.first_interval is used in the POST request https://misp.local/events/restSearch. I see on the doc a couple of fields that could match it: (Last, Publish_timestamp, Timestamp, From, To).
(Fields List of events/restSearch.

image
image1561×925 176 KB

source: MISP Core format

I don't know what to do in order to make it works. Please help me

2

Hi @Nicolas_Pelletier

I I'm not an expert on MISP but I do understand the basic concepts.

What the fileeat module does is load the threat intel data and make it available to be used to enrich incoming events.... Think of it as look up data.

The filebeat module on its own does not do the matching / enriching

In Order to to actually use the MISP data or match / enrich an incoming events you have to use either the Indicator Match rule or an enrich processor.

What that does is take the event that's coming in, use the proper field and match it against the lookup data. That's how it's actually used.

Did you do that?

This module ingests data from a collection of different threat intelligence sources. The ingested data is meant to be used with Indicator Match rules, but is also compatible with other features like Enrich Processors. The related threat intel attribute that is meant to be used for matching incoming source data is stored under the threat.indicator.* fields.

3

Thank you for your answer,

Actually i don't want to enrich any incoming events. I don't have any events in ELK.
I just want to use the Kibana dashboard because the MISP one is very ugly.

Do i need to pass by the Indicator Match rule or an enrich processor step to do that ?

4

Hello,

Sorry about my previous answer i didn't response clearly to the question but well i didn't do that:

This module ingests data from a collection of different threat intelligence sources. The ingested data is meant to be used with Indicator Match rules, but is also compatible with other features like Enrich Processors. The related threat intel attribute that is meant to be used for matching incoming source data is stored under the threat.indicator.* fields.

5

The reason you are not seeing any data is this:

"message": "cannot access method/field [size] from a null def reference"

Which version of the stack are you using?

6

Hello Marius,

Thank you for your response i'm using docker container from docker elk that is using the 8.6.2 version.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.