Network Condition Not Working?

[chapmanjc (Joe)]

Hello all,

I've set up an an Elastic Agent and I am using the Microsoft DHCP module. Everything is working great except there is a slight issue in regards to going into the "Advanced Options" inside of the Integration. I'm attempting to follow this guide(Configure network map data | Elastic Security Solution [8.5] | Elastic) to assist with mapping an our internal network.

This is an example of my config in Advanced Options:

- add_fields:
    when.network.host.ip: '10.113.0.0/16'
    fields:
      destination.geo.location:
        lat: 100.000
        lon: -100.000
    target: ''

However, I've noticed if I do the following, it will add the destination.geo.location fields I want:

- add_fields:
    when.contains.host.ip: '10.113.15.100'
    fields:
      destination.geo.location:
        lat: 100.000
        lon: -100.000
    target: ''

So, the conditional seems to run OK if I use the "contain" option but not "network." So, I don't know if this is a limitation of using Elastic Agent, a bug or I'm doing something wrong.

[andrewkroh]

On those documents, is host.ip a array or string?

I predict that if you change the Agent logging level to Debug that a message is being logged that the network condition did not match because the value is an array and the network condition expects to match against scalar values only.

The contains conditional is setup to check if any value in an array contains the given string.

You could do something similar using Elasticsearch Ingest Node and Painless scripting. It has a CIDR type with a contains function. You could loop over the values and add the geo fields if any one IP matches.

Or we could potentially enhance the network condition to support arrays.

[chapmanjc (Joe)]

Hi Andrew,

Thanks for the reply!

I have debug turned on and I don't see any errors in regards to not matching. From what I can tell, host.ip appears to a string? This was a section of an ingested document.

"host": {
      "ip": "10.113.15.100",
      "domain": "domain.local",
      "name": computer.domain.local",
      "id": "1234566",
      "mac": [
        "56-12-8D-56-RB-56"
      ]
    },```

[andrewkroh]

I tested the processor conditional using Beats Playground and it appears to be working as expected so there must be some other cause. Looking...

[andrewkroh]

The host.ip field is produced by the Elasticsearch Ingest Node pipeline (see integrations/dhcp.yml at 0a08e77d051f44acc2b4afa5e139f342d71835ef · elastic/integrations · GitHub). This means that the host.ip field does not exists at the time at which the Agent side processor is executing.

This is an implication of what's mentioned in the description for the processors in the UI. You would need to move this over to ingest node to have access to the parsed out host.ip field.

[chapmanjc (Joe)]

Thanks for the replies! So, just to reiterate, this needs to be set in the ingest node as opposed to being in the processor. And probably the best way to go would be using a painless script?

[andrewkroh]

Yes, a script is the only way that I know of in Ingest Node to get access to something that can do CIDR matching. This tutorial explains that if you create your own pipeline following this naming <type>-<dataset>@custom then Fleet will invoke that pipeline after it has parsed the data with the integration's main pipeline. (Be aware that there is a bug fix related to @custom pipelines that is not released yet [Fleet] Add the @custom pipeline only to the main datastream ingest pipeline by nchaulet · Pull Request #144150 · elastic/kibana · GitHub, it works but it has some unintended behavior).

Below is kind of a hack since it duplicates parsing, but this could be done via the processors in Agent. It exacts the IP to a temporary field such that you can use it in the add_fields.

- decode_csv_fields:
    fields:
      message: _tmp.csv
    fail_on_error: false
- extract_array:
    field: _tmp.csv
    mappings:
      _tmp.host.ip: 4
    fail_on_error: false
- add_fields:
    when.network._tmp.host.ip: 172.28.0.0/16
    target: host
    fields:
      geo:
        location:
          lat: 40.1
          lon: 56.0
- drop_fields:
    fields: [_tmp]
    ignore_missing: true

[chapmanjc (Joe)]

Hey Andrew,

I think I got it to work finally! I added a set processor to an ingest pipeline.

"set": {
      "field": "destination.geo.location",
      "value": "100.00, -100.00",
      "if": "ctx.host?.ip =~ /^10\\.113\\./"

I was still having a hell of a time to try and get CIDR notation to work but it seems to work as a regexp.