Translate Regex Not Working

Elastic Stack Logstash
1

Regular Expression on Translate filter isn't working as expected. All values in the destination field are matching low. Values will range from 0.0 to 10 with no more than 1 digit after the decimal.

translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0" => "none"
      "^0\.[1-9]|^[1-3]\.[0-9]" => "low"
      "^[4-6]\.[0-9]" => "medium"
      "^[7-8]\.[0-9]" => "high"
      "^9\.[0-9]|^10" => "critical"
    }
  }
2

Looks like I failed to accomodate for non-float numbers. I've modified my expressions to account for whole numbers, but it still hasn't resolved my issue.

  translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
  }
3

That filter works for me regardless of whether [cvss][score] is a string or a float.

4

I figured out how to make it work, kinda odd though. So the field originally comes in with the name cvss. Using mutate, I rename the field name to [cvss][score]. Performing translation directly on this wasn't working, but when I use mutate's convert function, the translation works properly.

Not sure why this is necessary, as the indexed value is seen as a number, maybe Elasticsearch is doing the conversion?

  mutate {
    rename => {
      "cvss" => "[cvss][score]"
    }
    convert => {
      "[cvss][score]" => "float"
    }
  }
  translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
  }
5

If you have dynamic mapping on (it is on by default) then numeric detection would make it a float if it looks like a float.

6

Dynamic mapping IS enabled, but I have this field explicitly defined in a template, does that matter?

7

Dynamic mapping will not override a template, as far as I know.

1 Like
8

Ugh...just finished ingesting the entire dataset again and while everything is not hitting the "low" translation, the value of 10 is not matching the final regex, "high". Seems like this would be a straightforward translation but apparently, it is not.

9

You mean "critical"?

10

Yes I'm sorry, it's not matching "critical". I've isolated an event for testing and changed things up a bit to see what happens. I figured this would result in an Elasticsearch indexing error but instead got a....weird...error.

  translate {
    field => "[cvss]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
    fallback => "none"
  }
  mutate {
    rename => {
      "cvss" => "[cvss][score]"
    }
  }

Error:

Something went wrong when attempting to translate from dictionary {:exception=>org.logstash.Accessors$InvalidFieldSetException: Could not set field 'rating' on object '0.1e2' to value 'low'.This is probably due to trying to set a field like [foo][bar] = someValuewhen [foo] is not either a map or a string, :field=>"[cvss]", :event=>#<LogStash::Event:0x43cd0a35>}

Original data being ingested:

{"Modified": "2021-02-12 01:00:00", "Published": "2021-02-10 19:15:00", "access": {"authentication": "NONE", "complexity": "LOW", "vector": "NETWORK"}, "assigner": "cve@mitre.org", "capec": [], "cvss": 10.0, "cvss-time": "2021-02-12 01:00:00", "cvss-vector": "AV:N/AC:L/Au:N/C:C/I:C/A:C", "cwe": "CWE-787", "id": "CVE-2021-27171", "impact": {"availability": "COMPLETE", "confidentiality": "COMPLETE", "integrity": "COMPLETE"}, "last-modified": {"$date": 1613091600000}, "references": ["https://pierrekim.github.io/blog/2021-01-12-fiberhome-ont-0day-vulnerabilities.html#telnet-cli-privilege-escalation"], "summary": "An issue was discovered on FiberHome HG6245D devices through RP2613. It is possible to start a Linux telnetd as root on port 26/tcp by using the CLI interface commands of ddd and shell (or tshell).", "vulnerable_configuration": [{"id": "cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*", "title": "cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*"}, {"id": "cpe:2.3:h:fiberhome:hg6245d:-:*:*:*:*:*:*:*", "title": "cpe:2.3:h:fiberhome:hg6245d:-:*:*:*:*:*:*:*"}], "vulnerable_configuration_cpe_2_2": [], "vulnerable_product": ["cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*"]}

Actually, reading it again, that was the error I expected to get, just got caught by Logstash. What I thought was weird was the 0.1e2....is it saying that is the field value it's seeing for cvss?

11

Turns out my "none" expression wasn't properly anchored, causing the incorrect match. Below is correct and everything SEEMS to be working now.

      "^0$|^0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
12

That is telling you that [cvss] is a field containing a value, not an object, so you cannot create [cvss][rating].

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.