# Translate Regex Not Working

**URL:** https://discuss.elastic.co/t/translate-regex-not-working/265628
**Category:** Logstash
**Created:** [February 26, 2021, 3:45pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628 "2021-02-26T15:45:46Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 3:45pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/1 "2021-02-26T15:45:46Z")

</div>

Regular Expression on Translate filter isn't working as expected. All values in the destination field are matching `low`. Values will range from 0.0 to 10 with no more than 1 digit after the decimal.

```auto
translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0" => "none"
      "^0\.[1-9]|^[1-3]\.[0-9]" => "low"
      "^[4-6]\.[0-9]" => "medium"
      "^[7-8]\.[0-9]" => "high"
      "^9\.[0-9]|^10" => "critical"
    }
  }

```

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 4:18pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/2 "2021-02-26T16:18:28Z")

</div>

Looks like I failed to accomodate for non-float numbers. I've modified my expressions to account for whole numbers, but it still hasn't resolved my issue.

```auto
  translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
  }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 26, 2021, 5:15pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/3 "2021-02-26T17:15:08Z")

</div>

That filter works for me regardless of whether [cvss][score] is a string or a float.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 6:01pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/4 "2021-02-26T18:01:47Z")

</div>

I figured out how to make it work, kinda odd though. So the field originally comes in with the name `cvss`. Using mutate, I rename the field name to `[cvss][score]`. Performing translation directly on this wasn't working, but when I use mutate's convert function, the translation works properly.

Not sure why this is necessary, as the indexed value is seen as a number, maybe Elasticsearch is doing the conversion?

```auto
  mutate {
    rename => {
      "cvss" => "[cvss][score]"
    }
    convert => {
      "[cvss][score]" => "float"
    }
  }
  translate {
    field => "[cvss][score]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
  }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 26, 2021, 6:04pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/5 "2021-02-26T18:04:49Z")

</div>

> [@wwalker](#):
>
> maybe Elasticsearch is doing the conversion?

If you have [dynamic mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html) on (it is on by default) then numeric detection would make it a float if it looks like a float.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 6:06pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/6 "2021-02-26T18:06:39Z")

</div>

Dynamic mapping IS enabled, but I have this field explicitly defined in a template, does that matter?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 26, 2021, 6:12pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/7 "2021-02-26T18:12:34Z")

</div>

Dynamic mapping will not override a template, as far as I know.

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 6:17pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/8 "2021-02-26T18:17:04Z")

</div>

Ugh...just finished ingesting the entire dataset again and while everything is not hitting the "low" translation, the value of `10` is not matching the final regex, "high". Seems like this would be a straightforward translation but apparently, it is not.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 26, 2021, 6:22pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/9 "2021-02-26T18:22:10Z")

</div>

> [@wwalker](#):
>
> the value of `10` is not matching the final regex, "high"

You mean "critical"?

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 6:29pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/10 "2021-02-26T18:29:17Z")

</div>

Yes I'm sorry, it's not matching "critical". I've isolated an event for testing and changed things up a bit to see what happens. I figured this would result in an Elasticsearch indexing error but instead got a....weird...error.

```auto
  translate {
    field => "[cvss]"
    destination => "[cvss][rating]"
    regex => true
    exact => true
    dictionary => {
      "^0$|0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"
    }
    fallback => "none"
  }
  mutate {
    rename => {
      "cvss" => "[cvss][score]"
    }
  }

```

Error:

```auto
Something went wrong when attempting to translate from dictionary {:exception=>org.logstash.Accessors$InvalidFieldSetException: Could not set field 'rating' on object '0.1e2' to value 'low'.This is probably due to trying to set a field like [foo][bar] = someValuewhen [foo] is not either a map or a string, :field=>"[cvss]", :event=>#<LogStash::Event:0x43cd0a35>}

```

Original data being ingested:

```auto
{"Modified": "2021-02-12 01:00:00", "Published": "2021-02-10 19:15:00", "access": {"authentication": "NONE", "complexity": "LOW", "vector": "NETWORK"}, "assigner": "cve@mitre.org", "capec": [], "cvss": 10.0, "cvss-time": "2021-02-12 01:00:00", "cvss-vector": "AV:N/AC:L/Au:N/C:C/I:C/A:C", "cwe": "CWE-787", "id": "CVE-2021-27171", "impact": {"availability": "COMPLETE", "confidentiality": "COMPLETE", "integrity": "COMPLETE"}, "last-modified": {"$date": 1613091600000}, "references": ["https://pierrekim.github.io/blog/2021-01-12-fiberhome-ont-0day-vulnerabilities.html#telnet-cli-privilege-escalation"], "summary": "An issue was discovered on FiberHome HG6245D devices through RP2613. It is possible to start a Linux telnetd as root on port 26/tcp by using the CLI interface commands of ddd and shell (or tshell).", "vulnerable_configuration": [{"id": "cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*", "title": "cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*"}, {"id": "cpe:2.3:h:fiberhome:hg6245d:-:*:*:*:*:*:*:*", "title": "cpe:2.3:h:fiberhome:hg6245d:-:*:*:*:*:*:*:*"}], "vulnerable_configuration_cpe_2_2": [], "vulnerable_product": ["cpe:2.3:o:fiberhome:hg6245d_firmware:*:*:*:*:*:*:*:*"]}

```

Actually, reading it again, that was the error I expected to get, just got caught by Logstash. What I thought was weird was the `0.1e2`....is it saying that is the field value it's seeing for cvss?

---

<div class="post-metadata">

### Author: ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)
#### Post date: [February 26, 2021, 6:45pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/11 "2021-02-26T18:45:37Z")

</div>

Turns out my "none" expression wasn't properly anchored, causing the incorrect match. Below is correct and everything SEEMS to be working now.

```auto
      "^0$|^0\.0|^$" => "none"
      "^0\.[1-9]|^[1-3](\.[0-9])?$" => "low"
      "^[4-6](\.[0-9])?" => "medium"
      "^[7-8](\.[0-9])?" => "high"
      "^9(\.[0-9])?|^10" => "critical"

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 26, 2021, 7:02pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/12 "2021-02-26T19:02:38Z")

</div>

> [@wwalker](#):
>
> `:exception=>org.logstash.Accessors$InvalidFieldSetException: Could not set field 'rating' on object '0.1e2' to value 'low'.`

That is telling you that [cvss] is a field containing a value, not an object, so you cannot create [cvss][rating].

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 26, 2021, 7:02pm UTC](https://discuss.elastic.co/t/translate-regex-not-working/265628/13 "2021-03-26T19:02:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
