Ingest pipeline - extract regex from events

Elastic Stack Elasticsearch
,
1

Hello
I need to extract some text string from existing index field: IMSChargingIdentifier and place it into new, separate field.
New, incoming events should be parsed exactly the same way.

Here is sample record from this field:

     PCSF:1-osbc01cfed-0-0-0000000065154a29-00000000001f4cdd
 _id
     Q5c724oBfJ0gntMI5D_h
 _index
     occf-test-index
 _score
     1

String I'm interested in is after dash - first letter, then osbc and next two digits.
What is the recommended way to achieve this?

  • custom GROK pattern in the ingest pipeline?
  • painless script with regex?

I 've tried use some pattern_definitions but I can't manage it
Would be appreciate for any suggestions.

 PUT /_ingest/pipeline/my-ingest-pipeline
 {
   "description": "extract ImsChargingIdentifier",
   "processors": [
     {
       "grok": {
         "field": "occf.IMSChargingIdentifier",
         "patterns": ["%codeSBC:NewField"],
         "pattern_definitions": {
           "codeSbc": "[%{SPACE:spaces}%{WORD:1st-word}.....what else...?]
         }
       }
     }
   ]
 }
2

Hello again!

I got following custom grok definition:

PUT /_ingest/pipeline/my-ingest-pipeline
{
  "description": "extract ImsChargingIdentifier",
  "processors": [
    {
      "grok": {
        "field": "occf.IMSChargingIdentifier",
        "patterns": ["%codeSBC:NewField"],
        "pattern_definitions": {
          "codeSbc": "[%{WORD:prefix}:%{NUMBER:value}-%{WORD:code}]"
        }
      }
    }
  ]
}

When testing sample data in grok debbugger it gives me:

{
  "code": "osbc01cfed",
  "value": "1",
  "prefix": "PCSF"
}

but when i try to update index by query with this pipeline:

POST /occf-test-index/_update_by_query?pipeline=my-ingest-pipeline
{
  "script": {
    "source": "ctx.op = 'index'; ctx._source['NewField'] = ctx.value;",
    "lang": "painless"
  }
}

I get an error:

      "index": "occf-test-index",
      "id": "Q5c724oBfJ0gntMI5D_h",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "Provided Grok expressions do not match field value: [PCSF:1-osbc01cfed-0-0-0000000065154a29-00000000001f4cdd]"
      },
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.