# Ingest pipeline - extract regex from events

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-extract-regex-from-events/345133>\
**Category:** Elasticsearch\
**Tags:** painless, ingest-pipeline\
**Created:** [October 16, 2023, 6:58pm UTC](https://discuss.elastic.co/t/ingest-pipeline-extract-regex-from-events/345133 "2023-10-16T18:58:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![xyz3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xyz3/32/106931_2.png) [@xyz3](https://discuss.elastic.co/u/xyz3)\
**Post date:** [October 16, 2023, 6:58pm UTC](https://discuss.elastic.co/t/ingest-pipeline-extract-regex-from-events/345133/1 "2023-10-16T18:58:55Z")

</div>

Hello  
I need to extract some text string from existing index field: IMSChargingIdentifier and place it into new, separate field.  
New, incoming events should be parsed exactly the same way.

Here is sample record from this field:

```auto
     PCSF:1-osbc01cfed-0-0-0000000065154a29-00000000001f4cdd
 _id
     Q5c724oBfJ0gntMI5D_h
 _index
     occf-test-index
 _score
     1

```

String I'm interested in is after dash - first letter, then osbc and next two digits.  
What is the recommended way to achieve this?

- custom GROK pattern in the ingest pipeline?
- painless script with regex?

I 've tried use some pattern\_definitions but I can't manage it  
Would be appreciate for any suggestions.

```auto
 PUT /_ingest/pipeline/my-ingest-pipeline
 {
   "description": "extract ImsChargingIdentifier",
   "processors": [
     {
       "grok": {
         "field": "occf.IMSChargingIdentifier",
         "patterns": ["%codeSBC:NewField"],
         "pattern_definitions": {
           "codeSbc": "[%{SPACE:spaces}%{WORD:1st-word}.....what else...?]
         }
       }
     }
   ]
 }

```

---

<div class="post-metadata">

**Author:** ![xyz3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xyz3/32/106931_2.png) [@xyz3](https://discuss.elastic.co/u/xyz3)\
**Post date:** [October 17, 2023, 6:36am UTC](https://discuss.elastic.co/t/ingest-pipeline-extract-regex-from-events/345133/2 "2023-10-17T06:36:18Z")

</div>

Hello again!

I got following custom grok definition:

```auto
PUT /_ingest/pipeline/my-ingest-pipeline
{
  "description": "extract ImsChargingIdentifier",
  "processors": [
    {
      "grok": {
        "field": "occf.IMSChargingIdentifier",
        "patterns": ["%codeSBC:NewField"],
        "pattern_definitions": {
          "codeSbc": "[%{WORD:prefix}:%{NUMBER:value}-%{WORD:code}]"
        }
      }
    }
  ]
}

```

When testing sample data in grok debbugger it gives me:

```auto
{
  "code": "osbc01cfed",
  "value": "1",
  "prefix": "PCSF"
}

```

but when i try to update index by query with this pipeline:

```auto
POST /occf-test-index/_update_by_query?pipeline=my-ingest-pipeline
{
  "script": {
    "source": "ctx.op = 'index'; ctx._source['NewField'] = ctx.value;",
    "lang": "painless"
  }
}

```

I get an error:

```auto
      "index": "occf-test-index",
      "id": "Q5c724oBfJ0gntMI5D_h",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "Provided Grok expressions do not match field value: [PCSF:1-osbc01cfed-0-0-0000000065154a29-00000000001f4cdd]"
      },

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 14, 2023, 6:37am UTC](https://discuss.elastic.co/t/ingest-pipeline-extract-regex-from-events/345133/3 "2023-11-14T06:37:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.