Ingest Pipelin Grok Regex

Elastic Stack Kibana
1

So, I am trying to generate a Grok pattern that matches a part of a longer string. My string looks something like this: "Reason: SOME_UPPERCASE_INFO" and I only want to extract my SOME_UPPERCASE_INFO. Do you have any suggestion how I could solve this problem?

2

Hi,

you can use the following Grok pattern:

Reason: %{WORD:info}

Regards

3

Thank you so much but my problem is my other logs also have "reasons" and I really only want those who are upper case.

4

Hi Lehu,

Where exactly you are trying to setup this grok pattern? on logstash or Grok processor on Elasticsearch?

5

On Elasticsearch

6

You can use grok processor on Ingest pipeline -

Here is the complete example:

PUT _ingest/pipeline/extract_uppercase_and_spaces
{
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": ["Reason: %{GREEDYDATA:full_message}"]
      }
    },
    {
      "script": {
        "lang": "painless",
        "source": """
          String fullMessage = ctx.full_message;
          StringBuilder uppercaseAndSpaces = new StringBuilder();
          for (int i = 0; i < fullMessage.length(); i++) {
            char c = fullMessage.charAt(i);
            if (Character.isUpperCase(c) || Character.isSpaceChar(c)) {
              uppercaseAndSpaces.append(c);
            }
          }
          ctx.uppercase_and_spaces = uppercaseAndSpaces.toString();
        """
      }
    }
  ]
}

POST my_index/_doc?pipeline=extract_uppercase_and_spaces
{
  "message": "API is down. Here is the Reason: API key IS INVALID"
}

GET my_index/_search

Response

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "my_index",
        "_id": "U2DIWZABe46rvky5I6nu",
        "_score": 1,
        "_source": {
          "message": "API is down. Here is the Reason: API key IS INVALID",
          "full_message": "API key IS INVALID",
          "uppercase_and_spaces": "API  IS INVALID"
        }
      }
    ]
  }
}