# Extract first 50 characters from message "body" into new field

**URL:** https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574
**Category:** Logstash
**Created:** [October 19, 2020, 3:25pm UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574 "2020-10-19T15:25:31Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![SecretSquizza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/secretsquizza/32/77449_2.png) [@SecretSquizza](https://discuss.elastic.co/u/SecretSquizza)
#### Post date: [October 19, 2020, 3:25pm UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/1 "2020-10-19T15:25:31Z")

</div>

Hello everyone,

Firstly, apologies for any formatting mistakes in this post - its my first one...

i'm relatively new to the Elastic stack and need to get a shortened version of the message body into another field so it can be used as a label within lens (I'm going to plump with the first 50 characters of the string).

Here's my logstash filter so far (this works as it simply copies the whole of "body" into a new field called "short\_body" (so i know that there's nothing fundamentally wrong with my filter, i'm just not understanding the Ruby part).

```
`filter {
  if "guidewire" in [tags] {
    grok {
      match => {
        "message" => "(?<serverid>([a-z]{3}[0-9]{2}[a-z]{2}[0-9]{2}){0,1})\s+(?<username>([A-Za-z0-9.@\-]*){0,1})\s+(%{TIMESTAMP_ISO8601:logdate})?\s+(%{LOGLEVEL:loglevel})\s+(?<classname>([^\s]+){0,1})\s+%{GREEDYDATA:body}"
      }
      add_field => ["received_at", "%{@timestamp}"]
      add_field => ["received_from", "%{host}"]
      tag_on_failure => ["_not_interested"]
    }
    ruby {
    code => 'shortbody = event.get("body")
             event.set("short_body",shortbody)'
    }
  if "_not_interested" in [tags] {
    drop { }
  }
  } else if "aggregator" in [tags] {
    grok {
      match => { "message" => "%{GREEDYDATA:body}" }
      add_field => ["received_at", "%{@timestamp}"]
      add_field => ["received_from", "%{host}"]
    }
  }
}`

```

The problem i have is that, whatever i do to try and substring the body field result in errors being written to the logstash log.

So far I've tried:

```
`code => 'shortbody = event.get("body")[0..50]
             event.set("short_body",shortbody)'`

```

which results in

Ruby exception occurred: undefined method `' for nil:NilClass

I tried many variations on this including using [0,50] as well as condensing the code to

`code=> 'event.set("short_body", event.get("body")[0..50])'`

Same problem

I also tried

```
`code => 'shortbody = event.get("body[0,50]")
             event.set("short_body",shortbody)'`

```

results in

Ruby exception occurred: Invalid FieldReference: `body[0,50]`

Can anyone offer any assistance , i thought i was trying to do something quite simple and I keep getting these errors thrown up and its driving me bananas.

Many thanks,  
Adrian.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 19, 2020, 5:15pm UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/2 "2020-10-19T17:15:04Z")

</div>

I don't know much of ruby, but you can do that using a simple grok filter.

Something like this:

```auto
 grok {
        match => { "message" => "(?<short_message>^.{0,50})" }
}

```

Testing this on my lab environment I get the following output.

```auto
{
         "@version" => "1",
             "host" => "elk",
    "short_message" => "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWX",
          "message" => "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ",
       "@timestamp" => 2020-10-19T17:14:07.417Z
}

```

My original `message` was 52 characters long, and my `short_message` is 50 characters long as you can see in the output.

In your case you would use your `body` and `short_body` fields.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 19, 2020, 5:21pm UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/3 "2020-10-19T17:21:14Z")

</div>

I agree that grok is the way to go.

Regarding the ruby errors

```
Ruby exception occurred: undefined method `' for nil:NilClass

```

That is telling you that event.get("body") returned nil, meaning there is an event that does not have a [body] field.

```
Ruby exception occurred: Invalid FieldReference: body[0,50]

```

To reference a field as an array you would use [body][0, 50]

---

<div class="post-metadata">

### Author: ![SecretSquizza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/secretsquizza/32/77449_2.png) [@SecretSquizza](https://discuss.elastic.co/u/SecretSquizza)
#### Post date: [October 19, 2020, 6:56pm UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/4 "2020-10-19T18:56:11Z")

</div>

Will the grok filter work?

I already have a match filter that takes message and part of that message (the tail end) becomes the body field. It's this body field (not the whole message) i need to substring.

Great if it does, i'm just not sure how to get this to work.

---

<div class="post-metadata">

### Author: ![SecretSquizza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/secretsquizza/32/77449_2.png) [@SecretSquizza](https://discuss.elastic.co/u/SecretSquizza)
#### Post date: [October 20, 2020, 7:44am UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/5 "2020-10-20T07:44:15Z")

</div>

Using another match filter on body did not seem to work. Logstash doesn't throw any log errors but there's no "short\_body" field coming through:

```
match => {
        "message" => "(?<serverid>([a-z]{3}[0-9]{2}[a-z]{2}[0-9]{2}){0,1})\s+(?<username>([A-Za-z0-9.@\-]*){0,1})\s+(%{TIMESTAMP_ISO8601:logdate})?\s+(%{LOGLEVEL:loglevel})\s+(?<classname>([^\s]+){0,1})\s+%{GREEDYDATA:body}"
        "body" => "(?<short_body>^.{0,50})"
      }

```

Sorry if i'm not getting what you are saying ... any further ideas?

Many thanks,  
Adrian.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 17, 2020, 7:44am UTC](https://discuss.elastic.co/t/extract-first-50-characters-from-message-body-into-new-field/252574/6 "2020-11-17T07:44:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.