# What does the field in Grok "message" do?

**URL:** <https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253>\
**Category:** Logstash\
**Created:** [June 9, 2015, 8:24pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253 "2015-06-09T20:24:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![michaellizhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaellizhou/32/4143_2.png) [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\
**Post date:** [June 9, 2015, 8:24pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/1 "2015-06-09T20:24:42Z")

</div>

Hi first post and new to the ELK system. So I am working on parsing through log4j logs and I cannot find what is the purpose of the "message" field. For example:

```
filter {
  grok {
    match => {"message =>

```

I am so confused on if it is just a naming convention or standard. This is probably a really basic question but I need to find out before moving on. Is message just something I can reference back to if I need to?

Thanks.  
Mike

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 9, 2015, 8:29pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/2 "2015-06-09T20:29:29Z")

</div>

The `message` field is like a default field. It's where most input plugins place the payload that they receive from the network, read from a file, or whatever. So no, it's not just a convention.

In many log formats the message field starts with a timestamp, maybe a severity level, possibly a hostname, and so on, and ends with the actual message. In such cases one typically extract the timestamp etc into fields of their own and remove them from the `message` field. In other cases like HTTP logs there is no free-text message.

---

<div class="post-metadata">

**Author:** ![michaellizhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaellizhou/32/4143_2.png) [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\
**Post date:** [June 9, 2015, 8:37pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/3 "2015-06-09T20:37:43Z")

</div>

Oh alright so the message will hold the bulk of default fields. Does this mean that after I write my message field to store say the timestamp, level, groupId...etc underneath I would filter out more of the log message. i.e

```
filter {
  grok{
    match => {"message" => {SYSLOGTIMESTAMP:installTime}
    match => {"someotherfield" => {more grok or regex}

```

Thanks for help!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 9, 2015, 8:56pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/4 "2015-06-09T20:56:04Z")

</div>

I'm not following, but let's take syslog messages as an example. Logstash ships with some grok patterns for syslog messages, like SYSLOGBASE which is defined like this:

```
$ grep SYSLOGBASE /opt/logstash/patterns/grok-patterns
SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:

```

When used like

```
filter {
  grok {
    match => ["message", "%{SYSLOGBASE}"]
  }
}

```

Logstash will attempt the match and extract the fields `timestamp`, `program`, `pid`, `logsource`, `facility`, `priority` and a few others from the `message` field. Then we could end up with a message looking like this:

```
{
  "logsource": "somehostname",
  "timestamp": "Jun 1 07:50:01",
  "program": "CRON",
  "pid": "22912",
  "message": "Jun 1 07:46:01 somehostname CRON[22912]: (root) CMD ( /path/to/some/program > /dev/null 2>&1)"
}

```

By changing the filter to

```
filter {
  grok {
    match => ["message", "%{SYSLOGBASE} %{GREEDYDATA:message}"]
    overwrite => ["message"]
  }
}

```

we capture the actual message part of the original message and save it back into the message field, yielding:

```
{
  "logsource": "somehostname",
  "timestamp": "Jun 1 07:50:01",
  "program": "CRON",
  "pid": "22912",
  "message": "(root) CMD ( /path/to/some/program > /dev/null 2>&1)"
}

```

Now things are starting to look useful.

---

<div class="post-metadata">

**Author:** ![michaellizhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaellizhou/32/4143_2.png) [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\
**Post date:** [June 9, 2015, 10:17pm UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/5 "2015-06-09T22:17:28Z")

</div>

I think I am starting to understand more now. So the SYSLOGBASE from your example picks up most of the overhead that comes with the log. Message takes everything afterwards and from your example you want message to be "(root) CMD..." Does this mean I can call message anything else i.e. data or msg?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 10, 2015, 3:48am UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/6 "2015-06-10T03:48:47Z")

</div>

Sure. There might be specific exceptions for some output plugins, but what you call the output fields is generally up to you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:37am UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253/7 "2017-07-06T05:37:56Z")

</div>