# Grok pattern to add some metadata to every log

**URL:** <https://discuss.elastic.co/t/grok-pattern-to-add-some-metadata-to-every-log/259432>\
**Category:** Elasticsearch\
**Created:** [December 22, 2020, 10:26pm UTC](https://discuss.elastic.co/t/grok-pattern-to-add-some-metadata-to-every-log/259432 "2020-12-22T22:26:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Minor\_Vargas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/minor_vargas/32/81279_2.png) [@Minor\_Vargas](https://discuss.elastic.co/u/Minor_Vargas)\
**Post date:** [December 22, 2020, 10:26pm UTC](https://discuss.elastic.co/t/grok-pattern-to-add-some-metadata-to-every-log/259432/1 "2020-12-22T22:26:24Z")

</div>

Hello all,

I wrote a grok pattern that matches the beginning of each log line (Writing table data). I need to take the timestamp that I already matched and append it at the beginning of each log line. I'm facing 2 issues; The first one is that it's only matching the first result, and the second one is that I don't know how to append it.. This is to create a custom ingest pipeline. The grok pattern is this:

```auto
"(?m)%{DATESTAMP_OTHER:timestamp}%{GREEDYDATA} (?<PWD>pwd=)%{UNIXPATH:path}%{GREEDYDATA}(?<LOGSTART>Writing table data)%{GREEDYDATA}"

```

and the logs file goes more or like this

```auto
Sun Dec 13 08:00:15 CST 2020: Starting docker, stream=prod, version=latest, pwd=/home/cldap/tools/prod/CapacityWriter
adding java runtime options: -Xmx4096m -Xms2048m
Writing table data, snapshotId=364596, siteName=Singapore, podId=2, datacenter=datacenter, esxName=esxName
Exception thrown while executing class SQLURL.com: DB2 SQL Error: SQLCODE=-551, SQLSTATE=42501, SQLERRMC=CLDCAPAPPL;SELECT;CLDCAP.CLDCAPX_VMINFO_LATEST, DRIVER=4.23.42
    Failed to write data to kafka, streamName=xseries.vminfo, java.lang.Exception: SQL failed to read data, SELECT * FROM CLDCAPX_VMINFO_LATEST WHERE SNAPSHOT_ID=?
Writing table data, snapshotId=364596, siteName=Singapore, podId=2, datacenter=datacenter, esxName=esxName
Exception thrown while executing class SQLURL.com: DB2 SQL Error: SQLCODE=-551, SQLSTATE=42501, SQLERRMC=CLDCAPAPPL;SELECT;CLDCAP.CLDCAPX_VMINFO_LATEST, DRIVER=4.23.42
    Failed to write data to kafka, streamName=xseries.vminfo, java.lang.Exception: SQL failed to read data, SELECT * FROM CLDCAPX_VMINFO_LATEST WHERE SNAPSHOT_ID=?
Sun Dec 13 08:01:45 CST 2020: Finished, rc=0
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 23, 2020, 6:48am UTC](https://discuss.elastic.co/t/grok-pattern-to-add-some-metadata-to-every-log/259432/2 "2020-12-23T06:48:39Z")

</div>

Grok is used to find and extract data - it does not modify it. You probably therefore need to extract the data using the grok filter and combine it with some other filter(s) to append.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 20, 2021, 9:32pm UTC](https://discuss.elastic.co/t/grok-pattern-to-add-some-metadata-to-every-log/259432/4 "2021-01-20T21:32:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.