# How to generate single object from multiple log lines in logstash filter?

**URL:** <https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215>\
**Category:** Logstash\
**Created:** [January 12, 2018, 5:52am UTC](https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215 "2018-01-12T05:52:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohits1995](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mohits1995](https://discuss.elastic.co/u/mohits1995)\
**Post date:** [January 12, 2018, 5:52am UTC](https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215/1 "2018-01-12T05:52:11Z")

</div>

Hi,  
I am new to Logstash and Grok filter. I want to parse logs like these -

```auto
2018-01-11 17:17:16,071 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | CommittedVirtualMemorySize :: 401186816 
2018-01-11 17:17:16,071 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | FreePhysicalMemorySize :: 1751130112 
2018-01-11 17:17:16,072 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | FreeSwapSpaceSize :: 4294967295 
2018-01-11 17:17:16,694 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | ProcessCpuLoad :: -1.0 
2018-01-11 17:17:16,694 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | ProcessCpuTime :: 47471104300 
2018-01-11 17:17:16,698 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | SystemCpuLoad :: 1.0 
2018-01-11 17:17:16,698 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | TotalPhysicalMemorySize :: 4285849600 
2018-01-11 17:17:16,698 | DEBUG | [Thread-2] | com.example.monitor.MonitorHelper:cpuMonitoring(307) | TotalSwapSpaceSize :: 4294967295 

```

to an object like this -

```auto
{
  "timestamp": "2018-01-11 17:17:16,071",
  "log_level": "DEBUG",
  "thread_name": "Thread-2",
  "class": "com.example.monitor.MonitorHelper",
  "method": "cpuMonitoring",
  "line_number": "307",
  "CommittedVirtualMemorySize": "401186816",
  "FreePhysicalMemorySize": "1751130112",
  "FreeSwapSpaceSize": "4294967295",
  "ProcessCpuLoad": "-1.0",
  "ProcessCpuTime": "47471104300",
  "SystemCpuLoad": "1.0",
  "TotalPhysicalMemorySize": "4285849600",
  "TotalSwapSpaceSize": "4294967295"
}

```

As of now my grok pattern is -

```auto
%{TIMESTAMP_ISO8601:timestamp} \| %{LOGLEVEL:log_level} \| \[(?<thread_name>\b[\w\-]+\b)\] \| %{JAVAFILE:class}\:%{JAVAMETHOD:method}\(%{NUMBER:line_number}\) \| %{GREEDYDATA:log_message}

```

which provides multiple output lines for each input log line. JSON object looks like this-

```auto
{
  "timestamp": "2018-01-11 17:17:16,071",  
  "log_level": "DEBUG",
  "thread_name": "Thread-2",
  "class": "com.example.monitor.MonitorHelper",
  "method": "cpuMonitoring",
  "line_number": "307",
  "log_message": "CommittedVirtualMemorySize :: 401186816 "
}

```

can you please help me with what I need to look for in order to achieve this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 16, 2018, 4:30pm UTC](https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215/2 "2018-01-16T16:30:47Z")

</div>

The aggregate filter should be able to help you.

---

<div class="post-metadata">

**Author:** ![mohits1995](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mohits1995](https://discuss.elastic.co/u/mohits1995)\
**Post date:** [January 16, 2018, 4:46pm UTC](https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215/3 "2018-01-16T16:46:03Z")

</div>

Thanks for helping me out and responding. I will look into aggregate filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2018, 4:46pm UTC](https://discuss.elastic.co/t/how-to-generate-single-object-from-multiple-log-lines-in-logstash-filter/115215/4 "2018-02-13T16:46:13Z")

</div>

Closed on February 13, 2018, 4:46pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.