# Parsing Pentaho logs using Grok in Logstash config

**URL:** <https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572>\
**Category:** Logstash\
**Created:** [February 8, 2019, 5:50am UTC](https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572 "2019-02-08T05:50:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![prasannnn](https://avatars.discourse-cdn.com/v4/letter/p/a587f6/32.png) [@prasannnn](https://discuss.elastic.co/u/prasannnn)\
**Post date:** [February 8, 2019, 5:50am UTC](https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572/1 "2019-02-08T05:50:24Z")

</div>

HI Team,

I have installed ELK 6.4.2. I have Pentaho logs generated from my applications. I have successfully installed and configured Elastic, Kibana, Logstash and Filebeat. The generated logs are consumed by the Filebeat and I am able to index them and see the the data in the discover tab of Kibana.

I am trying to index the logs generated by the ETL tool called Pentaho DI. I am using Grok pattern in the Logstash config file. here are a few lines of the log messages that are generated by the Pentaho ETL tool.

2019/02/04 15:19:42 - Pick JOB\_NAME.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)

2019/02/04 15:19:42 - Get JOB table.0 - Finished processing (I=1, O=0, R=1, W=1, U=0, E=0)

2019/02/04 15:19:42 - Does JOB\_NAME Exist?.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)

2019/02/04 15:19:42 - Get Inprocess Dependent jobs.0 - Finished processing (I=1, O=0, R=1, W=1, U=0, E=0)

2019/02/04 15:19:42 - Filter rows.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)

2019/02/04 15:19:42 - Set up 'In Process'.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)

I am able to index the files via filebeat and see the data in the Discover tab. However, I am having issues in querying the data.

I have set the below pattern for the logs in the grok filter

grok {

```
match => {"message" => "%{DATESTAMP:LogTimeStamp} - %{GREEDYDATA:StepName}- %{GREEDYDATA:LogMessage}" }

```

}

I am able to see the 3 fileds LogTimeStamp, StepName and LogMessage. The search query returns values only when the complete text in the field values are used. I need to be able to search partial words in the fields.

for eg: the below query does not return any result. however when LogMessage is substituted for the whole text, "Finished processing (I=1, O=0, R=1, W=1, U=0, E=0)", then it does result in showing up the data.

GET filebeat-6.4.2-2019.01.23/\_search

{

"query" : {

```
"term" : {

  "LogMessage": "R=1"

}

```

}

}

Please suggest me how the GROK pattern should be used to parse the above logs so that I can query the fields just by searching for lines of logs which has E=1 or R=1 etc.

Also, Once I am able to filter by the search string. I would like to see the next 5 to 10 lines in the logs to access the error based on the log messages.

Thanks a lot in advance

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 8, 2019, 5:50am UTC](https://discuss.elastic.co/t/parsing-pentaho-logs-using-grok-in-logstash-config/167572/2 "2019-03-08T05:50:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.