# Using ingest pipeline to parse two different log formats with multiple grok pattern
**URL:** https://discuss.elastic.co/t/using-ingest-pipeline-to-parse-two-different-log-formats-with-multiple-grok-pattern/102908
**Category:** Elasticsearch
**Created:** [October 5, 2017, 8:34pm UTC](https://discuss.elastic.co/t/using-ingest-pipeline-to-parse-two-different-log-formats-with-multiple-grok-pattern/102908 "2017-10-05T20:34:57Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@hungl99](https://discuss.elastic.co/u/hungl99)
#### Post date: [October 5, 2017, 8:34pm UTC](https://discuss.elastic.co/t/using-ingest-pipeline-to-parse-two-different-log-formats-with-multiple-grok-pattern/102908/1 "2017-10-05T20:34:57Z")
I'm using ingest nodes with pipeline. My log lines basically has two different formats. Basically we want to add another field named "controller" and be able to index it. Sample log lines:
2017-10-04 18:23:42 ERROR [34] GetAsync Exception, Url: api/vehicle/v2/inventory?bac=112093&vin=1GNSCBKC1FR507889&sellingsource=13&bfc=01&sourceZip=&destinationZip=&webId=&type=New&userId=, Response Content:
2017-10-04 18:21:55 ERROR [29] controller:|DS.Marketing.Platform.Deal.V1.Controllers.EligibilityController| DS.Marketing.Platform.Deal.V1.Domain.DeskingProviderNotFoundException: No desking provider has been assigned to this deal (9aac0c8e-a099-449e-9119-c79dbf937abc)
Here is my ingest pipeline but it's far from complete. Please help!
POST \_ingest/pipeline/\_simulate
{
"pipeline" :
{
"description": "grok\_controller\_mp\_combined",
"processors": [
{
"grok": {
"field": "message",
"patterns": ["%{CUSTOM\_CONTROLLER:log.controller}"],
"pattern\_definitions" : {
"CUSTOM\_CONTROLLER (^\s\*controller:.+)"
}
}
}]},
filter{
if ""%{CUSTOM\_CONTROLLER:log.controller}" in [message]
{
grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:log.datetime} %{DATA:log.level} \[%{DATA:log.thread}\] %{GREEDYDATA:log.controller}" } }
}
else
{
grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:log.datetime} %{DATA:log.level} \[%{DATA:log.thread}\] %{GREEDYDATA:log.message}" } }
}
}
"docs":[
{
"\_source": {"message":
"2017-05-09 14:51:37 ERROR [95] System.ArgumentException: Lead.PersonalIdentificationNumber or BusinessTaxId are required. \n at MP.StarbodLeadProcessing.Processing.StarbodMapping.AR.StarbodMappingSiebelArgentina.Validate(StarbodModel starbodModel"}
},
{
"\_source": {"message":
"2017-10-04 18:23:42 ERROR [34] GetAsync Exception, Url: api/vehicle/v2/inventory?bac=112093&vin=1GNSCBKC1FR507889&sellingsource=13&bfc=01&sourceZip=&destinationZip=&webId=&type=New&userId=, Response Content: "}
},
{
"\_source" {"message":"2017-10-04 18:21:55 ERROR [29] controller:|DS.Marketing.Platform.Deal.V1.Controllers.EligibilityController| DS.Marketing.Platform.Deal.V1.Domain.DeskingProviderNotFoundException: No desking provider has been assigned to this deal (9aac0c8e-a099-449e-9119-c79dbf937abc)."}
]
}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 2, 2017, 8:35pm UTC](https://discuss.elastic.co/t/using-ingest-pipeline-to-parse-two-different-log-formats-with-multiple-grok-pattern/102908/2 "2017-11-02T20:35:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.