# Issue with Multiple grok patterns in single log ingest pipeline

**URL:** https://discuss.elastic.co/t/issue-with-multiple-grok-patterns-in-single-log-ingest-pipeline/332842
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [May 8, 2023, 7:55pm UTC](https://discuss.elastic.co/t/issue-with-multiple-grok-patterns-in-single-log-ingest-pipeline/332842 "2023-05-08T19:55:07Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Jason\_Paralta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_paralta/32/118856_2.png) [@Jason\_Paralta](https://discuss.elastic.co/u/Jason_Paralta)
#### Post date: [May 8, 2023, 7:55pm UTC](https://discuss.elastic.co/t/issue-with-multiple-grok-patterns-in-single-log-ingest-pipeline/332842/1 "2023-05-08T19:55:07Z")

</div>

Hi All,

I am trying to read different log paths, this logs are of different patterns. I have single elastic template for this and have settings of default pipeline configured on same template. Now in log ingest pipeline while giving different patterns only one pattern is getting parsed and other is not.

Below is the pipeline:

```auto
PUT _ingest/pipeline/abc-ingest-pipeline
{
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          "%{TIMESTAMP_ISO8601:@timestamp:date}-%{WORD:host.name}-%{WORD:app.name}-%{WORD:request.id}-%{WORD:log.level}( *)-%{GREEDYDATA:log.logger}:%{NUMBER:log.origin.file.line:long} - (?<statement>(.|\n|\n)*)",
		  """%{TIMESTAMP_ISO8601:timestamp}-%{WORD:server_name}-%{DATA:python_module}-%{DATA:req_id}-%{LOGLEVEL:log_level}-%{DATA:method_name}:%{NUMBER:line_number} - %{WORD:usecase.uniqueId}\|%{WORD:abc.runningHost}\|%{WORD:abc.useCaseName}\|%{NUMBER:abc.pid}\|%{UNIXPATH:abc.logfile}\|%{TIMESTAMP_ISO8601:abc.startTime}\|%{TIMESTAMP_ISO8601:abc.endTime}\|%{DATA:abc.duration}\|%{TIMESTAMP_ISO8601:abc.nextEvent}\|%{NUMBER:abc.cpuUsage}\|%{NUMBER:abc.memoryUsage}\|%{NUMBER:abc.VM}\|%{NUMBER:abc.RM}\|%{NUMBER:abc.SM}\|%{DATA:abc.command}\|%{GREEDYDATA:abc.pythonPath}""" 
        ],
        "on_failure": [
          {
            "set": {
              "field": "error.message",
              "value": "{{ _ingest.on_failure_message }}"
            }
          }
        ]
      }
    },
    {
      "trim": {
        "field": "statement"
      }
    },
    {
      "set": {
        "field": "log.original",
        "value": "{{message}}"
      }
    },
    {
      "remove": {
        "field": [
          "message"
        ]
      }
    },
    {
      "rename": {
        "field": "host.hostname",
        "target_field": "agent.hostname",
		"ignore_failure": true
      }
    },
    {
      "rename": {
        "field": "host.ip",
        "target_field": "agent.ip",
		"ignore_failure": true
      }
    }
  ]
}

```

Can someone suggest how can I configure above pipeline correctly to parse different log patterns for different log files executed by filebeat.

Note: I am not using logstash for this.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 5, 2023, 7:55pm UTC](https://discuss.elastic.co/t/issue-with-multiple-grok-patterns-in-single-log-ingest-pipeline/332842/2 "2023-06-05T19:55:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.