# How come my ElasticSearch ingest pipelines with multiple grok patterns are all failing?

**URL:** <https://discuss.elastic.co/t/how-come-my-elasticsearch-ingest-pipelines-with-multiple-grok-patterns-are-all-failing/119702>\
**Category:** Elasticsearch\
**Created:** [February 13, 2018, 7:52pm UTC](https://discuss.elastic.co/t/how-come-my-elasticsearch-ingest-pipelines-with-multiple-grok-patterns-are-all-failing/119702 "2018-02-13T19:52:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ujjain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ujjain/32/27378_2.png) [@ujjain](https://discuss.elastic.co/u/ujjain)\
**Post date:** [February 13, 2018, 7:52pm UTC](https://discuss.elastic.co/t/how-come-my-elasticsearch-ingest-pipelines-with-multiple-grok-patterns-are-all-failing/119702/1 "2018-02-13T19:52:18Z")

</div>

I'm working on getting the Tomcat logs of some applications correctly processed in ElasticSearch, but unfortunately my ingest pipelines with multiple grok processors aren't working and they all end up in the failed-index.

catalina.out

```
13-Feb-2018 16:04:48.446 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxGCPauseMillis=200

```

catalina.log

```
13-Feb-2018 16:00:35.385 SEVERE [monkey.co.uk-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/opt/tomcat/apache-tomcat-8.5.16/webapps/email.war]
 java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[monkey.co.uk].StandardContext[/email]]
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:756)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:988)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1860)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

```

This is my ingest pipeline:

```
PUT _ingest/pipeline/tomcat_all

{  
   "description":"Pipeline tomcat_all",
   "on_failure":[  
      {  
         "set":{  
            "field":"_index",
            "value":"failed-{{ _index }}"
         }
      }
   ],
   "processors":[  
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}(?:AM|PM)%{SPACE}%{NOTSPACE:class}%{SPACE}%{NOTSPACE:type_log}%{SPACE}%{WORD:loglevel}:%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{TIME:timestamp}%{SPACE}\\|-%{WORD:loglevel}%{SPACE}in%{SPACE}%{NOTSPACE:class}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{MONTHDAY}[/-]%{MONTH}[/-]%{YEAR} %{TIME}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}(?:AM|PM)%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{YEAR}-%{MONTHNUM}-%{MONTHDAY}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}%{ISO8601_TIMEZONE}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      },
      {  
         "grok":{  
            "field":"message",
            "patterns":[  
               "%{MONTHDAY}.%{MONTHNUM}.%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}"
            ]
         }
      }
   ]
}

```

Grok patterns:

```
%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}(?:AM|PM)%{SPACE}%{NOTSPACE:class}%{SPACE}%{NOTSPACE:type_log}%{SPACE}%{WORD:loglevel}:%{SPACE}%{GREEDYDATA:log_text}
%{TIME:timestamp}%{SPACE}\\|-%{WORD:loglevel}%{SPACE}in%{SPACE}%{NOTSPACE:class}%{SPACE}%{GREEDYDATA:log_text}
"%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}
%{MONTHDAY}[/-]%{MONTH}[/-]%{YEAR} %{TIME}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}
%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}(?:AM|PM)%{SPACE}%{GREEDYDATA:log_text}
%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}
%{YEAR}-%{MONTHNUM}-%{MONTHDAY}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}%{ISO8601_TIMEZONE}%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}
%{MONTHDAY}.%{MONTHNUM}.%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}%{WORD:loglevel}%{SPACE}%{GREEDYDATA:log_text}

```

I have tried the grok patterns manually at [http://grokdebug.herokuapp.com](http://grokdebug.herokuapp.com) and that worked fine.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 13, 2018, 7:52pm UTC](https://discuss.elastic.co/t/how-come-my-elasticsearch-ingest-pipelines-with-multiple-grok-patterns-are-all-failing/119702/2 "2018-03-13T19:52:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.