# Multiline Pattern - Filebeat

**URL:** <https://discuss.elastic.co/t/multiline-pattern-filebeat/199409>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 13, 2019, 10:24am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409 "2019-09-13T10:24:38Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 13, 2019, 10:24am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/1 "2019-09-13T10:24:38Z")

</div>

Hello, I am having problems making a multiline match. The logs come from a file saved on the client machine. I send to the logstash server with filebeat. In the filebeat configuration I have the following:

```
multiline.pattern: '^%{LOGLEVEL} %{TIMESTAMP_ISO8601} \['
multiline.negate: true
multiline.match: after

```

The log file looks like this:

```
ERROR 2019-09-13 08:27:24,794 [pool-2-thread-17604] com.mirth.connect.server.userutil.MessageObject: The messageObject.getRawData() method is deprecated and will soon be removed. Please use connectorMessage.getRawData() instead.
ERROR 2019-09-13 08:27:24,794 [pool-2-thread-17604] com.mirth.connect.server.userutil.ResponseFactory: The getSuccessResponse(message) method is deprecated and will soon be removed. Please use getSentResponse(message) instead.
ERROR 2019-09-13 08:27:47,515 [Timer-40] com.mirth.connect.connectors.jdbc.DatabaseReceiver: Failed to poll for messages from the database in channel
com.mirth.connect.connectors.jdbc.DatabaseReceiverException: Unrecognized value returned from script in channel expected ResultSet or List<Map<String, Object>>: org.mozilla.javascript.Undefined@76ea45d7
	at com.mirth.connect.connectors.jdbc.DatabaseReceiverScript.poll(DatabaseReceiverScript.java:112)
	at com.mirth.connect.connectors.jdbc.DatabaseReceiver.poll(DatabaseReceiver.java:111)
	at com.mirth.connect.donkey.server.channel.PollConnector$PollConnectorTask.run(PollConnector.java:141)
	at java.util.TimerThread.mainLoop(Unknown Source)
	at java.util.TimerThread.run(Unknown Source)
ERROR 2019-09-13 08:27:47,906 [Timer-47] com.mirth.connect.connectors.jdbc.DatabaseReceiver: Failed to poll for messages from the database in channel "Tal"
com.mirth.connect.connectors.jdbc.DatabaseReceiverException: Unrecognized value returned from script in channel "Tal", expected ResultSet or List<Map<String, Object>>: org.mozilla.javascript.Undefined@76ea45d7
	at com.mirth.connect.connectors.jdbc.DatabaseReceiverScript.poll(DatabaseReceiverScript.java:112)
	at com.mirth.connect.connectors.jdbc.DatabaseReceiver.poll(DatabaseReceiver.java:111)
	at com.mirth.connect.donkey.server.channel.PollConnector$PollConnectorTask.run(PollConnector.java:141)
	at java.util.TimerThread.mainLoop(Unknown Source)
	at java.util.TimerThread.run(Unknown Source)
ERROR 2019-09-13 08:28:05,267 [pool-2-thread-17603] com.mirth.connect.server.userutil.SerializerFactory: The getHL7Serializer() method is deprecated and will soon be removed. Please use the "Convert HL7 v2.x" templates from the References tab instead. Look at the tooltips to see the available property keys. The new method will strip namespaces by default unless the 'stripNamespaces' property is set to false.
ERROR 2019-09-13 08:28:12,424 [pool-2-thread-17604] com.mirth.connect.server.userutil.SerializerFactory: The getHL7Serializer() method is deprecated and will soon be removed. Please use the "Convert HL7 v2.x" templates from the References tab instead. Look at the tooltips to see the available property keys. The new method will strip namespaces by default unless the 'stripNamespaces' property is set to false.
ERROR 2019-09-13 08:28:14,049 [pool-2-thread-17603] transformer: TypeError: Cannot read property "CD47y" from undefined
ERROR 2019-09-13 08:28:14,143 [pool-2-thread-17603] com.mirth.connect.server.userutil.ResponseFactory: The getSuccessResponse(message) method is deprecated and will soon be removed. Please use getSentResponse(message) instead.
ERROR 2019-09-13 08:28:14,940 [pool-2-thread-17603] com.mirth.connect.server.controllers.DonkeyEngineController: Could not find channel to route to: 1cf4c80f-3cfc-4686-8e93-8b39b2d6b537
com.mirth.connect.donkey.server.channel.ChannelException
	at com.mirth.connect.server.controllers.DonkeyEngineController.dispatchRawMessage(DonkeyEngineController.java:511)
	at com.mirth.connect.server.userutil.VMRouter.routeMessageByChannelId(VMRouter.java:154)
	at com.mirth.connect.server.userutil.VMRouter.routeMessageByChannelId(VMRouter.java:139)
	at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)

```

When de logs arribe in logstash they match in a message and not in different ones for eachline that begins with the "error" and the content below.

Any idea?? Thx

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 16, 2019, 9:21am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/2 "2019-09-16T09:21:40Z")

</div>

@gerard.ramos,

> [@gerard.ramos](#):
>
> ERROR 2019-09-13

Please try the below pattern in `multiline.pattern`

```auto
multiline.pattern: '[A-Z]{5} [0-9]{4}-[0-9]{2}-[0-9]{2}'

```

Thanks.

---

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 16, 2019, 9:41am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/3 "2019-09-16T09:41:37Z")

</div>

Heey Tek, thanks for your reply. I'm still getting the same output.

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 16, 2019, 9:49am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/4 "2019-09-16T09:49:18Z")

</div>

@gerard.ramos, Have you restarted the filebeat service after making the changes? If yes please provide the configuration file of filebeat.

Thanks.

---

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 16, 2019, 9:55am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/5 "2019-09-16T09:55:23Z")

</div>

Yees i restarted the filebeat service, this is my configuration file:

```
filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - C:\Program Files\Mirth Connect\logs\mirth*  
      exclude_files: ['.log$']
      multiline.pattern: '[A-Z]{5} [0-9]{4}[0-9]{2}-[0-9]{2}' 
      multiline.negate: true
      multiline.match: after
    filebeat.config.modules: 
      path: ${path.config}/modules.d/*.yml 
      reload.enabled: false 
    setup.template.settings:
      index.number_of_shards: 1
    setup.kibana:
    output.logstash:
      hosts: ["192.168.1.76:5443"]
    processors:
      - add_host_metadata: ~
      - add_cloud_metadata: ~
```

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 16, 2019, 10:18am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/6 "2019-09-16T10:18:35Z")

</div>

@gerard.ramos,

```auto
filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - C:\Program Files\Mirth Connect\logs\*  
      exclude_files: ['.log$']
      multiline.pattern: '[A-Z]{5} [0-9]{4}[0-9]{2}-[0-9]{2}' 
      multiline.negate: true
      multiline.match: after
    filebeat.config.modules: 
      path: ${path.config}/modules.d/*.yml 
      reload.enabled: false 
    setup.template.settings:
      index.number_of_shards: 1
    setup.kibana:
    output.logstash:
      hosts: ["192.168.1.76:5443"]
    processors:
      - add_host_metadata: ~
      - add_cloud_metadata: ~

```

In you above configuration its seems that your are excluding the files with `.log` extension. What is the extension of your files which you are trying to parse? If they have `.log` extension please remove the line `exclude_files` from your config.

And use my config i have made small change.  
Thanks.

---

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 16, 2019, 10:38am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/7 "2019-09-16T10:38:18Z")

</div>

If I am excluded messages and generating my own test. The problem is not that they do not arrive but that they are not separated into different messages.

---

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 16, 2019, 10:50am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/8 "2019-09-16T10:50:54Z")

</div>

Hello again tek, I have already found the error, in the pattern that you had passed me a - like this:

```
multiline.pattern: '[A-Z]{5} [0-9]{4}[0-9]{2}-[0-9]{2}' 
multiline.pattern: '[A-Z]{5} [0-9]{4}-[0-9]{2}-[0-9]{2}'

```

Thanks for the replys!!

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [September 16, 2019, 10:53am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/9 "2019-09-16T10:53:39Z")

</div>

@gerard.ramos

> [@gerard.ramos](#):
>
> multiline.pattern: '[A-Z]{5} [0-9]{4}-[0-9]{2}-[0-9]{2}'

is it working now? That was typo mistake. Now i have edited my post.

Thanks.

---

<div class="post-metadata">

**Author:** ![gerard.ramos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gerard.ramos/32/46958_2.png) [@gerard.ramos](https://discuss.elastic.co/u/gerard.ramos)\
**Post date:** [September 16, 2019, 11:05am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/10 "2019-09-16T11:05:52Z")

</div>

Yes is working, thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2019, 11:05am UTC](https://discuss.elastic.co/t/multiline-pattern-filebeat/199409/11 "2019-10-14T11:05:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.