# Logstash - filebeat

**URL:** <https://discuss.elastic.co/t/logstash-filebeat/242216>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 22, 2020, 2:41pm UTC](https://discuss.elastic.co/t/logstash-filebeat/242216 "2020-07-22T14:41:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arezki76](https://avatars.discourse-cdn.com/v4/letter/a/9fc29f/32.png) [@Arezki76](https://discuss.elastic.co/u/Arezki76)\
**Post date:** [July 22, 2020, 2:41pm UTC](https://discuss.elastic.co/t/logstash-filebeat/242216/1 "2020-07-22T14:41:34Z")

</div>

Hi,

I'm new in ELK and I'm trying to parse log file as follow:  
2019 Nov 12 14:58:01:211 GMT -0500 BW.ypg\_bis\_ful\_campaignservice\_bw-ypg-bis-ful-campaignservice-bw Info [BW-User] UPSTREAM\_REQUEST Job-1000 [ESBCore/Log/System/Log.process/Log Message]: \<?xml version="1.0" encoding="UTF-8"?\>  
\<ns0:LogEntry xmlns:ns0="urn:com.ypg.core.schemas.logging.v2"\>  
\<ns1:ContextHeader xmlns:ns1="urn:com.ypg.common.types.v1"\>  
ns1:OriginatorIdWebServices\</ns1:OriginatorId\>  
ns1:SystemApplicationNameypg.bis.ful.campaignservice.v1\</ns1:SystemApplicationName\>  
ns1:SystemServiceNameCampaignService\</ns1:SystemServiceName\>  
ns1:SystemOperationNameListCampaignProducts\</ns1:SystemOperationName\>  
ns1:SystemTrackingId20d3dd47-fca7-4885-926a-e562cbb9fe42\</ns1:SystemTrackingId\>  
ns1:SystemTimeCreated2019-11-12T14:58:01.197-05:00\</ns1:SystemTimeCreated\>  
ns1:TibcoProcessId1000\</ns1:TibcoProcessId\>  
\</ns1:ContextHeader\>  
ns0:LogEntryRequest  
ns0:LogLevelINFO\</ns0:LogLevel\>  
ns0:MsgCodeUPSTREAM\_REQUEST\</ns0:MsgCode\>  
ns0:MessageRequest received\</ns0:Message\>  
ns0:Payload\<?xml version="1.0" encoding="UTF-8"?\>  
\<Payload\>\<urn:ListCampaignProductsRequest xmlns:urn="urn:com.ypg.bis.ful.campaignservice.v1" xmlns:soapenv="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" xmlns:urn1="urn:com.ypg.common.service.types.v1" xmlns:urn2="urn:com.ypg.common.types.v1" xmlns:SOAP-ENV="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" timezone="+00:00" validateOnly="false"\>\<urn1:ContextHeader\>\<urn2:OriginatorId\>WebServices\</urn2:OriginatorId\>\</urn1:ContextHeader\>\<urn:MerchantId\>623778\</urn:MerchantId\>\</urn:ListCampaignProductsRequest\>\</Payload\>\</ns0:Payload\>  
ns0:Timestamp2019-11-12T14:58:01.206-05:00\</ns0:Timestamp\>  
\</ns0:LogEntryRequest\>  
\</ns0:LogEntry\>

I want that the below part will be tagged as the message  
UPSTREAM\_REQUEST Job-1000 [ESBCore/Log/System/Log.process/Log Message]: \<?xml version="1.0" encoding="UTF-8"?\>  
\<ns0:LogEntry xmlns:ns0="urn:com.ypg.core.schemas.logging.v2"\>  
\<ns1:ContextHeader xmlns:ns1="urn:com.ypg.common.types.v1"\>  
ns1:OriginatorIdWebServices\</ns1:OriginatorId\>  
ns1:SystemApplicationNameypg.bis.ful.campaignservice.v1\</ns1:SystemApplicationName\>  
ns1:SystemServiceNameCampaignService\</ns1:SystemServiceName\>  
ns1:SystemOperationNameListCampaignProducts\</ns1:SystemOperationName\>  
ns1:SystemTrackingId20d3dd47-fca7-4885-926a-e562cbb9fe42\</ns1:SystemTrackingId\>  
ns1:SystemTimeCreated2019-11-12T14:58:01.197-05:00\</ns1:SystemTimeCreated\>  
ns1:TibcoProcessId1000\</ns1:TibcoProcessId\>  
\</ns1:ContextHeader\>  
ns0:LogEntryRequest  
ns0:LogLevelINFO\</ns0:LogLevel\>  
ns0:MsgCodeUPSTREAM\_REQUEST\</ns0:MsgCode\>  
ns0:MessageRequest received\</ns0:Message\>  
ns0:Payload\<?xml version="1.0" encoding="UTF-8"?\>  
\<Payload\>\<urn:ListCampaignProductsRequest xmlns:urn="urn:com.ypg.bis.ful.campaignservice.v1" xmlns:soapenv="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" xmlns:urn1="urn:com.ypg.common.service.types.v1" xmlns:urn2="urn:com.ypg.common.types.v1" xmlns:SOAP-ENV="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" timezone="+00:00" validateOnly="false"\>\<urn1:ContextHeader\>\<urn2:OriginatorId\>WebServices\</urn2:OriginatorId\>\</urn1:ContextHeader\>\<urn:MerchantId\>623778\</urn:MerchantId\>\</urn:ListCampaignProductsRequest\>\</Payload\>\</ns0:Payload\>  
ns0:Timestamp2019-11-12T14:58:01.206-05:00\</ns0:Timestamp\>  
\</ns0:LogEntryRequest\>  
\</ns0:LogEntry\>

I used the follwoing regex in my logstash config file, but seems not working:  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:timestamp} %{DATA:TZ} %{DATA:GMT} %{GREEDYDATA:ApplicationName} %{LOGLEVEL:LOG-Level} [%{DATA:Logger}] (?(.|\r|\n)\*)"}

Can someone help please ?

Thanks

---

<div class="post-metadata">

**Author:** ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)\
**Post date:** [July 29, 2020, 5:27pm UTC](https://discuss.elastic.co/t/logstash-filebeat/242216/2 "2020-07-29T17:27:18Z")

</div>

Before I put my grok patterns in logstash I test them in [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/).. you should be able to trouble shoot whats wrong with you grok pattern there. It looks like your pattern works up to here %{SYSLOGTIMESTAMP:timestamp}%{DATA:TZ} %{DATA:GMT} %{GREEDYDATA:ApplicationName} %{LOGLEVEL:LOG-Level}.. but the rest of it causes the error

---

<div class="post-metadata">

**Author:** ![Arezki76](https://avatars.discourse-cdn.com/v4/letter/a/9fc29f/32.png) [@Arezki76](https://discuss.elastic.co/u/Arezki76)\
**Post date:** [July 29, 2020, 5:32pm UTC](https://discuss.elastic.co/t/logstash-filebeat/242216/3 "2020-07-29T17:32:38Z")

</div>

Thank you @Archie_Crawford,

Yes I tested my pattern in the grokdebug but I don't know why the multiline is bypassed.

Regards.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 26, 2020, 7:32pm UTC](https://discuss.elastic.co/t/logstash-filebeat/242216/4 "2020-08-26T19:32:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.