How to write more than one multiline patterns for two different type of Logs

faiz · November 12, 2019, 3:40pm

HI,
I need to ship logs of my tomcat alongs with my applications logs.
My application logs and tomcat logs are of different time stamp, There fore I need to write more that one multiline patterns in the filebeat prospector.

How could I ship logs from the same filebeat to logstash.

Any suggestion would be helpful.

Thanks,

Slavik_Fursov · November 12, 2019, 11:20pm

I have never tried,
But you can have multiple patterns on the ingestion pipelines.
For example, existing Apache pipeline does it:

github.com

elastic/beats/blob/master/filebeat/module/apache/access/ingest/default.json#L7-L11


"patterns": [
    "%{IPORHOST:destination.domain} %{IPORHOST:source.ip} - %{DATA:user.name} \\[%{HTTPDATE:apache.access.time}\\] \"(?:%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}|-)?\" %{NUMBER:http.response.status_code:long} (?:%{NUMBER:http.response.body.bytes:long}|-)( \"%{DATA:http.request.referrer}\")?( \"%{DATA:user_agent.original}\")?",
    "%{IPORHOST:source.address} - %{DATA:user.name} \\[%{HTTPDATE:apache.access.time}\\] \"(?:%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}|-)?\" %{NUMBER:http.response.status_code:long} (?:%{NUMBER:http.response.body.bytes:long}|-)( \"%{DATA:http.request.referrer}\")?( \"%{DATA:user_agent.original}\")?",
    "%{IPORHOST:source.address} - %{DATA:user.name} \\[%{HTTPDATE:apache.access.time}\\] \"-\" %{NUMBER:http.response.status_code:long} -",
    "\\[%{HTTPDATE:apache.access.time}\\] %{IPORHOST:source.address} %{DATA:apache.access.ssl.protocol} %{DATA:apache.access.ssl.cipher} \"%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}\" (-|%{NUMBER:http.response.body.bytes:long})"

But multiline setting have to be defined on the filebeat side. For that, I guess you will need to find settings, which works for both App and Tomcat

faiz · November 18, 2019, 5:15pm

Hi thanks for the response, but I am comfortable on the ingestion side, i am facing blocker on the filebeat side, if i could add mulitple multiline pattern. ;(

Any example of the same would be much appreciable.

system · December 16, 2019, 5:15pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.