Using multiline codec for python stacktrace

Elastic Stack Logstash
1

Basically i have three types of lines in my log file.

  1. Syntax error
  2. Error timestamp internal server error url blah blah
  3. traceback

For the traceback part i am just checking if line begins with a space and appending it into previous, thereby i am getting all of the traceback in a single event.
But in real my event should consist all 3 lines above mention in a single event. With my current config i am getting all three parts in different events with "tags" = [multiline, _grokparsefailure] in traceback event.
How can i club all 3 into one event and remove that multiline tag too.

Config file:

input {
        file {
                path => "/home/rajdeep/Desktop/e9"
                codec => multiline {
                         pattern => "^\s"
                         what => "previous"
                 }
                 start_position => "beginning"
        }
}

filter {
        grok {
                match => [

                           "message", "SyntaxError: %{DATA:class}",
                           "message", "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url}",
                           "(?m)message", "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url} %{GREEDYDATA:traceback}"
                ]
        }

        date {
                match => ["logdate", "YYYY-MM-dd HH:mm:ss,SSS"]
        }
}
2

You can supress the tag by adding

multiline_tag => ""

to the codec. There is no way to say how else to change the codec without seeing the lines you wish to combine.

3

Hi @Badger Thanks for taking time out. I want to club following lines into 1 event. Whats happening with the above code is all three parts are coming as separate events. I even used aggregation, i can provide code for aggregation if there is not other way out.

SyntaxError: invalid syntax
ERROR 2019-08-12 13:56:20,544 Internal Server Error: /api/v1/userapp/notification/seen/
Traceback (most recent call last):
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 119, in get_response
    resolver_match = resolver.resolve(request.path_info)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 368, in resolve
    sub_match = pattern.resolve(new_path)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 368, in resolve
    sub_match = pattern.resolve(new_path)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 368, in resolve
    sub_match = pattern.resolve(new_path)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 240, in resolve
    return ResolverMatch(self.callback, args, kwargs, self.name)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 247, in callback
    self._callback = get_callable(self._callback_str)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/utils/lru_cache.py", line 101, in wrapper
    result = user_function(*args, **kwds)
  File "/home/ubuntu/virtualenvs/parkwheels-core-api/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 106, in get_callable
    mod = import_module(mod_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "./parkwheels/userapp/views/notification_views.py", line 132
    seen=True, seen_time=parkwheels_utils.get_current_datetime() created=created, updated=created)
4

To do that using a multiline codec you need a pattern that matches the first line.

codec => multiline { pattern => '^SyntaxError:' negate => true what => "previous" auto_flush_interval => 1 } }

If you need to match multiple errors then maybe '^[a-zA-Z]+Error:'.

5

Hi thank you for that. But how can i have separate fields for all three merged events? Like first line goes to error_type 2nd into message and third into traceback.?

6

You can use grok to split the lines up. You can match a line using "one or more not-newline followed by newline"

    grok { match => { "message" => "\A(?<errorType>[^
]+)
(?<errorMessage>[^
]+)
%{GREEDYDATA:stacktrace}" } }
7

sorry but i am not so familiar with regex patterns. can i put logstash defined patterns in [^] instead of whole regex pattern for the the log line?

Also can i just use below mentioned grok block and changes message fields to message, message1, message2 respectively and use them to pass through [^] like [{"%message1"}]. is it logically correct?

8

Yes, you could use individual patterns to match each part of the event.

9

But this pattern is not working.
I have added the codec in input file (with changes).

 filter {
                grok {
                        match => [

                               "message", "SyntaxError: %{DATA:class}",
                               "message1", "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url}",
                               "(?m)message2", "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url} %{GREEDYDATA:stacktrace}"
                    ]
            }


            grok { match => { "message" => "\A(?<errorType>["%{message}"]+) (?<errorMessage>["%{message1}"]+) %{GREEDYDATA:stacktrace}" } }


            date {
                    match => ["logdate", "YYYY-MM-dd HH:mm:ss,SSS"]
            }
    }
10

If you want to match multiple patterns then use

    grok {
        break_on_match => false
        match => {
            "message" => [
                "SyntaxError: %{DATA:class}",
                "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url}",
                "%{LOGLEVEL:severity} %{TIMESTAMP_ISO8601:logdate} Internal Server Error: %{URIPATHPARAM:url} %{GREEDYDATA:stacktrace}"
            ]
        }
    }

The first one does not match for reasons I am unclear on. The third does not match because the %{URIPATHPARAM:url} is not followed by a space in your event.

11

Hi @Badger first one is matching now. Apparently a space was missing in the multiline codec where we define pattern to look for '^SyntaxError:' as you can see after colon we need a space.
Can you tell me how can i have different fields for for all the 3 matches in a single event without using this method.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.