Dissect field with unknown length?

Elastic Stack Logstash
1

Hi,

Based on other posts I've made, I've learned that I should be able dissect a field (in this case, it's eventData.ObjectName into sub-fields, however I'm having an issue where some messages don't have the full length of my dissect, and therefore throw a _dissectfailure

My dissect is as follows:

if [eventData.ObjectName] =~ "folder1\/folder2" {
            dissect { mapping => { "eventData.ObjectName" => "%{share_name}/%{folder1}/%{folder2}/%{customername}/%{objectname}" } }
        }

Sample messages in field:

(share);/folder1/folder2/customer1
(share);/folder1/folder2/customer1/file1.txt
(share);/folder1/folder2/customer1/file2.txt
(share);/folder1/folder2/customer2
(share);/folder1/folder2/customer2/file1.txt

What I'm seeing is that the messages in the example: 2, 3, and 5 will parse fine, but 1 and 4 will throw the failure. Note, this won't break, but I'm dealing with ~100,000 messages per minute, so I need this to be efficient, accurate, and now throw failures. Thanks in advance.

Hope you don't mind if I mention you @Badger since you're extremely helpful. I owe you a beer (or something else if you don't drink).

2

As the documentation notes, you can make application of the dissect conditional upon the field matching the pattern. For example

    if [message] =~ /\/[^\/]+\/[^\/]+\/[^\/]+\// {
        dissect { mapping => { "message" => "%{share_name}/%{folder1}/%{folder2}/%{customername}/%{objectname}" } }
    } else {
        dissect { mapping => { "message" => "%{share_name}/%{folder1}/%{folder2}/%{customername}" } }
    }

Alternatively (and this would be my preference), dissect off the fixed prefix and conditionally do a second dissect.

    dissect { mapping => { "message" => "%{share_name}/%{folder1}/%{folder2}/%{customername}" } }
    if [customername] =~ /\// {
        dissect { mapping => { "customername" => "%{customername}/%{objectname}" } }
    }
1 Like
3

Thank you, I chose this method, with a few modifications:

        if [eventData.ObjectName] =~ "folder1" {
            dissect { mapping => { "eventData.ObjectName" => "%{eventData.share.name}/%{eventData.ObjectName.folder1}/%{eventData.ObjectName.folder2}/%{eventData.ObjectName.customername}" } }
            if [customer.name] =~ /\// {
                 dissect { mapping => { "eventData.ObjectName.customername" => "%{eventData.ObjectName.customername}/%{eventData.ObjectName.filepath}" } }
            }
        }
        else if [eventData.ObjectName] {
            dissect { mapping => { "eventData.ObjectName" => "%{eventData.share.name}/%{eventData.filepath}" } }
        }
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.