How to create new fields and documents based on variable length log?

Elastic Stack Logstash
1

I have a log message with a format like:
status : name message ; status: name message ; status: name message ; status: name message.....

I need to split based on the semicolon, and create separate Elasticsearch documents that have fields for each of status, name and message. For example, if my log message was like this:
WARNING: A B ; CRITICAL: C D ; OK E F ; CRITICAL G H
Then I would need four different documents. The first document would have three fields, where status is WARNING, name is A and message is B. The second document would have status CRITICAL, name C and message D, and so on.

How do I do this?

2

I would start by using mutate+split to separate the string into an array of four (or fewer?) strings. Then use a split filter to make each string a separate document. Then use dissect or grok to pull out the three fields.

3

That worked! Thank you :slight_smile:

For future reference, in case someone else has a similar problem, here is a sample code:

filter {
    mutate {
        split => { "message" => ";" }
    }
    split {
        field => "message"
    }
    grok {
        match => { "message" => "%{WORD:level}: %{WORD:name} %{GREEDYDATA:msg}" }
        remove_field => ["message"]
    }
}