Two field visualization in one table column, each in it's own row

Elastic Stack Kibana
1

Hello, trying to figure out solution for the following case.

I have log line which is structured as:

Action=MarketingAction,Max Duration=400ms,Max Calls=3

Any ideas how to parse message using logstash filter in order to get data that would allow me to create table visualization in Kibana as follows?

Action Name     | Property Name | Property Value
___________________________________________________
MarketingAction | Max Duration  | 400ms
MarketingAction | Max Calls     | 3
2

Hello @s.buksa

Welcome back!!

If you are receiving this log line still we can create this table using ES|QL.

Example below :

FROM marketing
| EVAL Action_Name = action
| EVAL Property_Names = mv_append("Max Duration", "Max Calls")
| EVAL Property_Values = mv_append(TO_STRING(max_duration), TO_STRING(max_calls))
| EVAL paired = mv_zip(TO_STRING(Property_Names), TO_STRING(Property_Values))
| MV_EXPAND paired
| EVAL Property_Name = MV_FIRST(SPLIT(paired, ","))
| EVAL Property_Value = MV_LAST(SPLIT(paired, ","))
| KEEP Action_Name, Property_Name, Property_Value

image
image1548×282 19.5 KB

If this needs to be done at logstash end it needs to be reviewed further. Also you can tag logstash to the question.

Thanks!!

3

Thank you for the suggestion.

Please, I'm interested in an option on the Logstash end, Elastic Stack > Logstash logstash

1 Like
4

Still trying to figure out and looking for the solution at Logstash end.

5

I'm not too sure what you mean by "an option on the Logstash end"?

Do you just mean you need a logstash filter to split the log

Action=MarketingAction,Max Duration=400ms,Max Calls=3

into 3 fields, probably named Action, Max Duration, and Max Calls, something like?

filter {
  grok {
    match => {
      "message" => "Action=%{WORD:action},Max Duration=%{NUMBER:max_duration}ms,Max Calls=%{NUMBER:max_calls}"
    }
  }
}

Once fed to Elasticsearch then you can create Kibana tables like the one you shared based on this index/indexing.

6

Thanks,

This is what I have in my Logstash filter at the moment. What I'm trying to achieve is "Max Duration" and "Max Calls" reflection as Property Name in one table column, and their corresponding value reflection in the next column.

7

Because I'm curious, whats the advantage of doing this in logstash rather than the method @Tortoise gave?

But seems you want to parse your single input line effectively into 2 outputs:

{"action":"MarketingAction","Property Name":"Max Duration","Property Value":"400ms"}
{"action":"MarketingAction","Property Name":"Max Calls","Property Value":"3"}

?

8

This looks like something I have imagined. :slight_smile: If I have a field named "Property Name", I would be able to select it and show all property names such as "Max Duration" and "Max Calls". Same would be for "Property Value" field.

9

If you like 2 events, you have to split the message. You can also convert to numeric values with the unit

input {
  generator {
       message => 'Action=MarketingAction,Max Duration=400ms,Max Calls=3'
       count => 1
  }
}
filter{
  grok {
    match => {
      "message" => "Action=%{WORD:action},%{DATA:PropertyName}=%{DATA:max_duration},%{DATA:[@metadata][PropertyValue2]}=%{NUMBER:max_calls}"
     #"message" => "Action=%{WORD:action},%{DATA:PropertyName}=%{NUMBER:max_duration}(?<unit>[A-Za-z]+),%{DATA:[@metadata][PropertyValue2]}=%{NUMBER:max_calls}" 
    }
  }
    # mutate{ convert => { # if you like numeric values
			# "max_duration"=> "integer" 
			# "max_calls"=> "integer" 
	        # } 
	# }
    split {
     field  => "message"
     terminator => "Max Calls="
     target => "[@metadata][PropertyValue]"
    }
  
	 if [@metadata][PropertyValue] =~ /^Action/ {
		 mutate{ remove_field => [ "max_calls"] } 
	 }
	 else {
		 mutate{ 
		 remove_field => [ "unit","max_duration", "PropertyName"] 
		 } 
		 mutate{ 
		 rename => { "max_calls" => "Property Value" }
		 rename => { "[@metadata][PropertyValue2]" => "Property Name" }}
	 }
 
    mutate{  
	rename => { "action" => "Action Name"
	"max_duration" => "Max Duration"
	"PropertyName" => "Property Name" 
	}
	remove_field => [ "event",  "host", "@timestamp", "@version", "message"] 
	}

}
output {
    stdout { codec => rubydebug{ metadata => false}} 
}

Output:

{
    "Property Name" => "Max Duration",
      "Action Name" => "MarketingAction",
     "Max Duration" => "400ms"
}
{
     "Property Name" => "Max Calls",
       "Action Name" => "MarketingAction",
    "Property Value" => "3"
}

PS. I hope nobody will arrest me for this kind of code

10

Don't shoot me either.

input {
  generator {
       message => "Action=MarketingAction,Max Duration=400ms,Max Calls=3"
       count => 1
       ecs_compatibility => "disabled"
  }
}

filter {
  grok {
    match => {
      "message" => "Action=%{WORD:action},Max Duration=%{WORD:max_duration},Max Calls=%{WORD:max_calls}"
    }
  }

  clone {
    clones => ["duration_event"]
    ecs_compatibility => "disabled"
  }

  if [type] == "duration_event" {
    mutate {
      add_field => {
        "[Property Name]" => "Max Duration"
        "[Property Value]" => "%{max_duration}"
      }
      remove_field => [ "event" , "max_calls" , "max_duration" , "host" , "type" ]
    }
  } else {
    mutate {
      add_field => {
        "[Property Name]" => "Max Calls"
        "[Property Value]" => "%{max_calls}"
      }
      remove_field => [ "event" , "max_calls" , "max_duration", "host" ]
    }
  }

}

output {
  file {
    path => "/tmp/output"
  }
}

Input was as shown, output is (piped to jq)

{
  "@timestamp": "2025-06-10T14:10:31.882527473Z",
  "@version": "1",
  "Property Name": "Max Duration",
  "Property Value": "400ms",
  "action": "MarketingAction",
  "message": "Action=MarketingAction,Max Duration=400ms,Max Calls=3",
  "sequence": 0
}
{
  "@timestamp": "2025-06-10T14:10:31.882527473Z",
  "@version": "1",
  "Property Name": "Max Calls",
  "Property Value": "3",
  "action": "MarketingAction",
  "message": "Action=MarketingAction,Max Duration=400ms,Max Calls=3",
  "sequence": 0
}

btw, the filter there was mostly generated by ChatGPT, I just tweaked its idea a little bit.

2 Likes