Logstash not able to get nested json in desired output using ruby

Elastic Stack Logstash
1

Below is my sample output log


{
    "traceId": "o0uyveRU/8BkjL+lbpDlnQ==",
    "spanId": "73rmqIRmo34=",
    "tags": [
        {
            "key": "otel.library.name",
            "vStr": "com.mdi.core.workmgmt.TypedConsumer"
        },
        {
            "key": "ta.work.receivedTime",
            "vType": "INT64",
            "vInt64": "1721628429423"
        },
        {
            "key": "internal.span.format",
            "vStr": "otlp"
        }
    ],
    "process": {
        "serviceName": "TRADE_SINGLETON",
        "tags": [
            {
                "key": "deployment.environment",
                "vStr": "PSR"
            },
            {
                "key": "telemetry.sdk.version",
                "vStr": "1.34.0"
            }
        ]
    }
}

Need output as below:

{
    "traceId": "o0uyveRU/8BkjL+lbpDlnQ==",
    "spanId": "73rmqIRmo34=",
	"tags.otel.library.name":"com.mdi.core.workmgmt.TypedConsumer",
	"tags.ta.work.receivedTime":"1721628429423",
	"tags.internal.span.format":"otlp",
	"process.serviceName":"TRADE_SINGLETON",
	"process.tags.deployment.environment":"PSR",
	"process.tags.telemetry.sdk.version":"1.34.0"
	
}

Below logstash config i have tried to get first [tags] section fields, but no fields getting added:

filter 
{ 
	json 
	{
		source => "message"
		target => "tmessage"
	}
	
	ruby {
		code => '
			event.get("[tmessage][tags]").each { |a|
			name = a["Name"]
			value = a["Value"]
			event.set( "[tmessage][#{name}]", value)
			}
		'
		}
}

Kindly let me know the solution?

2

Your tags do not have a "Name" and "Value", they have a "key" and another hash entry, the name of which depends on "vType", which may not be present. So you could change those two lines to

                name = a["key"]
                value = a["vStr"]

but that will not work for "vType": "INT64" (or any type other than string). You may also need to add type conversion to make ints ints, booleans booleans, etc. That said, the two line change above will get you

       "otel.library.name" => "com.mdi.core.workmgmt.TypedConsumer",
    "ta.work.receivedTime" => nil,
    "internal.span.format" => "otlp"

which is a step towards what you want.

3

Thanks for the response, after changing the values, facing below ruby exception. pls let me know where would be the issue?

Updated Filter:

filter 
{ 
	json 
	{
		source => "message"
		target => "tmessage"
	}
	
	ruby {
		code => '
			event.get("[message][tags]").each { |a|
			name = a["key"]
			value = a["vStr"]
			event.set( "[message][tagsFattened]#{name}", value)
			}
		'
		}
}

Below error


[2024-07-25T01:17:48,516][ERROR][logstash.filters.ruby    ][main][e6fe0e97cf931545d8bb7438d328202eacfe609144d4e9a0f02c03d25d306cfb] Ruby exception occurred: undefined method `each' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):3:in `block in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:158:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:176:in `block in multi_filter'", "org/jruby/RubyArray.java:1989:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:173:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:133:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in start_workers'"]}
4

Able to get it work, issue with event.set statement

event.set( name, value)

Thanks Badger