Kafka output plugin is not picking up value from @metadata field

Elastic Stack Logstash
1

Hello Team,

I have http endpoint as input. Based on the value received in header tag for application , I need to generate the jass configuration at run time for kafka output topic.

input {
http {
host => "127.0.0.1" # default: 0.0.0.0
port => "9600" # default: 8080
}
}
filter {
if [headers][app] == "cop" {
mutate { add_field => { "[@metadata][topic]" => "data_cop_test_topic" } }
mutate { add_field => { "@[metadata][jass]" => "C:\kafka_kerberos\jass1.conf" } }
mutate { add_field => { "@[metadata][kerberos_config]" => "C:\kafka_kerberos\krb5.conf" } }
}
}

output {

kafka {
bootstrap_servers => "xxx"
topic_id => "%{[@metadata][topic]}"
jaas_path => "%{[@metadata][jass]}"
kerberos_config =>%{[@metadata][kerberos_config]}"
sasl_kerberos_service_name => "kafka"
security_protocol => "SASL_PLAINTEXT"
sasl_mechanism => "GSSAPI"
}

}

This picked up topic name correctly , but throws below error for jass path and kerberos config

Invalid setting for kafka output plugin:

output {
kafka {
# This setting must be a path
# File does not exist or cannot be opened %{[@metadata][kerberos_config]}
kerberos_config => "%{[@metadata][kerberos_config]}"
...
}
}
[2019-08-12T15:32:59,506][ERROR][logstash.outputs.kafka ] Invalid setting for kafka output plugin:

output {
kafka {
# This setting must be a path
# File does not exist or cannot be opened %{[@metadata][jass]}
jaas_path => "%{[@metadata][jass]}"
...
}

Can someone please suggest how can we provide jass config at runtime? I am using logstash 7.1

2

jaas_path and kerberos_config are passed straight through to the environment, so no sprintf substitutions are done. It cannot be a runtime variable.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.