Can Output elasticsearch plugin credential use @metadata?

Elastic Stack Logstash
1

Dear All,

I am trying to use @metadata that has defined in filter for the user and password fields in output, is that possible or is there another way? Please see below configuration:

output {
elasticsearch {
hosts=> ["https://xxxxxxxxxxxxxxxxx"]
ssl => true
ssl_certificate_verification => true
cacert => "/etc/logstash/xxxxxxxxxxxxxxxxx"
user=> "%{[@metadata][logstash_username]}"
password=> "%{[@metadata][logstash_password]}"
index => "xxxxxxxxxxxxxxxxx}"
}
jdbc {
connection_string => "%{[@metadata][connString]}"
statement => xxxxxxxxxxxxxxxxx
}

jdbc {
connection_string => "%{[@metadata][connString]}"
statement => xxxxxxxxxxxxxxxxx
}
}

2

The Elasticsearch output plugin sends bulk requests, so setting a parameter that is applied per request (multiple documents) based on a field that can vary across events in that batch does not IMHO make sense. What is it you are trying to achieve?

3

Also, my reading of the code is that the basic_auth configuration is done at startup time, so there is no event from which fields or metadata can be referenced.

4

That is probably true as well.

5

Hello Badgar and Dahlqvist,

I am using ruby to call a servlet in filter and returned the username and password of ElasticSearch. I am trying to store the values as metadata and use those in the output. Could you suggest another way to achieve this?

Thank you.

6

It is possible that an environment reference rather than a sprintf reference would work. I have not tested it, but perhaps you can use logstash or curl or something else to do the servlet call and then pass the variables to elastic as via the environment.

7

Do you mean pass the variable to Logstash instead of elastic?
Also, can I reference the environment variable in the output elasticsearch plugin?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.