I'll do a high level overview. We are using IBM's APIC v10. It uses Kubernetes which I'm all but a few days new to it. They seemt o have used logstash on top of that for their propriety "analytics server".
I feel like IBM is getting stumped here surprisingly so worse case we'll just fall back on them but I would love to figure this out on my own so I'm learning a lot already.
Anyhow we are offloading to SPLUNK. We are using the HTTP output plugin
SPLUNK is on HTTPS but has no cert requirements...
We were getting this error:
:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target", :class=>"Manticore::ClientProtocolException",
IBM surprisingly stated it's coming from SPLUNK to which we had to go through that pain to prove not. They haven't come back yet since...
In looking over things, I setup a "secret" and loaded to kubernetes
Referenced that secret in my config which still had the same error. I added the truststore field:
url => "https://***************"
http_method => "post"
codec => "json"
content_type => "application/json"
id => "offload_http"
headers => ["Authorization", "Splunk ***************"]
truststore => "/etc/velox/analytics-ingestion.jks"
which now gives
:message=>"undefined method `value' for nil:NilClass", :class=>"NoMethodError",
Shooting in the dark here but I'm guessing I can't use the truststore like that. Basically playing around with this. Honestly I thought it would have just worked but I guess it's not using some default TLS here.