Logstash SSL enable to connect elastic cloud instance

[Rahul12 (Rahul)]

Hi,
I am using elastic cloud version 7.5.1. I have enable SAML to my elastic cloud instance.
Before enabling SAML my logstash which is at my local able to connect with elastic cloud instance.

But after enabling SAML settings I was getting certificate error. I have generated certificate(.PEM file) using openssl command and added the path of my certificate to Logstash config file in output section.

elasticsearch {
          hosts => '${ELASTIC_URL}'
          user => "${ELASTIC_USERNAME}"
          password => "${ELASTIC_PASSWORD}"
          ssl_certificate_verification => true
          cacert => "/etc/logstash/ssl/elastic-ece-ca-cert.pem"
          index => "%{[@metadata][beat]}-%{[@metadata][version]}"
          }

But after doing this setting certificate error resolved but not we are getting error

"Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')"
at [Source: (byte)"^M

^M ^M SSSL Inspection^MSL Inspection^M ^M ^M^M h2{font-size:16px;font-weight:bold;color:#196390;}^M #content {^M border:3px solid#aaa;^M background-color:#fff;^M margin:1.2em;^M padding:1.2em;^M font-family:Tahoma,Helvetica,Arial,sans-serif;^M font-size:1em;^M }^M h1 {^M font-size:1.2em;^M font-weight:bold;^M color:#196390;^M }^M b {^M.

Can you please help me with this issue?

[Rahul12 (Rahul)]

When I am doing curl from logstash server using the certificate file. I am getting below output.

SSL Inspection h2{font-size:16px;font-weight:bold;color:#196390;} #content { border:3px solid#aaa; background-color:#fff; margin:1.2em; padding:1.2em; font-family:Tahoma,Helvetica,Arial,sans-serif; font-size:1em; } h1 { font-size:1.2em; font-weight:bold; color:#196390; } b { font-weight:normal; color:#196390; } form td, form input { font-size: 1em; font-weight: bold; } #formtable { height: 100%; width: 100%; } #formtd { vertical-align: middle; } #formdiv { margin-left: auto; margin-right: auto; }

SSL Inspection

If you proceed with this session then, in accordance with company security policy, SSL encrypted traffic that you initiate will be temporarily decrypted so that it can be inspected for viruses, spyware, and malware.

After inspection, the traffic will be re-encrypted and sent to its destination.

IP:

Category: any

Would you like to proceed with this session?

 

[Badger]

That looks like you are connecting to a corporate proxy that speaks HTTP / HTML and not an elasticsearch instance that responds with HTTP / JSON.

[ikakavas (Ioannis Kakavas)]

@Rahul12

I am using elastic cloud version 7.5.1. I have enable SAML to my elastic cloud instance.
Before enabling SAML my logstash which is at my local able to connect with elastic cloud instance.

But after enabling SAML settings I was getting certificate error. I have generated certificate(.PEM file) using openssl command and added the path of my certificate to Logstash config file in output section.

Some clarifications:

• The change that affects you is not specifically that you enabled SAML, but that as part of the process of configuring SAML, you are required to enable TLS For the HTTP layer of elasticsearch. Now Logstash talks to Elasticsearch over the http layer so it makes sense that you need to update the elasticsearch output plugin configuration to be able to talk to elasticsearch over HTTPS.

• As such you can't just generate a certificate and use that as the cacert option. You need to use the CA certificate that has signed the certificate that you use in your elasticsearch configuration. It's not clear how you got elastic-ece-ca-cert.pem , and this is why I'm mentioning this here.

As with regards to your actual problem, @Badger is spot on, there is a proxy doing SSL inspection between your Logstash and Elasticsearch instance. You would need to bypass that or add a whitelist for it so that it doesn't attempt to prompt. Eitherway, this is something you need to talk to your corporate IT about, it will be very hard for anyone in these forums to assist you with.