Logstash not able to connect to Elasticsearch deployed through Kubernetes pod

[Shubham_Dhote]

Hi, I have deployed Logstash and elasticsearch pod on EKS cluster. When I am checking the logs for logstash pod it is showing unreachable elasticserach server. Though my elasticsearch is up and running. Please find below yaml files and log error.

configMap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: "logstash-configmap-development"
  namespace: "development"
  labels:
    app: "logstash-development"
data:
  logstash.conf: |-
    input {
         http {
         
      }
    }
    
    filter {
        json {
            source => "message"
        }
    }
    output {
      elasticsearch {
            hosts => ["https://my-server.com/elasticsearch-development/"]
            index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
        }
        
        stdout {
            codec => rubydebug
      }
    }

deployment.yaml

---
  apiVersion: "apps/v1"
  kind: "Deployment"
  metadata: 
    name: "logstash-development"
    namespace: "development"
  spec: 
    selector: 
      matchLabels: 
        app: "logstash-development"
    replicas: 1
    strategy: 
      type: "RollingUpdate"
      rollingUpdate: 
        maxSurge: 1
        maxUnavailable: 1
    minReadySeconds: 5
    template: 
      metadata: 
        labels: 
          app: "logstash-development"
      spec: 
        containers: 
          - 
            name: "logstash-development"
            image: "logstash:7.10.2"
            imagePullPolicy: "Always"
            env: 
              - 
                name: "XPACK_MONITORING_ELASTICSEARCH_HOSTS"
                value: "https://my-server.com/elasticsearch-development/"
                
              -
                name: "XPACK_MONITORING_ELASTICSEARCH_URL"
                value: "https://my-server.com/elasticsearch-development/"
                
              -
              
                name: "SERVER_BASEPATH"
                value: "logstash-development"
                
                
                
            securityContext:
                privileged: true
                
            ports: 
              - 
                containerPort: 8080
                protocol: TCP
                
            volumeMounts:
                -
                  name: "logstash-conf-volume"
                  mountPath: "/usr/share/logstash/pipeline/"
                  
        volumes:
            -
              name: "logstash-conf-volume"
              configMap:
                 name: "logstash-configmap-development"
                 items:
                    - key: "logstash.conf"
                      path: "logstash.conf"
        imagePullSecrets: 
          - 
            name: "logstash"

Logstash pod logs error

[2021-06-09T08:22:38,708][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://my-server.com/elasticsearch-development/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://my-server.com/elasticsearch-development/][Manticore::ConnectTimeout] connect timed out"}

Note:- Elasticsearch is up and running. And when I hit the logstash url it is giving as status ok.

Please help for the issue.