Logstash into another server

[Allie_Yang (Allie Yang Yang)]

Just want to confirm I am doing the right thing: if i want to logstash to another ES server at https://example.com, should i just do below in logstash config output section? I do need the port there, right?

output {

        elasticsearch {
             #hosts => [ "localhost:9200" ]
             hosts => ["https://example.com:9200"]
             user => elastic
             password => changeme
             index => "generic_logs"
             document_type => "generic"
        }
      
        stdout { codec => rubydebug }
    }

[magnusbaeck (Magnus Bäck)]

That looks fine. The port should default to 9200.

[Allie_Yang (Allie Yang Yang)]

What does below error possibly mean? The server is hosted on S3, could it be I need authentication?

[WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTPS:0x34b76f0f URL:https://~hidden~:~hidden~@search-adobestock-logs-dd67iqphmey3gudexe7azmrnwu.us-east-1.es.amazonaws.com:9200>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://~hidden~:~hidden~@search-adobestock-logs-dd67iqphmey3gudexe7azmrnwu.us-east-1.es.amazonaws.com:9200][Manticore::ConnectTimeout] connect timed out"}

[magnusbaeck (Magnus Bäck)]

Logstash appears to be having connection problems. Perhaps you need to access the server via a proxy?

AWS, not S3. S3 is the storage service and you can't run ES there.

[Allie_Yang (Allie Yang Yang)]

Yea sorry I mean AWS. Could you point the direction how can I access via a proxy? Thanks!!

[Allie_Yang (Allie Yang Yang)]

Never mind, i managed to make it work in this way in output section (terminal with cli access to aws)

However, if i don't use cli access, but use access-key in comment below, it says but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided?. Both cli and the access key come from same person... Would you know why? Thanks!

amazon_es {
    hosts => ["example.com"]
    region => "us-east-1"
    # aws_access_key_id, aws_secret_access_key optional if instance profile is configured
    #aws_access_key_id => 'aaaaaaaa'
    #aws_secret_access_key => '99999999'
    index => "generic_logs"
}