TL;DR
I want to ship logs from remote deployments via filebeat or logstash to a central Elastic/Elastic stack via HTTP exposing just domains
I have considered two solutions, cant get any to work
The setup is as follows:
- there is a central big framework on the company's server A,
- and multiple clients services each one in a different server B,C,D etc.
NONE is in the same network. All services are on docker containers with compose.
We set up Elastic Stack on A.
The initial plan was to add filebeat on each server B C D and ship to logstash-elastic on A
However A sits behind a company firewall and haproxy etc. They are able to open a domain and subdomains and map to a specific ports. For elastic and logstash
https://elastic.mycompany.com : 9200
https://logstash.mycompany.com : 5044
- The plan to ship logs with filebeat doesnt work, I guess because it can only connect via TCP and not HTTP
The following two tries fail
output.logstash:
enabled: true
hosts: ["https://logstash.mycompany.com"]
Failed to connect to backoff(async(tcp://https://logstash.mycompany.com)): lookup https on 127.0.0.28:53: no such host
and setting just the domain it adds the 5044
output.logstash:
enabled: true
hosts: ["logstash.mycompany.com"]
Failed to connect to backoff(async(tcp://logstash.mycompany.com):5044)): dial tcp [2606:asdfasdf::asdfasdf::]:5044: connect: network is unreachable
Going to plan 2
- Deploy filebeat and logstash in each server B C D and ship to elastic on A
The following setup doesnt work, the 9200 is added after the domain
logstash.yml (on servers B C D)
output {
elasticsearch {
hosts => ["https://elastic.mycompany.com"]
index => "logs-%{+YYYY.MM.dd}"
document_type => "nginx_logs"
user => "elastic"
password => "changeme"
}
stdout { codec => rubydebug }
}
][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elastic.mycompany.com:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@elastic.mycompany.com:9200/][Manticore::ResolutionFailure] elastic.mycompany.com"}
I am thinking another solution, maybe use the http plugin and send via REST requests ? I need some help with this configuration, what I have until now is the following (not working)
output {
http {
url => "https://elastic.mycompany.com"
http_method => "post"
content_type => "text/xml;charset=UTF-8"
index => "logs-%{+YYYY.MM.dd}"
headers => {
"Authorization" => "Basic ZWxhc3RpYzpjaGFuZ2VtZQ=="
}
format => "message"
}
}
Thank you for your help!