Unable to collect data in elastic cloud using internet proxy

[Gauti (Gautham)]

Hey Guys,

I have been cracking my head from past three days.
Got my elastic cloud instance, when i'm trying to send data to elastic cloud from my servers i'm getting error like it was not able to resolve the elastic cloud host name.

My servers doesn't have a direct internet, i have configured internet proxy in my servers, i was not able to send any of the data to elastic cloud through http_poller method. I have even tried installing metric beat as well...but no luck..

There was no issue at all wen i was doing it on premise, this cloud instance is killing me.... somebody please help me.... What setting i'm missing here???

Output test fyr

PS C:\Program Files\metricbeat> .\metricbeat.exe test output
elasticsearch: http://120bf473d8e57f32e5de3f7713d8c0e9.ap-southeast-1.aws.found.io:9243...
** parse url... OK**
** connection...**
** parse host... OK**
** dns lookup... ERROR lookup 120bf473d8e57f32e5de3f7713d8c0e9.ap-southeast-1.aws.found.io: getaddrinfow: This is usual**
ly a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server.

Thanks
Gautham

[andrewkroh]

What kind of proxy are you using? How is it configured? What version of Metricbeat? What OS+Version?

The config for a proxy is like this:

output.elasticsearch:
  hosts: ["https://elastic-cloud:9243"]
  proxy_url: squid:3128

And you will see a log message like:

2018-05-25T22:10:08.161-0400	INFO	elasticsearch/elasticsearch.go:109	Using proxy URL: http://localhost:3128

[Gauti (Gautham)]

Thanks @andrewkroh after using the proxy config u sent it started working.

Thanks
Gautham

[Gauti (Gautham)]

Got an another issue @andrewkroh .
it is working properly if if execute with below command
metricbeat -e -E cloud.id="" -E cloud.auth="<cloud.auth>"

if i enter the cloud id and authentication details on metricbeat.yml i'm getting the same error like unable to resolve

Error FYI:
PS E:\metricbeat-6.2.3> .\metricbeat.exe setup
Loaded index template
Exiting: Error importing Kibana dashboards: fail to create the Kibana loader: Error creating Kibana client: fail to get
the Kibana version:HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get https://eda9d69a1aaf
7252e823276e9c884b70.ap-southeast-1.aws.found.io:443/api/status: lookup eda9d69a1aaf7252e823276e9c884b70.ap-southeast-1.
aws.found.io: getaddrinfow: This is usually a temporary error during hostname resolution and means that the local server
did not receive a response from an authoritative server.. Response: .

Thanks
Gautham

[andrewkroh]

Perhaps the setup code for talking to Kibana is not honoring the proxy settings?

Can you try setting and environment variable like HTTP_PROXY=squid:3128 before running the setup command. My guess is that the Kibana code doesn't actually honor that setting, but as a workaround it might be reading that environment variable because it's the default behavior for Go's http library.