cURL does not download the Elastic Agent, only on one of our sites in Amsterdam (otherwise cURL works fine)

Hi there,
Has anyone ever encountered this issue, if so we would appreciate some guidance?

Thanks in advance, Br, SedonD

++++edit:
What I have seen so far....(something like this)

and/or

Sorry, I read it here so here it is in text:

type or paste code here 
root@eu2:~# curl -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   364  100   364    0     0   1612      0 --:--:-- --:--:-- --:--:--  1617
root@eu2:~# curl -I http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geopoint/documents.json.bz2
HTTP/1.1 200 OK
x-amz-id-2: +9Z3gSfuGsO3x7UiZkZN0No4NoJRVdCwWoka2TNms7ji9b1nMA35WxyvQ7KXeSvrK336iHFoUSloqd1olyMkdfo3lDKhXN4wHNBmOKlgzCU=
x-amz-request-id: P1EX75CVFG42NAJ8
Date: Sun, 25 Jun 2023 12:36:02 GMT
Last-Modified: Mon, 06 Apr 2020 12:45:46 GMT
ETag: "89af2e68151caf319ae4e1b3e821bcdc-61"
x-amz-server-side-encryption: AES256
x-amz-version-id: Naom.wGhv.aOMQcQ.V6N9P7rvzxKuq40
Accept-Ranges: bytes
Content-Type: application/json
Server: AmazonS3
Content-Length: 505542241

root@eu2:~# curl -I https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
HTTP/2 403
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

root@eu2:~# curl -L -O -I https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
root@eu2:~# curl -L -O https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.16-osx10.11-x86_64.dmg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  339M  100  339M    0     0  27.6M      0  0:00:12  0:00:12 --:--:-- 27.0M
root@eu2:~# curl -L -O -I https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
root@eu2:~# curl -L -O  https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   364  100   364    0     0   1552      0 --:--:-- --:--:-- --:--:--  1548

Welcome to our community! :smiley:

You are likely running into Are you getting 403's when downloading? Please read here first, so please add a post there.

@warkolm Hi there,
I don't get 403 :grinning:, that would be easy, because I would know what's behind it...
As you can see on the attachment image and in text section, download size is 364KiB and as if everything is fine, it exits the cURL from the process.... ergo it says that the content is downloaded

At first I didn't even notice what was happening, because it's a 10gig connected server and all downloads are fast and there is no substantial difference between 300-400KiB and 500MiB, when the unpack (tar xzvf) ran with an error and stopped I noticed the fact that I don't have the full - elastic-agent-8.8.1-linux-x86_64.tar.gz file

BTW:
As you can see I tested the server and the cURL with an old MySQL 5.7 download, which was 339M and ran without problems, but otherwise I can download everything else with wget and/or cURL, except the Elastic files

So the only way I could solve the agent installation was to SCP upload the installer file and run it, because I can't download Elstic stuff from this Amsterdam DC location I don't have this problem in Zurich and Frankfurt on our similar clusters...

+++edit:
Pls. look what I just did now (Logstash / Kibana):

root@eu2:~# curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   348  100   348    0     0    726      0 --:--:-- --:--:-- --:--:--   728
root@eu2:~# curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   344  100   344    0     0   1168      0 --:--:-- --:--:-- --:--:--  1166
root@eu2:~#

These are not normal sizes: 348 & 344 ..... always downloads these few hundred kilobytes and exits without an error message (says it is 100% downloaded)

That looks like a 403 hence why I mentioned it. Your post and output was a little confusing to be honest as you didn't explain what you were doing and it had references to mysql and another topic on Elasticsearch discovery.

Can you run this with a curl -v -O and post the output please?

Sorry I sometimes phrase things a bit confusingly......
on this topic users are asked to insert text instead of pictures, and I noticed this too late, so I apologized again
I didn't mean to imply anything else by this linked thread, and indeed it is not closely related to the current cURL topic :slightly_smiling_face:

Okhay cURL, here is the output:

root@eu2:~# curl -v -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2600:1901:0:1d7:::443...
* Connected to artifacts.elastic.co (2600:1901:0:1d7::) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* TLSv1.2 (IN), TLS header, Certificate Status (22):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.2 (IN), TLS header, Finished (20):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4658 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.2 (OUT), TLS header, Finished (20):
} [5 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=appsearch.elastic.co
*  start date: May  8 03:19:58 2023 GMT
*  expire date: Aug  6 04:13:30 2023 GMT
*  subjectAltName: host "artifacts.elastic.co" matched cert's "artifacts.elastic.co"
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1D4
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x55bdd0286560)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
> GET /downloads/beats/elastic-agent/elastic-agent-8.8.1-linux-x86_64.tar.gz HTTP/2
> Host: artifacts.elastic.co
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [267 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [267 bytes data]
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
< HTTP/2 403
< content-type: text/html; charset=UTF-8
< referrer-policy: no-referrer
< content-length: 364
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
<
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
100   364  100   364    0     0   1324      0 --:--:-- --:--:-- --:--:--  1323
* Connection #0 to host artifacts.elastic.co left intact

What version of curl are you using, and what OS?

root@eu2:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy
root@eu2:~# curl -V
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

And

Your curl outputs clearly says that you are getting a 403 error as @warkolm mentioned earlier.

The fact that you can download other things doesn't matter, Elastic is blocking some IP Addresses ranges as mentioned and this post.

You need to comment on that post and add the IP address your server is using to download.

Yuppp, - and thanks @leandrojmp , I've sent to @warkolm our IP address and AS info, - 1 hour before this.... (in PM))

+++edit:
@leandrojmp "You need to comment on that post and add the IP address your server is using to download."

sorry, but I would not make public our server IPs :wink:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.