Elasticsearch POST API call gives 401 unauthorized

Elastic Stack Elasticsearch
1

Hello,

We are using ELasticsearch 6.8 version and have enabled https for same. Accessing the URL is fine and it is also able to communicate with Kibana.

However we are trying to send some data using POST method to elasticsearch using this URL:

username:password@hostmachine:9200 along with https at the beginning

It is giving us an error as 401 unauthorized even if we have given the correct user and password.

Would anyone will be able to help or guide me to the place where I can troubleshoot the issue?

2

Hi @ashutos 6.8 is extremely old and you should update as a matter of urgency...

Run this command and show full results

curl -v -k -u username:password https://hostmachine:9200

1 Like
3

Is the version creating a hindrance?

Here is the result of command:

*   Trying 10.20.135.175:9200...
* Connected to hostmachine (10.20.135.175) port 9200
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server did not agree on a protocol. Uses default.
* using HTTP/1.x
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: hostmachine:9200
> Authorization: Basic ZWxhc3RpYzp6cjdZOHZXdkxXY3YzeDZFMmsxMw==
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 511
<
{
  "name" : "hostmachine-master",
  "cluster_name" : "elastic-cluster",
  "cluster_uuid" : "r1kHTZlmQ2SQpuopUkkJbQ",
  "version" : {
    "number" : "6.8.23",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "4f67856",
    "build_date" : "2022-01-06T21:30:50.087716Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.3",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host hostmachine left intact
4

Please always show the command plus the full result one without the other is only 50% of the information

Hmmm I think you edited the result because

There should have been some information about your certificate

* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.us-west1.gcp.elastic-cloud.com
*  start date: Feb  7 02:24:04 2024 GMT
*  expire date: May  7 02:24:03 2024 GMT
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* using HTTP/2

In short Elastic is working... with that username and password combo the -k says ignore cert issues.

You are having and issue with the POST since you have not shared... I do not know... what it is ... it may be a cert issues

So try this... is will create a test-index

curl -X POST -v -k "https://elastic:password@hostname:port/test-index/_doc" -d '{"foo" : "bar"}' -H "Content-Type: application/json"

Then try is without the -k

5

Thank you @stephenb for your response.

I ran the command in another remote machine and not on hostmachine which is why there is no info of certs. I have only edited the ip and host name.

Here is the command I used to create index and have attached the output as well:

curl POST -v -k "https: //elastic:password@hostmachine:9200/test-index-2/_doc" -d '{"foo" : "bar"}' -H "Content-Type: application/json"

Output:

* Rebuilt URL to: POST/
* Could not resolve host: POST
* Closing connection 0
curl: (6) Could not resolve host: POST
*   Trying 10.35.135.175...
* TCP_NODELAY set
* Connected to hostmachine (10.35.135.175) port 9200 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=hostmachine
*  start date: Apr  2 07:34:47 2024 GMT
*  expire date: Apr  2 07:34:47 2027 GMT
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using Basic with user 'elastic'
> POST /test-index-2/_doc HTTP/1.1
> Host: hostmachine:9200
> Authorization: Basic ZWxhc3RpYzp6cjdZOHZXdkxXY3YzeDZFMmsxMw==
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 15
>
* upload completely sent off: 15 out of 15 bytes
< HTTP/1.1 201 Created
< Location: /test-index-2/_doc/18lfvI4Bo2uLaphFZqln
< Warning: 299 Elasticsearch-6.8.23-4f67856 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template"
< content-type: application/json; charset=UTF-8
< content-length: 179
<
* Connection #1 to host hostmachine left intact
{"_index":"test-index-2","_type":"_doc","_id":"18lfvI4Bo2uLaphFZqln","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":0,"_primary_term":1}

It is able to create an index if the command is ran on host machine or any of the node machine but throws an error whenever ran from some other remote machine.

Also, please note that I am using self signed certificate generated by elasticsearch itself.

6

Unless you've gone out of your way to setup IP filters, Elasticsearch itself is not going to care which host the client is running on.

You might have networking issues, or some sort of proxy in place, but it's unlikely that Elasticsearch is giving a 401 based on the client address.

Can you please provide the same level of output for a failing request?

7

Hi Tim,

I haven't setup any IP filters.

Here is the output which I am getting when executed in remote machine:

* Could not resolve host: POST
* Closing connection
curl: (6) Could not resolve host: POST
*   Trying 10.35.135.175:9200...
* Connected to hostmachine (10.35.135.175) port 9200
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server did not agree on a protocol. Uses default.
* using HTTP/1.x
* Server auth using Basic with user 'elastic'
> POST /test-index-3/_doc HTTP/1.1
> Host: hostmachine:9200
> Authorization: Basic ZWxhc3RpYzp6cjdZOHZXdkxXY3YzeDZFMmsxMw==
> User-Agent: curl/8.4.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 5
>
< HTTP/1.1 400 Bad Request
< Warning: 299 Elasticsearch-6.8.23-4f67856 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template"
< content-type: application/json; charset=UTF-8
< content-length: 438
<
{"error":{"root_cause":[{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"not_x_content_exception: Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}},"status":400}* Connection #1 to host hostmachine left intact
* URL rejected: Port number was not a decimal number between 0 and 65535
* Closing connection
curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535
curl: (3) unmatched close brace/bracket in URL position 4:
bar}'
   ^
8

You have a malformed curl

Again, I asked that you show the command you ran exactly as you rant it except for you can change the password and the results when you don't show the command. It's very hard help

I noticed in the command above you left out -X before the POST

Also, please take a moment to format your code... I am doing it for You because I cannot read the results otherwise.

3 bactick before and after

Probably missing double quotes or something

9

I haven't used -X as it said it as unnecessary. Here is the command and it's output.

curl -X POST -v -k "https://elastic:password@hostname:9200/test-index-0/_doc" -d '{"foo" : "bar"}' -H "Content-Type: application/json"

Note: Unnecessary use of -X or --request, POST is already inferred.
* Could not resolve host: hostname
* Closing connection 0
curl: (6) Could not resolve host: hostname
[swxbuilduser@hostmachine ~]$ curl -X POST -v -k "https://elastic:password@hostmachine:9200/test-index-0/_doc" -d '{"foo" : "bar"}' -H "Content-Type: application/json"
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 10.35.135.175...
* TCP_NODELAY set
* Connected to hostmachine (10.35.135.175) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=hostmachine
*  start date: Apr  2 07:34:47 2024 GMT
*  expire date: Apr  2 07:34:47 2027 GMT
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using Basic with user 'elastic'
> POST /test-index-0/_doc HTTP/1.1
> Host: hostmachine:9200
> Authorization: Basic ZWxhc3RpYzp6cjdZOHZXdkxXY3YzeDZFMmsxMw==
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 15
>
* upload completely sent off: 15 out of 15 bytes
< HTTP/1.1 201 Created
< access-control-allow-credentials: true
< Location: /test-index-0/_doc/pNmBvo4BOVTCTwvyZlzb
< Warning: 299 Elasticsearch-6.8.23-4f67856 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template"
< content-type: application/json; charset=UTF-8
< content-length: 179
<
* Connection #0 to host hostmachine left intact
{"_index":"test-index-0","_type":"_doc","_id":"pNmBvo4BOVTCTwvyZlzb","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":0,"_primary_term":1}
10

Ok So that worked... what exactly is not working...

11

When I use the same command to create an index from another remote machine, it doesn't work. The error is attached in my previous reply.

One more thing, in documentation there is a certificate authority that we will have to mention along with private key and node certificate. In case of self signed I created the certificate authority using certutil, but when we use signed certificate what will be the certificate authority?

12

That typically means a connectivity issue/ firewall etc

Can you ping the elasticsearch host from the remote machine? Can you telnet to the elasticsearch, hostname and Port those are the way you check connectivity.

or the network interface you have Not bound elasticsearch to The correct network interface.

In your elasticsearch.yml what do you have this set to

network.host

I do not know which previous reply your referring to because there's been so many so you'll have to show exactly... So perhaps you should show again exactly what the error is with the command and the result from the remote machine.

In that case, your organization needs to provide the CA or or some organizations pre-install the CA all machines.. or if it's publicly signed by an authority say let's encrypt then you don't need a CA because it will be recognized.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.