What is HEAD request about?

Elastic Stack Elasticsearch
1

Hi! I ran into two questions during indexing.

  1. I saw a lot of HEAD request with 404 Not Found. I tried to find what HEAD request is about, but couldn't :frowning:

  2. The cluster health is in yellow - although all three nodes work actively.

Thanks in advance!

2
  1. Where do you see these requests? on which URL?
  2. Yellow means that not all shards have been started. Have a look at
    /_cat/shards ( https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-shards.html ) to see which shards are causing issues.
3

Hi Yannick,

For 1) I see those in elasticsearch log (in server).
2) would it be the same case when one of the nodes (I have three) is randomly shown unhealthy ?

4
  1. The HEAD requests are initiated either by a user or an application. Please give the full log lines. https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods
  2. I don't know what you mean by that. Please post your output for /_cluster/health?pretty
5
  1. Sorry I had to delete the actual names :frowning:
    [2016-03-24 17:59:42,900][INFO ][request ] [] HEAD /abc/ab/11111 - 404 NOT_FOUND 0 0
    [2016-03-24 17:59:42,900][INFO ][request_json ] {"time":"2016-03-24T17:59:42.900Z","starttime":"2016-03-24T17:59:42.900Z","localaddr":"12.12.1.31","localport":9200,"remoteaddr":"12.12.1.31","remoteport":23644,"scheme":"http","method":"HEAD","path":"/abc/ab/11111","querystr":"-","code":404,"status":"NOT_FOUND","size":0,"duration":0,"year":"2016","month":"03","day":"24","hour":"17","minute":"59","dow":"Thu","cluster":""}

{
cluster_name: "",
status: "yellow",
timed_out: false,
number_of_nodes: 2,
number_of_data_nodes: 2,
active_primary_shards: 18,
active_shards: 36,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 18
}

The number of unassigned_shards is 18 because one of the nodes fails alternatively between nodes. Like Node1 was gone for a while then Node1 got back without me doing anything and at the same time Node2 was gone..

6

  1. With "remoteaddr":"12.12.1.31" you can see that an application is issuing a HEAD request to a document in your cluster from that IP address. Do you have an application running that is accessing your cluster?

  2. Seems like you have index.number_of_replicas set to 2 (which means that you have 3 copies of each shard). As only 2 nodes are in the cluster, a third of the shards cannot be allocated. Why are nodes failing? You have to look into the logs.

7

  1. yeah.. actually, my original question was what HEAD request is about. :slight_smile: for example, we know PUT is to update, GET to search/get, POST to add/save, but I have not seen HEAD request until today.

  2. back to my original question this as well... there are three actively alive nodes, which are still indexing the given documents, but the cluster keeps saying only two of them are healthy, and the unhealthy node changes continuously.

8
  1. I linked you to Wikipedia which has the explanation. Do you want me to copy-paste the text here? :slight_smile: Googling might also help. HEAD requests are nothing Elasticsearch-specific.
  2. I asked you to look into the logs. If you don't provide any other information, I cannot help you.
9

Oh my bad! I'm sorry for missing the wikipedia link :sweat_smile:

As for the second, maybe I should wait until indexing is done and see what happens.. I only see 200s except the HEAD 404s in all three nodes.

Thank you for your help! (I should've said this earlier)

10

You're welcome.

11

A HEAD request in ES is used to determine if a document exists or not, so a 200 means the doc exists while a 404 means it doesn't.