Not able to create watcher

Elastic Stack Elasticsearch
1

Hi,

I have created a watcher. When I am trying to execute the same, I am getting below error:

{
  "error": {
    "root_cause": [
      {
        "type": "cluster_block_exception",
        "reason": "blocked by: [SERVICE_UNAVAILABLE/2/no master];"
      }
    ],
    "type": "cluster_block_exception",
    "reason": "blocked by: [SERVICE_UNAVAILABLE/2/no master];"
  },
  "status": 503
}

Please let me know, what to do to resolve this?

2

your cluster is lacking a master node. This also means you cannot index new data, so you should fix this now.

--Alex

3

Hi @spinscale, how can I fix the same? I can see below in my Overview page.

tiebreaker-0000000000 	running 	1024 	tiebreaker 	AlQvnwUMTgi5XcDtVwFtRg 	6.2.3 	
24%
	
0.0%
instance-0000000002 	running 	4096 	zone-1 		6.2.3 	
58%
	
2.3%
instance-0000000001 	running 	4096 	zone-0 	AlQvnwUMTgi5XcDtVwFtRg 	6.2.3 	
6%
	
0.0%
4

can you show the output of GET _cat/nodes?v

5

here is the output of the same:

ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.25.166.159           30          98  13    0.41    0.50     0.59 mi        *      tiebreaker-0000000000
172.25.108.147           11          97   1    0.41    1.05     1.06 mdi       -      instance-0000000001
172.25.5.91              57          99  21   11.50   13.87    12.76 mdi       -      instance-0000000002
6

you do have a master node here, can you retry? If it does not work, can you restart watcher and paste the responses?

Also can you show the exact comands you executed to create/execute the watch?

7

Hi @spinscale, I am trying from long time, but it is not working.
Sorry, but I do not have an idea on how to restart only watcher in x-pack?
Actually watcher is not created only!
When I am trying to create the watcher(qa-mon-oom) in dev tool, I am getting error, I mentioned earlier.

PUT _xpack/watcher/watch/qa-mon-oom
{
  "trigger": {
    "schedule": {
      "interval": "1s"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": "qa-mon-serverlogs*",
        "types": "doc",
        "body": {
          "query": {
            "match_all": {}
          },
          "sort":[
            {
              "logTimestamp":{
                "order": "desc"
              }
            }
          ]
        }
      }
    }
  },
  "condition": {
    "compare":{
      "ctx.payload.hits.hits.0._source.syslog_message": {
        "gt": "OutOfMemoryErrro"
      }
    }
  },
  "actions": {
  "send_email" : { 
    "throttle_period": "10m",
    "email": {
      "to": "abc@def.com", 
      "subject": "QA-mon-OOM", 
      "body": "Error message: {{ctx.payload.hits.hits.0._source.syslog_message}} "
    }
  }
 }
}
8

You should start reading your logfiles, and find out why no master node is being elected (I have the assumption there might be more than one node elected, btu it is super hard to tell with the given information, you would need to run the cat nodes API against all nodes).

This is not a watcher issue, but a cluster issue. You could also restart your nodes, but I have no idea if this will solve your issue, so reading logfiles first makes more sense.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.