HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat"

Elastic Stack Beats
1

The metric beats service status showing that connection estabilished to elasticsearch but I get this errors
metricbeats.yml

 path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression

setup.kibana:
  host: "https://192.168.21.111:5601"

output.elasticsearch:
  hosts: ["http://192.168.21.111:9200"]
  username: "elastic"
  password: "elastic"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"info","@timestamp":"2023-01-25T13:19:55.084Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":187},"message":"Non-zero metrics in the last 30s","service.name":"metricbeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":547759543}},"memory":{"mem":{"usage":{"bytes":62668800}}}},"cpu":{"system":{"ticks":4530,"time":{"ms":230}},"total":{"ticks":10210,"time":{"ms":550},"value":10210},"user":{"ticks":5680,"time":{"ms":320}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":9},"info":{"ephemeral_id":"140f7ccf-3e4c-4b81-9448-1dd8e6ac48e3","uptime":{"ms":513193},"version":"8.6.0"},"memstats":{"gc_next":26245352,"memory_alloc":21400904,"memory_total":862290848,"rss":147709952},"runtime":{"goroutines":93}},"libbeat":{"config":{"module":{"running":4}},"output":{"events":{"acked":74,"active":0,"batches":6,"total":74},"read":{"bytes":2654},"write":{"bytes":153924}},"pipeline":{"clients":19,"events":{"active":0,"filtered":6,"published":74,"total":80},"queue":{"acked":74}}},"metricbeat":{"elasticsearch":{"ccr":{"events":3,"failures":3},"cluster_stats":{"events":3,"failures":3},"enrich":{"events":3,"failures":3},"index":{"events":3,"failures":3},"index_recovery":{"events":3,"failures":3},"index_summary":{"events":3,"failures":3},"ml_job":{"events":3,"failures":3},"node_stats":{"events":3,"failures":3},"shard":{"events":3,"failures":3}},"system":{"cpu":{"events":3,"success":3},"filesystem":{"events":10,"success":10},"fsstat":{"events":1,"success":1},"load":{"events":3,"success":3},"memory":{"events":3,"success":3},"network":{"events":9,"success":9},"process":{"events":18,"success":18},"process_summary":{"events":3,"success":3},"socket_summary":{"events":3,"success":3}}},"system":{"load":{"1":0.07,"15":0.01,"5":0.04,"norm":{"1":0.0044,"15":0.0006,"5":0.0025}}}},"ecs.version":"1.6.0"}}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.cluster_stats: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.ccr: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.index_recovery: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.shard: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.ml_job: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.enrich: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.090Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.index_summary: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.091Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.node_stats: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
Jan 25 13:19:55 elasticsearch06s01 metricbeat[77710]: {"log.level":"error","@timestamp":"2023-01-25T13:19:55.091Z","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.index: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
2

Are username/pass OK? Test with curl

3 
curl -u elastic:elastic  -X GET "192.168.21.111:9200/_cluster/health?pretty"
{
  "cluster_name" : "kds",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 19,
  "active_shards" : 19,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 3,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 86.36363636363636
}
4

Set user/pass for Kibana
Do you have enabled modules? If you do have, for instance the elasticsearch module, set also there in the module yml file.

5 
 metricbeat modules list
Enabled:
elasticsearch-xpack
kibana
system

vim /etc/metricbeat/modules.d/kibana.yml

- module: kibana
  #metricsets:
  #  - status
  period: 10s
  hosts: ["https://192.168.21.111:5601"]
  #basepath: ""
  username: "elastic"
  password: "elastic"

elastic is super user, I can login via kibana interface

6

You have enabled: elasticsearch-xpack and kibana. For both you have to set params.
Note HTTP is for ES, and HTTPS for kibana. Check also for elasticsearch-xpack, use refence

7

thanks for reply,
when I set xpack.security.enabled: false in elasticsearch.yml then metricbeats connecting and fetching data well, but securitiy section in kibana gui returning with errors.
Actually I'm a little bit confused about certificates. elasticsearch-certutil generate *.p12 files but in metricbeat.yml claim #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] a ca.pem file
How can I generate ca.pem to metricbeats that compatible with elasticsearch ssl
In elasticsearch I follow the documentation and set ssl by

/usr/share/elasticsearch/bin/elasticsearch-certutil ca
/usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
8

I found this command and its work
openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -out elastic-ca.pem
topic can be closed

1 Like
9

Steps to Fix the 401 Unauthorized Error
Confirm the URL Is Correct. This might sound obvious, but the 401 error code might appear if the user entered the wrong URL in the browser's address bar.
Clear User End Issues.
Check Authentication Credentials.
Disable Password Protection.
Troubleshoot the Code.

Regards,
Rachel Gomez

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.