Metricbeat WARN Cannot index event

I've seen a few posts like this, but none of them solved my problem, so:

I created a elastic cluster following this tutorial:

I've set the built in passwords, and I can reach elasticsearch

$ curl -k --user elastic:$ELASTIC_PWD https://localhost:9200/
  "name" : "es01",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "hqrGkTUGR0W2Clsaxp75pQ",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  "tagline" : "You Know, for Search"

I copied the certificate authority to the host machine (where I will be running metricbeat) as follows:

sudo -s
mkdir -p /etc/pki/root
docker exec kibana cat /usr/share/elasticsearch/config/certificates/ca/ca.crt > /etc/pki/root/ca.pem

I can check the certificate is there:

$ cat /etc/pki/root/ca.pem

So far so good, then, I installed metricbeat following this tutorial:

And set the following configs at /etc/metricbeat/metricbeat.yml:

  host: "https://localhost:5601"
  ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  hosts: ["https://localhost:9200"]
  username: "metricbeat"
  password: "<password>"
  ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

The metricbeat user I've created in Kibana following this:

Basically, I created a role called metricbeat_setup with cluster privileges monitor and manage_ilm and index privilege manage on metricbeat-*, then, I created the metricbeat user with roles metricbeat_setup, kibana_admin, ingest_admin, beats_admin, beats_system

I think my problem might be here, I didn't quite understand the necessary permissions so I just went overkill, anyway, continuing...

I ran the setup sudo metricbeat setup -e and everything went fine, no errors in setup, then I started it with sudo service metricbeat start, but nothing gets to elasticsearch, and when I look at sudo service metricbeat status, I get multiple lines of WARN Cannot index event

I tried to look at the logs in /var/log/metricbeat/ but found nothing useful there, also in elasticsearch logs in docker, no useful information as well.

The metricbeat-* index is created and the shard is allocated:

$ curl -k --user elastic:$ELASTIC_PWD https://localhost:9200/_cat/indices/metricbeat-*?pretty
green open metricbeat-7.7.0-2020.05.25-000001 p_0nuiX1S8SARk9QZK01EA 1 1 0 0 566b 283b
$ curl -k --user elastic:$ELASTIC_PWD https://localhost:9200/_cluster/allocation/explain?pretty
  "error" : {
    "root_cause" : [
        "type" : "illegal_argument_exception",
        "reason" : "unable to find any unassigned shards to explain [ClusterAllocationExplainRequest[useAnyUnassignedShard=true,includeYesDecisions?=false]"
    "type" : "illegal_argument_exception",
    "reason" : "unable to find any unassigned shards to explain [ClusterAllocationExplainRequest[useAnyUnassignedShard=true,includeYesDecisions?=false]"
  "status" : 400

So, please, what am I missing here? I can't find any relevant information in logs or in posts, I've seen posts here saying this was related to lack of disk space, I have 14G free, so I guess that's not the problem. Also, I know metricbeat can connect to both elasticsearch and kibana, because it was able to create the index and the dashboard, I can open the dashboard in kibana but there's no data.

What am I doing wrong?

Thank you


Can you post the output of the following CAT APIs?




Also, it may help if you enable debug logging in metricbeat via the logging.level flag. If there's any errors reported at the debug level, could you paste them as well?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.