No matching indices found: No indices match pattern "filebeat-*"

[napoleon182]

Hi All, I have big problem with my recent deploy of ES version 7.8 using docker. I'm trying to connect my filebeat sources, but I have following ERROR in the Stack Management 7.8.0 (see screenshot below). Some of the sources point me into the Roles/Users, interestingely i dont see anything under Stack Management > Roles - any help much appreciated!

[fl33t]

Hi @napoleon182 , new guy still cutting his chops on the stack and forums, so pls bear with me. The message seems to indicate that you have no indices with that name pattern. Are there any indices in your Index Management that match that pattern?

I.e.: filebeat-8.1.3-2022.04.22

If your beats haven't been able to connect or send yet, it might be normal.

Also, are you sending directly to elastic or going through logstash?

[napoleon182]

hey mate, nws thanks for replying. I'm sending directly to elastic. I dont see any Indices in my Index Management.

Do you think this could be because my beats does not have enough permissions when sending data?

[stephenb (Stephen Brown)]

@napoleon182

Do you think this could be because my beats does not have enough permissions when sending data?

Yes possible or there is an Auth or connectivity issue.

The easiest way to tell is to look at the filebeat startup logs...

Take a look at them if you see something let us know...

You should see a connection issue or failed to index event or something.

You can also triple check by going to Kibana -> Dev Tools and running the command

GET _cat/indices/filebeat*/?v

[napoleon182]

@stephenb

GET _cat/indices/filebeat*/?v

Hi Stephen, I have run the GET _cat command in Dev Tools, this is the output i have:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

As part of troubleshooting I also started the windows metricbeat on my laptop i have this error:

Exiting: error loading template: failed to load template: couldn't load template: 400 Bad Request: {"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:331501] [index_template] unknown field [data_stream]"}],"type":"x_content_parse_exception","reason":"[1:331501] [index_template] unknown field [data_stream]"},"status":400}. Response body: {"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:331501] [index_template] unknown field [data_stream]"}],"type":"x_content_parse_exception","reason":"[1:331501] [index_template] unknown field [data_stream]"},"status":400}

Will that mean the user permissions are not setup correctly? I haven't setup nothing in the metricbeat.yaml:

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["lenovo-thinkcentre.local:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  #username: "elastic"
  #password: "changeme"

And thats because in the Stack Management 7.8.0 i dont see Users section, see screenshot

So when i try to load any of my dashboards i see this - "No matching indices" (on the right hand):

Hope it makes sense

[stephenb (Stephen Brown)]

I do not believe this has to do with users / roles, the reason you do not see those sections as you have not enabled security... no security ... no users and roles.

What version of metricbeat are you running.?

I suspect you are running a new 8.x metricbeat with an old 7.8 Elasticsearch and they are not easily compatible.

That error message indicates metricbeat is try to send a datastream.. which 7.8 is not ready to receive.

I suggest download the same version of metricbeat as your elasticsearcht.. 7.8 and try that.

Is there a reason you are running such an old version of Elasticsearch? You should try to match the versions.

Also are you running setup before starting metricbeat?

metricbeat setup -e

[napoleon182]

hey Stephen, thank you so much, you were on point, i was using metricbeat version 8.0 with Elasticsearch 7.8

After correcting i can see indexes being created. I will think of upgrading ES version to 8.0 - at the moment I'm learing deployment in docker and 7.8 is the easiest for me so far.

And you are right about security, i have not enabled security in my Elasticsearch.yml file, hence i dont see users and roles.

Thanks again for your assistance, Tomek

[system]

Elasticsearch 7.8 is EOL and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns )

[stephenb (Stephen Brown)]

@napoleon182
Cool! At Least use 7.17.3 security is still not required and it is WAAAAAY further advanced featurewise than 7.8 which as our bot said is past EOL... you same compose should work with 7.17.3

[napoleon182]

yeah, just seen the bot message about EOL on 7.8 I will look into the docker for ELK 8.0 and you are right about security, at this point i dont want overcomplicate.

Cheers and have a good day!