Custom index not showing in Kibana V8.8.0

Elastic Stack Kibana
1

I have multiple filebeats v 7.17.5 are configured on different remote servers with custom index names. I have recently updated my ELk stack to 8.8.0 and also updating the filebeat version to 8.8.0. I also try to add some new remote ubuntu servers in elk but the issue is i try to create custom index and start the filebeat but that index is not showing in kibana v8.80 data view. here is the my custom index settings.

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: "http://x.x.x.x.x:9200"
  username: "citsale"
  password: "mFVKkXXNnOuUzgahLpLpp"
  # Protocol - either `http` (default) or `https`.
  ssl.verification_mode: none

  # Authentication credentials - either API key or username/password.
  index: "test"
setup.template.name: "test"
setup.template.pattern: "test-*"
2

Does it show in GET _cat/indices?v

3

Nops, its not showing in `GET _cat/indices?v'

4

Ok then Filebeat isn't sending anything to Elasticsearch. You might need to check the logs of it.

1 Like
5

As Mark said, check FB logs and the network communication. If you don't have logs, enable it at least temporary

For test purpose, you can see does FB read a log by changing:

output.console:
  pretty: true
6

I found some error but didn't know why its occurring i was following the same procedure in FB 7.17.7 to load the template in ELK and that was working, but when i tried to same steps for FB 8.8.0 i am getting the below error.

Exiting: error loading template: failed to put data stream: could not put data stream: 400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [linux]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [linux]"},"status":400}. Response body: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [linux]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [linux]"},"status":400}

7

i also tried to disable the data stream but no luck.

8

So you had 7.17.7 and go for FB 8.8.0? Do you have additional modules enabled? It sounds like you have the system module enabled.

9

Yes, i had using 7.17.7 and then upgraded to 8.8.0. Yes, i have enabled the modules apache
nginx, system

10

The most likely you haven't completed the upgrade process.
Follow the upgrade steps.

./filebeat setup --index-management
./filebeat setup --dashboards

If this will not work then filebeat setup -e.
There is an option to export templates for only 3 modules, and manually create, described here , and you will need dashboards, export, import.

11

Hi @Rios thanks for the help. I can see the index now in Index Templates, but when i trying to create a DataView to view the logs of filebeat so that index is not showing there

12

while running the filebeat setup -e the index loaded in index template but getting the below error.
Exiting: error loading template: failed to put data stream: could not put data stream: 400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [staging]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [staging]"},"status":400}. Response body: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"no matching index template found for data stream [staging]"}],"type":"illegal_argument_exception","reason":"no matching index template found for data stream [staging]"},"status":400}

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.