Dashboard: Could not locate that index-pattern-field (id: system.syslog.hostname) and other errors

Elastic Stack Kibana
1

Hi all.
In my Dasbords I have lot of messages like

Could not locate that index-pattern-field (id: system.syslog.hostname)

I'am using
Kibana 6.4.2
Logstash 6.4.2
Elasticsearch 6.4.2.

While setting up Filebeat 6.4.2 I made it to use Logstash, so I uploaded indexes manualy like this:
filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

and uploaded Dashboards like this:

filebeat setup -e \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['127.0.0.1:9200'] \
  -E setup.kibana.host=elk.mydomain.ru:5601

So new data is seen in Discovery but there are errors in Dashboards like I've mentioned in subj.

2

Did you do an import at some point? It sounds like some of the visualizations on your dashboard are pointing to an index pattern that doesn't exist. For each of those visualizations you can go into the saved object editor and change the index pattern ID to the correct one.

3

For "clean experiment" I've stoped Filebeat, completely deleted all Visualizations, Dashboards, cleared indexes with
curl -XDELETE 'http://localhost:9200/filebeat-*'

After It I've added indexes:
# curl -XDELETE 'http://localhost:9200/filebeat-*'
{"acknowledged":true}
# curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/filebeat-6.4.2 -d@filebeat.template.json
{"acknowledged":true}

uploaded Dashboards:
# filebeat setup -e
> -E output.logstash.enabled=false
> -E output.elasticsearch.hosts=['127.0.0.1:9200']
> -E setup.kibana.host:5601=elk.zonatelecom.ru
2018-10-22T10:06:15.979+0300 INFO instance/beat.go:544 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2018-10-22T10:06:15.979+0300 INFO instance/beat.go:551 Beat UUID: bded7bde-a6e8-43e5-aadb-1807a1d5ed35
2018-10-22T10:06:15.979+0300 INFO [beat] instance/beat.go:768 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "bded7bde-a6e8-43e5-aadb-1807a1d5ed35"}}}
2018-10-22T10:06:15.980+0300 INFO [beat] instance/beat.go:777 Build info {"system_info": {"build": {"commit": "e193f6d68b25b7ddbe3a3ed8d60bc07fea1ef800", "libbeat": "6.4.2", "time": "2018-09-26T12:42:46.000Z", "version": "6.4.2"}}}
2018-10-22T10:06:15.980+0300 INFO [beat] instance/beat.go:780 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":64,"version":"go1.10.3"}}}
2018-10-22T10:06:15.984+0300 INFO [beat] instance/beat.go:784 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-10-18T15:43:48+03:00","containerized":false,"hostname":"elk.zonatelecom.ru","ips":["127.0.0.1/8","::1/128","172.20.71.119/24","fe80::92b1:1cff:fefd:8942/64"],"kernel_version":"4.9.0-7-amd64","mac_addresses":["90:b1:1c:fd:89:42","90:b1:1c:fd:89:44","90:b1:1c:fd:89:46","90:b1:1c:fd:89:48"],"os":{"family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"9 (stretch)","major":9,"minor":0,"patch":0,"codename":"stretch"},"timezone":"MSK","timezone_offset_sec":10800,"id":"1484af230da94415825fe965660c4e4d"}}}
2018-10-22T10:06:15.985+0300 INFO [beat] instance/beat.go:813 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/root", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 46516, "ppid": 14709, "seccomp": {"mode":"disabled"}, "start_time": "2018-10-22T10:06:15.740+0300"}}}
2018-10-22T10:06:15.985+0300 INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.2
2018-10-22T10:06:15.986+0300 INFO elasticsearch/client.go:163 Elasticsearch url: http://127.0.0.1:9200
2018-10-22T10:06:15.986+0300 INFO pipeline/module.go:98 Beat name: elk.zonatelecom.ru
2018-10-22T10:06:15.986+0300 INFO elasticsearch/client.go:163 Elasticsearch url: http://127.0.0.1:9200
2018-10-22T10:06:15.989+0300 INFO elasticsearch/client.go:712 Connected to Elasticsearch version 6.4.2
2018-10-22T10:06:15.993+0300 INFO template/load.go:129 Template already exists and will not be overwritten.
Loaded index template
Loading dashboards (Kibana must be running and reachable)
2018-10-22T10:06:15.996+0300 INFO elasticsearch/client.go:163 Elasticsearch url: http://127.0.0.1:9200
2018-10-22T10:06:15.998+0300 INFO elasticsearch/client.go:712 Connected to Elasticsearch version 6.4.2
2018-10-22T10:06:15.998+0300 INFO kibana/client.go:113 Kibana url: http://elk.zonatelecom.ru:5601
2018-10-22T10:06:44.142+0300 INFO instance/beat.go:659 Kibana dashboards successfully loaded.
Loaded dashboards
2018-10-22T10:06:44.143+0300 INFO elasticsearch/client.go:163 Elasticsearch url: http://127.0.0.1:9200
2018-10-22T10:06:44.145+0300 INFO elasticsearch/client.go:712 Connected to Elasticsearch version 6.4.2
2018-10-22T10:06:44.145+0300 INFO kibana/client.go:113 Kibana url: http://elk.zonatelecom.ru:5601
2018-10-22T10:06:44.206+0300 WARN fileset/modules.go:388 X-Pack Machine Learning is not enabled
2018-10-22T10:06:44.263+0300 WARN fileset/modules.go:388 X-Pack Machine Learning is not enabled
Loaded machine learning job configurations

Well I opened Dashboard (for example [Filebeat System] SSH login attempts) I see
#### No results found
It's OK - I have no data collected yet.

I've started Filebeat:
# systemctl start filebeat

In Discovery I can see new data;

Now I go to Dashboards / [Filebeat System] SSH login attempts and see...
Could not locate that index-pattern-field (id: system.auth.ssh.event)
Could not locate that index-pattern-field (id: system.auth.ssh.method)
Could not locate that index-pattern-field (id: system.auth.user)
Could not locate that index-pattern-field (id: system.auth.ssh.geoip.location)

What am I doing wrong?
How can I fix this Issue?

4

Hmm, could you try going into your index pattern settings and refresh the index pattern?

5

I've tried Management - Index Patterns - Refresh field list (you mean this, yes?) - I still have lots of error like
Could not locate that index-pattern-field (id: system.auth.ssh.event)
in Dashboards

6

So I've found simple solution:

I've deleted all indices, visualusations, dashboards. In /etc/filebeat/filebeat.yml I've configured
output.elasticsearch and commented output.logstash.
Then I've just run
filebeat setup
It uploaded template and dashboards. I've started filebeat and so new data in Deiscovery - and no errors in Dashboards!

So I came back to /etc/filebeat/filebeat.yml, commented output.elasticsearch and uncommented output.logstash.

No it works fine, sending data to Logstash

Thank you for help!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.