Hi,
I have successfully configured filebeat clients to stream data to centralized ELK server. Now i want to to visualize the mongo query performance data in various dashboards. For this is followed the instructions as below :
On ELK server
mkdir mongo_kibana_template ; cd mongo_kibana_template
curl -L -O http://download.elastic.co/beats/dashboards/beats-dashboards-1.1.1.zip
unzip beats-dashboards-1.1.1.zip
cd beats-dashboards-1.1.1/
./load.sh
I also appended the following snippet of config on the filebeat client running on Mongo server.
On Filebeat client
protocols:
mongodb:
ports: [27017, 27019]
send_request: true # index the request payload
send_response: true # index the response payload
max_docs: 10 # maximum number of documents to index per request/response
max_doc_length: 1024 # maximum document size to index
Besides the above rest of the config like TLS certificate addition, log file path were added as usual.
Post the above steps I got new dashboard and visualizations on my existing ELK server
but Data was missing.
Please point out if I am missing a step ?
Please note : data is being streamed to the ELK server normally from the FileBeat client.
(verified by checking checking KIBANA for logs from specific hostname.)
But same is not being picked up by the dashboard.
Sample dashboard i got following the above steps was like following but Without Data
http://goo.gl/jTPG63
Reference links for above steps :
Ref1: https://goo.gl/tcW954
Ref2: https://goo.gl/89gMNE
PFB : FileBeat client config
Filebeat.yml
PFA the screenshots
Data is being streamed normally to the ELK server since same is searchable based on Mongo Server hostname - as below
Index Mapping config warning -- post running load.sh
Dashboard data not being fetched;
Error : Could not locate that index-pattern-field (id: resource)
System info :
Filebeat Client
filebeat-1.1.1-1.x86_64
OS: CentOS release 6.7 (Final)
Linux ip-172-31-57-210 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
ELK server
OS : Ubuntu 14.04.3 LTS (GNU/Linux 3.19.0-49-generic x86_64)
Kibana : 4.3.0
Elastic search details :
"name" : "Zip-Zap",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.2.0",
"build_hash" : "8ff36d139e16f8720f2947ef62c8167a888992fe",
"build_timestamp" : "2016-01-27T13:32:39Z",
"build_snapshot" : false,
"lucene_version" : "5.4.1"
},
Config File for filebeat client running mongodb