Hi guys,
I have installed Elasticsearch 5.4.3 and Kibana 5.4.3 in the same server. They are communicating through their ip address at ports 9200 and 5601 respectively. Kibana is being accessed behind a proxy.
I have created multiple machine learnings jobs from Elasticsearch ML APIs, and through Kibana’s UI. However, at the time of listing the jobs I get errors such as “Jobs list could not be retrieved”, “undefined”, “datafeeds list could not be retrieved”, “Job details could not be retrieved”.
I need to note that I have created jobs from kibana’s UI, but they cannot be
retrieved at the Machine Learning Tab, even when they can be listed after created in elasticsearch. I am logging in into Kibana as “elastic” superuser, and have also tried “kibana”; but both of them raise the same error.
Based on some Google forums I have attempted the below
implementations:
-
Restart the cluster: I have restarted many times and there is no difference. https://www.elastic.co/guide/en/x-pack/current/ml-troubleshooting.html
-
Possible basePath error at kibana’s configuration: This link (https://github.com/elastic/kibana/issues/11950) represents the same issue that I have. However, they mention that the issue was fixed in version 5.4.1. I am running kibana behind a proxy and I cannot remove the basePath “/kibana” because it would not be practical for a production proxy.
Related to this advice, I found some logs that might be interesting.
Note: The only errors that I see at /var/log/kibana/kibana.stdout is a 404 request where there is a double “/” at the url.
{"type":"response","@timestamp":"2017-07-03T16:45:27Z","tags":[],"pid":11938,"method":"get","statusCode":404,"req":{"url":"//api/ml/datafeeds","method":"get","headers":{"host":"ip_address:5601","accept":"application/json,
text/plain,
/","kbn-system-api":"true","kbn-version":"5.4.3","user-agent":"Mozilla/5.0
(X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115
Safari/537.36","referer":"https://www-proxy/kibana/app/ml","accept-encoding":"gzip,
deflate, br","accept-language":"en-US,en;q=0.8","x-forwarded-for":"
ip_address","x-forwarded-host":"www-proxy ","x-forwarded-server":"www-proxy
","connection":"Keep-Alive"},"remoteAddress":"
ip_address ","userAgent":" ip_address ","referer":"https://www-proxy /kibana/app/ml"},"res":{"statusCode":404,"responseTime":10,"contentLength":9},"message":"GET //api/ml/datafeeds
404 10ms - 9.0B"}
- Used the Dev Tools Tab from Kibana’s UI and typed: “GET _xpack/ml/anomaly_detectors/” and was able to view all the machine learning jobs that are listed in elasticsearch without any problem.
{
"count": 4,
"jobs": [
{
"job_id": "apache_logs_3july",
"job_type": "anomaly_detector",
"description": "example advanced",
"create_time": 1499091675682,
"analysis_config": {
"bucket_span": "30m",
...........
I would like to get some advice related to this issue. Please let me know if I need to provide more information. More configuration details can be sent as requested.
Thanks in advance,
- Jordan