Kibana 5.2 +x-pack error when generating reports

Elastic Stack Kibana
62

@Apache_HOU you are correct regarding the different timeouts.

If the hard-coded application query selector timeout is hit, you'll get the Kibana took too long to load error.

If you are hitting the configurable xpack.reporting.queue.timeout, you'll get an error similar to Max attempts reached (3).

63

@Brandon_Kobel

Ok, thanks for the explanation.

I continued my test and found some new things :

  • If the VM who installed ELK can access Internet directly , the reporting works well
  • If the VM who installed ELK can't access Internet or need an account proxy to access Internet, the reporting doesn't work !

Could you please confirm if the reporting fuction needs Internet ? If yes, what does it do to access Internet and which URL it needs to access ? In addition, how to do for the company who use proxy to access Internet ?

64

@Apache_HOU

The server/VM running Kibana shouldn't require access to the public internet. It does however need access to Kibana itself. The way that Reporting works is that the Kibana server creates an instance of PhantomJS (a headless browser) that opens up Kibana and captures images of the reports.

That's what the Reporting KibanaServer properties are for, it tells the Kibana server the information it needs to use PhantomJS to open up Kibana in the headless browser to capture the images.

So, if you have the server running Kibana firewalled to block outbound traffic to whatever port Kibana is running on, then you will get the Kibana took too long to load error.

65

@Brandon_Kobel

  1. The firewall was stopped and I use URL http://IP:5601 to access Kibana so there's no firewall blocked
  2. My VM use bridge mode. With the same configuration, when it's in company network need proxy to access Internet, it doesn't work; When i use home network which can access Internet directly, it works well. Furthermore, if i change the VM to host-only mode with no network environment, it doesn't work either.
66

@Apache_HOU I just confirmed that we don't need public internet access to generate reports.

When you're connected to your work internet, if you curl http://IP:5601 from your VM, what do you see?

67

@Brandon_Kobel

There's no output when I do curl http://IP:5601 but the output is shown as below when I added the user and password since it needs authentification after installing x-pack:

[root@machine ~]# curl -u elastic:elastic http://10.151.27.179:5601

<script>var hashRoute = '/app/kibana';
var defaultRoute = '/app/kibana';

var hash = window.location.hash;
if (hash.length) {
  window.location = hashRoute + hash;
} else {
  window.location = defaultRoute;
}</script>
68

@Apache_HOU can you try removing the xpack.reporting.kibanaServer settings, and generating a report?

69

@Brandon_Kobel

With removing all xpack.reporting.kibanaServer settings in kibana.yml, it still doesn't work. The same error Error: Kibana took too long to load - Timeout exceeded (30000)

70

@Apache_HOU can we try stopping everything besides Kibana and ES, especially anything that is actively indexing documents in Elasticsearch (including metricbeat) to ensure that nothing else is interfering?

71

@Brandon_Kobel

I just tested your suggestion but it still doesn't work and always has the same error.

The VM is dedicated to ELK so in my test case there were only ES and Kibana running.

72

@Apache_HOU I'm investigating a similar issue from another user, I'll let you know what we find out regarding their underlying issue.

73

@Brandon_Kobel Ok, looking for forward to your news

74

@Apache_HOU it ends up that Kibana itself does make a request to the public internet when rendering visualizations to support the Tilemap. There's an underlying issue with the way that this is handled in Kibana that can cause Reporting to timeout because the 'loading' can just stall out - https://github.com/elastic/kibana/issues/10659

75

@Apache_HOU as a temporary workaround you can set the following in your Kibana.yml that will disable the call to retrieve the Tilemap manifest that is timing out:

tilemap:
  url: "https://tiles.elastic.co/v2/default/{z}/{x}/{y}.png?elastic_tile_service_tos=agree&my_app_name=kibana"
  options:
    minZoom: 1
    maxZoom: 10
    attribution: "© [Elastic Tile Service](https://www.elastic.co/elastic-tile-service)"

It should be noted that this is a temporary work-around, and it might not work long-term as we're essentially hard-coding information that could potentially change at any point in time. I'd recommend only relying upon the above until https://github.com/elastic/kibana/issues/10205 is resolved and upgrading to that version.

76

@Brandon_Kobel

Thanks for the updates.

It works by changing the tilemap.url to a 3rd party service.

However, could you please tell me in which case tilemap manifest will be requested ? Why x-pack reporting function will trigger it ?

As I don't use tilemap visualizations in my dashboard and I don't have any problem when Kibana loads visualization or dashboard.

77

@Apache_HOU it's a bug that is being tracked in https://github.com/elastic/kibana/issues/10205. The request to get the Tilemap mainfest is executed before any visualization will render.

78

@Brandon_Kobel

I understand well this is a bug but my question is why the reporting function by x-pack will trigger it. Others Kibana features like discover, visualization and daskboard work correctly to me except for reporting function. Compared with other features, what does reporting do more ?

79

@Apache_HOU Reporting runs a headless browser to create the report, so if your firewall on the Kibana server is configured to drop packets it's triggering the issue. Only Reporting is using the headless browser on the Kibana server.

80

@Brandon_Kobel

Ok, it's more clear for me, thanks.

81

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.