Automated tests for Kibana UI

Elastic Stack Kibana
1

Sup, guys. We are using Kibana in production to display several dashboards and we want to apply automated UI-testing for Kibana and I'm just wondering what tools are used by Elastic team to implement that kind of testing when they develop new functionality for Kibana in their company. Does anybody know what tools they use for that? I want to make UI testing for Kibana dasboards. Using tools like Selenium and testing through XPath/ID/Class name/Javascript is not a good idea, because Kibana significantly changes every new version. Please, maybe you can recommend some tools that are suitable for my purpose?

2

Hi Koomar,
I'm a new QA person on the Kibana team and I'm working on UI test automation. We're using a stack of Intern/Leadfoot/WebDriverJS. So this allows local testing with a standalone selenium server or remote testing on BrowserStack or the like.
I'm initially working on automation for happy path tests taking some of the default values on Settings, Discover, and Visualization tabs (haven't touched Dashboard yet). We're getting this integrated into our CI build system.
I'm using the page object model, so all those Xpaths and CssSelectors, etc. are localized for each tab. I know they're not ideal and likely to break when Kibana goes through significant UI changes, so I'm also looking to add ids to the objects where possible.
This new code still needs some reviews before it's ready for public consumption but that shouldn't be long.

3

Thanks for answering! This info is very useful to me.

4

Hi Koomar,
Based on the CONTRIBUTING document, I simply do the following commands for the 4.2.1 release of Kibana:
wget https://github.com/elastic/kibana/archive/v4.2.1.tar.gz
tar xvf v4.2.1.tar.gz
node --version
v0.12.7
npm --version
2.11.3

cd kibana-4.2.1
npm install (I get a bunch of 'unmet dependency' warnings but no errors)
npm run test:server

I get errors with the npm run test:server Do I have to install a selenium server as you mention in your message? I am using Ubuntu 14.04 in a VM.

5

Hi BlondAngel,
If you're trying to run the automated UI tests you can do either of these;
1). run 'npm run test:ui:server' in one window (this downloads esvm and selenium and starts them)
and run 'npm run test:ui:runner' in another window (this runs the tests)

2). run 'npm run test:ui' (does all of the above in one session)

If you just run 'npm run test' it should run all the unit and UI tests.

6

Turns out when I use a VM and it is text-based -- ie it doesn't have a GUI, then the tests will not work well. Once I switched to an Ubuntu with a desktop GUI, then the tests work well. For example 'npm run test:dev' will launch a firebox browser and you can launch the tests from there. (this is with Kibana4)

Two questions:
How do I know that the unit testing covers all of the functionality? In other words, how complete is it? I could imagine that a person could add a new function but not add a unit test for the new function, right?

Are there any unit/functional testing for Kibana3? If no, any suggestions in my first attempt to write up unit testing for Kibana3? I doubt that I could mimic Kibana4 unit testing in Kibana3 since both are fundamentally different from each other.

7

  1. We don't currently have automated unit test coverage reports for Kibana 4.x. It's something we all want to get in place. The developers are pretty strict about requiring unit tests on all new and changed code so I would expect the coverage is actually pretty high. The automated UI tests would add a bit more coverage but it would be a pretty small percentage at this point.

  2. Kibana 3.x went end-of-life last month. I think this is the github repository for it https://github.com/kibana-community/kibana3 and I see some tests here kibana3/src/vendor/bootstrap/less/tests/.