In Kibana 4.3, using server.basePath is really useful to reverse proxy Kibana inside another website. However, when the base path is changed (from the default), the first time Kibana is started it will rebuild the static bundle files with a message like "Optimizing and caching bundles for kibana and statusPage. This may take a few minutes".
This is actually taking about 30 seconds in my server and raises the memory usage to 600MB. I'm using Docker containers using the official Kibana image.
I'm wondering how can I avoid such dynamic bundle compilation, and deliver a Docker image with the proper static files already built. Is there any way to force the compilation of these files without starting the server?
There's no way to disable it, that build step is required.
That said, once you run it once, it shouldn't run again. As in, if you stop and restart Kibana, that step shouldn't happen - so you could ship your container once you run it the first time.
Maybe I didn't express myself correctly. I don't want to disable that build step. Instead I want to have control on when the build is done.
Kibana normally doesn't require more than 200MB to run in my case, and it's a pity that I need to assign around 600MB just to run a one time task. In a real world example, the new AWS t2.nano instances looks perfect to run Kibana servers but they are limited to 512MB.
Another part of the story, Kibana still have some memory leaks that I probably should research and ask on a separate question, but you can try yourself running the official Kibana docker image and doing just nothing the memory used goes up and up until it uses all the server memory. I let Kibana running on one of my servers during the weekend and at the end the OS was swaping memory in and out continuously. To mitigate this problem, I limited the docker container to 200MB. Docker will restart the container one the limit is reached, but this doesn't work if Kibana has to rebuild the asset bundles.
An ideal solution to me would be to have a command I can execute during the docker build, that forces the asset bundles to be rebuilt. In that way I don't need to use the server resources to execute this task.
If you go chasing down memory leaks, please do open an issue on github. We're working on fixing the browser side issues, but we haven't been able to confirm any server side yet, and there's currently no open issues about it.
Instead I want to have control on when the build is done.
That's not something we can offer, it has to happen with changes since the bundle needs to be current. Of course, if you don't make any changes, the build step does not re-run.
Docker will restart the container one the limit is reached, but this doesn't work if Kibana has to rebuild the asset bundles.
It should not ever run that build step again. The first time you change the basePath setting and run Kibana, it will do a rebuild, because it has to set up that change across the entire app. After that happens though, the app will just run when you start it again. The exception would be if you make other changes, then it'll have to rebuild again.
So, after you change that setting and run it once (in a larger container, temporarily), you're done. If you continue to work from that snapshot of your container, it shouldn't run the build step again, so you can use a smaller memory container from then on.
That's not something we can offer, it has to happen with changes since the bundle needs to be current.
Making sure that the bundle is up to date is something that I can be responsible of. In production servers, if the bundle is out of date, an error message is acceptable. This is actually quite similar to Rails application, where in production mode (with the default settings) you must run rake assets:precompile to get the bundles built.
When you say it's done once, keep in mind it's done once per server. Also this could be considered a build step that should not be run in the production server. In other words, you don't want to use the production server resources (memory and cpu) to execute some task that could be perfectly done in preparation scripts during deployment.
Going even further, I wish the server were running without unnecessary writing privileges to the disk. That's just another reason to build the bundles before deploying to production.
Ah, yeah, that's valid. You can build the bundle on another machine before you deploy it, but it sounds like maybe that's not good enough for what you're trying to do here.
I'd say open an issue on github about it, it's possible that others are running in to the same problem deploying kibana as well. Feel free to link back to this discussion too.
I'd like to pre-optimize the bundles (with a changed basePath) before I start deploying the code.
Is there any way I can run the optimize task standalone? So that the command used returns instead of hanging on the running server? (So that I can script the building of the package to be deployed.)
Basically I'd like to be able to do something like this: # <some-command> log [13:22:45.273] [info][optimize] Optimizing and caching bundles for kibana and statusPage. This may take a few minutes log [13:24:22.372] [info][optimize] Optimization of bundles for kibana and statusPage complete in 97.09 seconds log [13:24:22.417] [info][status][plugin:kibana] Status changed from uninitialized to green - Ready # <- back at prompt
There is not currently a way to do this, but it's something that already happens when you use the plugin installer, so adding a command line option for that should be pretty easy.
I don't currently see an issue to track this in the Kibana issue tracker. Would you mind opening one?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.