after the new controls came out and replaced the beta controls, i started to add them to every dashboard i'm managing. There was no exists query possible at all back then.
Then a new update came out, introducing the exists query, which is a nice feature to have, but for some reason it was enabled by default on every control. Even the ones already defined. Which was really annoying, as i didn't need it in a single one.
But there was an option to disable it on a control level basis. So i went through some hundred controls and unchecked "allow exists query" on all of them.
So now i updated again, to kibana 8.7 and what do i see? It's back! On all of them!
Additionally the checkbox "allow exists query" is removed:
So i am really wondering why new kibana versions are released that change the behaviour of existing controls in a way that may be undesired.
How can i disable the exists query now? I hope i don't have to do it manually for every control again.
Even worse would be if it is just always there. The Dashboards are used by a lot of non english speaking users who aren't really tech affine and i can already see that this will cause confusion.
Please don't tell me i have to explain why suddenly there is a new entry in every control that we do not need at all!
I apologize for the bad experience you have had with this. A migration should have happened that would make this change not impact scenarios like yours: https://github.com/elastic/kibana/pull/147216
It looks like we have missed a bug, I will investigate locally and create an issue for it.
While the UX for the removed toggles will be removed, the logic is kept - this means that, for the Control Group API that solutions will be using, this functionality could still be exposed to the consumers.
sounds like it should still be possible to disable the exists query. Can you explain how?
@Jonas_S, I completely recognize that this is a frustrating change. The new Controls have been a little bit "under construction" for the 8.x timeframe, so adding and changing of features is to be expected. This change was intentional, and not a bug. We removed the toggle, and restored the value of allowExistsQuery as part of an initiative to simplify the experience of authoring Controls, but I recognize that having the "exists" option there can add complexity to their usage.
When removing the toggle, we chose to reset allowExistsQuery back to true, because without a toggle, any user that had turned it off would be unable to turn it back on.
Right now there is no longer a way to hide the exists setting from within the Kibana UI, but there should be a way to edit the Dashboard saved objects to change these settings. If you'd like I can work on a script that can do it.
I can't see how taking away options from the dashboard authors is streamlining the process. The author should be the one with the knowledge and should be given as much customization options as possible! I think you are heading in the wrong direction when you take away options from the authors.
On the other hand, the user of the dashboard are the ones which should see a "streamlined" dashboard. They should be the ones protected from unnecessary information. I think we don't need to debate this: From a data visualization standpoint, less is more. No unnecessary information is always the way to go.
But with the new version the user now always sees the exists and i already got a complaint why there is suddenly a number behind the entries in the control, which shows in how many documents the value is present. Our users don't even know what "document" means. You have to think of the older generation which are now confronted with new technology.
So i, as an author, want to control which information they should see and which they should not see. Instead, options are taken away from me.
This discussion set aside: If you can provide me with instructions (or a script if you want) that gives me control over enabling or disabling the exists query, then yes, i'd like to have it.
Edit: Also, i have more control related issues in 8.7, so maybe you know more about that as well:
Hi @Jonas_S you're very correct that author-flexibility is important for these controls. They are meant to provide authors with a system that can simplify the experience for analysts using the dashboard. We will look into restoring this setting - perhaps at the Dashboard-level so you wouldn't have to turn it on or off for individual controls.
In the meantime, I've developed a script which you can run against your cluster to hide 'exists' in all dashboards. This script might be a little complex to set up because it requires Node, Yarn and a special user role, but once you've got it running it should be pretty quick.
We'll continue the conversation about the expensive queries setting on that thread, but that setting should be on by default. We'll try to diagnose this with you.
@devon.thomson That sounds good, but i'm not sure if a dashboard-level setting is enough. What if you have a single field out of 20 where you want to have the exists query?
So if you ask me, i would like to have a dashboard-level switch plus a control-level switch and have the dashboard-level switch control all control-level switches, but not the other way around. So you could first make the desicion if you mainly want it or not, and then go into the individual controls for fine tuning.
You do currently have to go to each control to set the default sort - after all, we support fields that don't support alphabetical sorting (for example, IP fields) and we have a range slider control that don't support the concept of sorting at all , so having it be a control group setting would be inconsistent and confusing.
But i think it would be great to have this as a dashboard-level setting which sets it for all controls where it is applicaple and after that you can go to the individual controls for finetuning.
@Jonas_S, we do use the inheritance pattern you describe elsewhere, so if we were going to reintroduce these settings it would likely work in a similar way.
To track it I've opened this issue and have prioritized it into our next phase of the Controls project.
The link was broken because the repo was private, I've made it public, and the link should work properly now.
Let me know if you have any questions about running the script.
You're entering the correct format for the endpoint. it looks like it's a security configuration on your on prem cluster that's causing this issue. I ran this against a 8.7.0 cloud cluster and had no issues.
It seems like the error might be TLS related but I don't know enough about security to really help. If you have access to the yml settings, you could try temporarily setting elasticsearch.ssl.verificationMode: certificate.
@devon.thomson Thank you anyway. I already have an assumption but it has to do with the local setup. Maybe i can fix it.
And otherwise i hope the elastic team changed it's mind about author control and the options will be back in a future release.
I will keep track of the issues.
Hi @Jonas_S, sorry for the late reply, I've just returned from vacation.
I'm glad you were able to get around the security issue! This new one looks like I missed something when building the script. You've got a dashboard with no controlGroupInput at all and we need to handle that case. I've modified the script to fix this and pushed it to the repo so you can download it.
Hopefully this works for you, but if you run into any other problems I'll be around!
Thank you again @devon.thomson, now it worked! All Exists are gone.
Hope it will stay like that after upgrading to new versions soon.
But thank you for the great support!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
