Filter visualisation not providing full terms list, autocomplete settings are not making any differences

[Luukv (Luuk Vlug)]

Hello,
In the past months I have made several filter/control panel visualisations for multiple dashboards. These have all worked well, but with the most recent one I've ran into some problems.

We were experiencing some crashes in Kibana, so decided to add our scripted fields to the index to save some performance. One of these fields is the month field, this is a general field that just shows in which month this dataset (/doc) was added. We obviously have all 12 months in this field, which works great. But when trying to add a dropdown filter on this field so users can select a month, it only shows 3 out of the 12 months. Typing something in the bar also does not result in any more suggestions. These 3 (january, march, april) are all the months it can find in this filter.

We tried the following:

• Disabling the dynamic search options and trying several different values ranging up to 9999. No changes.
• Increasing the autocomplete.Timeout setting in the .yml from 1000 --> 5000, no changes.
• Increasing the autocomplete.Terminate setting in the .yml from 100000 --> 200000, no changes.
• Disabling the autocomplete.Terminate setting in the .yml and leaving the autocomplete.Timeout on 5000, no changes.

The fact that these somewhat big changes to these settings are not even adding a single value gives me the indication I might be looking in the wrong place.

Any other suggestions on things we can try?

[mattkime (Matthew Kime)]

Hello @Luukv

Which version of the elastic stack are you running? Which license?

I'm sorry to hear that you're having trouble with browsers crashing. Which browser and version?

The dashboard controls will only show options available for the time range set in the search bar. Are you certain that your time range provides for all the values you expect to see?

Also, if you're experiencing performance issues and you're using scripted fields you might consider changing your documents so they provide the data as you wish to use it. This would be mean updating the ingest process and existing documents.

Thanks,
Matt

[Luukv (Luuk Vlug)]

Hello,

This post is focussed on the missing values in the control visualization, not so much the crash. I already made another post regarding the crash a while back and we managed to improve the crashes, so that's all good.

We are using version 7.8.0, in Chrome.

Now regarding the controls visualisation. It gives that triangle stating more values can be loaded with a longer duration before timeout of the autocomplete setting. However 5 seconds is already quite a long time, there is no terminateafter amount set, so that's also not the problem.

Another thing to note is that it only finds the values 'january', 'march', 'april'. I tried setting the time period to last 5 years, which also changed nothing. Our first docs in Kibana are from December 2015, so if it starts looking through the docs for available fields (during the 5 second duration timer we have set in the .yml file), should it not at least find December? And where is February for example.

This same problem occurs with 1 other field, which has locations (cities) stored. It only finds a certain amount of cities, which stayed the same. Even after changing the autocomplete settings in the .yml multiple times, refreshing the index, changing time periods (from last 15 minutes to last 5 years). I also tried disabling the dynamic options setting, and setting the size (/amount of values retrieved) to 9999.

And we have a few other dropdown filters in the same visualisation. These have hundreds of available values, as they're categories. All of these appear. Same with hour of day, day of week and week of year, all those values also show up and work properly. It's just the 2 fields, month and the city locations.

I did read something about these fields maybe being nested because the fields are .keyword, but all the fields are .keyword and besides the month and city, the other fields function properly.

Hope this information clarifies the situation a bit. It's a tough one as the problem/result has not changed once even though I've tried multiple possible solutions.

If you have any ideas, suggestions or links we can try/check out, please do share these. I have looked at quite a few forum posts, blogs and articles regarding similar problems, but there is always a possibility I might have missed something!

Edit: wanted to add that this problem both occurs when the visualisation is in a dashboard, or individually loaded. That does not change the outcome.

[mattkime (Matthew Kime)]

could you create a visualization and add it to the dashboard that displays the values you're looking for? Such as a tag cloud or similar? I think its worthwhile to verify that the expected values are actually being returned from the search.