Sorting issue on keyword

Hey @formiaczek,

Thanks for taking the time to write up your thoughts on this. I agree that the current behavior of sorting on first bucket is not intuitive, and in particularly confusing in some scenarios (especially stacked bar charts).

There are a few fundamental problems which makes this a non-trivial issue to solve:

First, there's the question of UX and what a user would reasonably expect (sorting each bucket on values of its subaggs vs sorting based on the entire set of data overall). I like your idea of making a user-selectable option for how the sorting is applied. This is similar to what someone suggests in the comments on the Github issue. Removing orderBy for subbuckets is also an interesting idea for a short-term solution, but my main concern with that would be that it's a very widespread change to address a very narrow issue... for example sorting alphabetically still works as expected because there is no bucket-to-bucket variance in the results.

Second, there's the technical issue of how we model the data before rendering the visualization. The current way the data are modeled simply doesn't preserve the sort order from Elasticsearch by the time it makes its way to a chart, as demonstrated here. Changing this to preserve sort order would be an update that affects all point series visualizations, so it is something that will require an extra level of planning and care when implemented, since it's not a one-off bug fix.

I'd encourage you to share any more thoughts you have on the Github issue (or just :+1: the original comment). That's the best way to make sure the right folks see it, and watching +1s helps us to best prioritize. In the meantime, I'll ping the team that works in this area to make sure it's still being monitored.

Cheers,

Luke