Getting list of valid filters/facets with search results

I suspect any irrelevant facets you are seeing are a side effect of the fuzzy matching approach used by default.
See this comment on making the query less fuzzy to increase precision. This will help remove irrelevant facets.