Will the filter cache become a bad thing in some situation?

For example, if you have one thousand query. There are 1000 filter which keep in memory and being updated periodically.

Will it be a big cost?

For the most part keeping the filter cache doesn't come up as an issue. I've seen issues where people want more control over what gets cached on this forum, but not seen issues with overhead.