Performance issues with search

I have a 2.4 ES cluster. Short story is that it's been humming along fine but lately I have an issue.

500 Nodes
100 Shards with 1 replica. Sometimes the query time goes up to 250ms - it's usually around 15ms. Processor usage is around 50% - what can I go look at?