I'm not sure if cache clearing works with "all" indices implicitly. You may have to use the
_all special identifier or a wildcard, e.g.:
That said... why do you want to clear the cache for testing? It won't give you an accurate understanding of the performance of the system, since Elasticsearch cache things by default. A better way to test is to use a realistic production workload (real documents, a variety of real queries, etc) so that caching effects are spread across the data and you get a more accurate picture.
Also, there is OS level caching which you can't control. That may be what you're running into anyway. E.g. the operating system will see that you are accessing the same files (lucene segments) repeatedly and cache them in memory for fast access. That could be why you see a fast response even after restarting elasticsearch... the OS has cached the relevant files.
Benchmarking is usually about using a realistic workload so that caching is applied like the real world, rather than trying to fight and eliminate caching for a "pure" benchmark.