Just in case this helps anybody.
I was seeing really nice elastic search response times and went to
bring it into production. I wrote a /etc/init.d/es script and
restarted with it. I was (accidentally) then running elasticsearch as
root with an fd limit of 1024. The log file was spewing with 'too
many files'. When I fixed that to use the user elasticsearch who has
a high fd limit, things started working beautifully again.