Took a quick look at the stacktrace but the vast majority of threads belong to cassandra or the cassandra connector. That's not to say that Cassandra is to blame here and/or the Elasticsearch connector has no impact rather it's hard to understand what is causing the JVM to eat all the CPU since all these apps look like they are running within the same VM.
Can you try potentially run each app in a separate VM; this will help isolate the hungry CPU process. Further more, while writing data to Elastic, can you run the
hot threads API to see how Elasticsearch behaves?
Further more, you can try using Marvel or other monitoring plugins to understand while indexing, what's the impact on Elastic.
Where does the data in Elastic comes from? Can you minimize your example and eliminate for example Cassandra and index only to Elastic. Further more, can you first index the data from HDFS or the file-system and then add Kafka and see whether it makes a difference?
There are a lot of moving parts and it's unclear whether it's a certain component that eats the CPU or whether it's their interaction that it's causing the issue...