Just to follow this up, I managed to trace it down through doing debug logging, between 5 and 6 it seems the config.automatic.reload setting changed from being in seconds to microseconds if it hasn't got a time unit suffix and I had it set to 3 so logstash was constantly reloading the configuration causing the high CPU usage.
I've changed the setting from 3 to 3s and it's dropped the logstash CPU usage from 100% to circa 10-15% which is where it was before.