Resolved the issue, it was a problem with jboss/wildfly. In wildfly's default docker image, the standalone configuration sets the stdout to have a colored pattern for the terminal.
Since the the app was running in docker, and it's console stdout was being scooped up by logspout and then sent to logstash, it was capturing that extra colored formatting information as special characters in the log message.
For posterity: anyone out there running wildfly in docker and collecting logs in logspout to send to logstash-- you need to change your wildfly configuration and set the console's named formatter to just "PATTERN" . You need to make a custom standalone xml configuration for your wildfly docker image to do that.
See here on how to customize your wildfly image:
https://goldmann.pl/blog/2014/07/23/customizing-the-configuration-of-the-wildfly-docker-image/
This is the setting you'll want to change in your custom standalone xml doc:
https://docs.jboss.org/author/display/WFLY8/Handlers#Handlers-namedformatter