So it turns out you can set:
dead_letter_queue.enable: true
path.dead_letter_queue: /opt/logstash/dead_letter_queue
That will give you the messages that are cause 400 or 404 error while indexing.
(the path is not required, if not set it defaults to LOGSTASH_HOME/dead_letter_queue/main)