OK, for any other poor sods who came here trying to do the same thing, here's how I got it working:
First, you need to find the filebeat-[version]-nginx-access-pipeline and Clone it.
Name it something like: filebeat-[version]-awscloudwatch-nginx-access-pipeline.
When editing the clone, change the failure processors to remove the Set processor and add a Pipeline processor targeting the filebeat-[version]-nginx-error-pipeline. I didn't put any conditions in, but you could, if you're confident with how that works.
Next, update your filebeat config to include:
setup.template.settings:
...
index.final_pipeline: "filebeat-[version]-awscloudwatch-nginx-access-pipeline"
Now, if you're like me, and you're working with an existing filebeat index template that you now can't edit, or your pushing stuff in from filebeats across your arch and you don't want to risk any nasty side-effects, you can configure a new index template like so:
setup.template.name: filebeat-awscloudwatch-nginx
setup.template.pattern: filebeat-awscloudwatch-nginx-*
setup.ilm.rollover_alias: filebeat-awscloudwatch-nginx
...
output.elasticsearch:
...
index: "filebeat-awscloudwatch-nginx-%{+yyyy.MM.dd}"
Now, when cloudwatch logs get indexed into that index by filebeat's awscloudwatch module, they'll go through the filebeat-[version]-awscloudwatch-nginx-access-pipeline first, when the grok fails, it'll send it onto the filebeat-[version]-awscloudwatch-nginx-error-pipeline and you'll get all the lovely enrichments that you always wanted.
YOU ARE WELCOME
