Retrying failed action with response code: 429

I would remove these 4 lines and see what happens with the defaults.

16 workers is generally far more than necessary. I wouldn't make this setting more than 2 unless you're doing over 10,000 events per second. Flush size is also too big. Because of the retry logic (which is why you get 429 response codes), you should probably work in smaller batches (I believe the default is 512 now). The plain codec simply doesn't do anything here, as elasticsearch requires JSON.

I also note that you are separating your indices by app. How many "apps" do you have per day? How many indices do you have, total, on your cluster? What's your data retention policy? Are you using the default 5+1 shard count? I ask these questions because having too many shards on a single node can overload a node's index management ability. It only gets to use a percentage of the heap for this, and exhausting the memory creates pressure which can dramatically affect index caching (which might be what's resulting in more 429s).