The problem is in the http output of the "env_tcp" pipeline. If you go through your stats and look at the "events" entries the only one that has an imbalance is
"events": {
"in": 1983728,
"filtered": 1891560,
"out": 1891560,
....
"pipelines": {
....
"env_tcp": {
"events": {
"queue_push_duration_in_millis": 40873948,
"duration_in_millis": 151879287,
"filtered": 1713729,
"out": 1713729,
"in": 1805897
},
There are 30,000+ events going in and not coming out. Check Guy's post here. This is not a slow input.
"outputs": [ ....
{
"id": "c685636696572b6d74ea68ff726974823e22f806987cff4044310f1b953496a5",
"name": "http",
"events": {
"duration_in_millis": 151778979,
"out": 1713729,
"in": 1759809
}
}
]
},
Again, there are tens of thousands of event reaching the http output and not leaving. It is taking nearly a tenth of a second per event. It could be that encoding the events is really expensive, but I would concur with Leandro that the first place to look is the http destination.