# Pipeline stats API: in/out/filtered

**URL:** <https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742>\
**Category:** Logstash\
**Tags:** elastic-stack-monitoring\
**Created:** [January 10, 2019, 1:29pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742 "2019-01-10T13:29:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmb/32/14312_2.png) [@jmb](https://discuss.elastic.co/u/jmb)\
**Post date:** [January 10, 2019, 1:29pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742/1 "2019-01-10T13:29:40Z")

</div>

What do the `in`/`out`/`filtered` values in the `events` section of the [pipeline stats API](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats) actually represent?

I had understood them to mean

- `in`: total events entering the pipeline (sum of input plugins production)
- `out`: total events successfully processed by output plugins
- `filtered`: how many events made it through the filter stages (or possibly, how many were filtered _out_)

This doesn't seem to be the case - as far as I can see, all three values are identical. We have been recording these metrics to Graphite, and see no difference between in/out in any historical data. I was hoping to use this to flag any unexpected lossage in the pipelines.

Here's the stats output for a sample run using the `generator` plugin, showing the output plugin receiving zero events, but the `out` metric still showing 100:

```
"pipeline-metrics-test" : {
      "events" : {
        "duration_in_millis" : 62,
        "in" : 100,
        "out" : 100,
        "filtered" : 100,
        "queue_push_duration_in_millis" : 0
      },
      "plugins" : {
        "inputs" : [ {
          "id" : "random-generator-100",
          "events" : {
            "out" : 100,
            "queue_push_duration_in_millis" : 0
          },
          "name" : "generator"
        } ],
        "filters" : [ {
          "id" : "...",
          "events" : {
            "duration_in_millis" : 1,
            "in" : 100,
            "out" : 0
          },
          "name" : "drop"
        } ],
        "outputs" : [ {
          "id" : "...",
          "events" : {
            "duration_in_millis" : 0,
            "in" : 0,
            "out" : 0
          },
          "name" : "stdout"
        } ]
      },
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 10, 2019, 2:28pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742/2 "2019-01-10T14:28:53Z")

</div>

I agree that these numbers do not represent what I [expect](https://discuss.elastic.co/t/event-emitted-vs-events-received/140601) them to represent.

---

<div class="post-metadata">

**Author:** ![jmb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jmb/32/14312_2.png) [@jmb](https://discuss.elastic.co/u/jmb)\
**Post date:** [January 16, 2019, 12:56pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742/3 "2019-01-16T12:56:57Z")

</div>

Is this the same as [https://github.com/elastic/logstash/issues/8752](https://github.com/elastic/logstash/issues/8752) ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 16, 2019, 1:16pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742/4 "2019-01-16T13:16:40Z")

</div>

Not certain, but I think so.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2019, 1:16pm UTC](https://discuss.elastic.co/t/pipeline-stats-api-in-out-filtered/163742/5 "2019-02-13T13:16:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.