# \[ERROR\] Total Ingest Pipeline Count/Time/Failed Metrics are all returning 0 **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [June 13, 2022, 7:52pm UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069 "2022-06-13T19:52:43Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![raphaelli1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raphaelli1/32/106733_2.png) [@raphaelli1](https://discuss.elastic.co/u/raphaelli1)\ **Post date:** [June 13, 2022, 7:52pm UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069/1 "2022-06-13T19:52:43Z")
Hi, it seems that whenever we get the final score for ingest pipeline count/time/failed metrics all return 0 when we run esrally race. The command we are running is: `esrally race --track=http_logs --target-hosts=http://xxx.xxx.xxx.xxx:xxxx/ --pipeline=benchmark-only --report-file=es_report_E4_32_actualgb.md --report-format=csv --kill-running-processes` The x's represent the IP address. The program finishes and returns results! But we observe a: ```auto Total Ingest Pipeline count,,0, Total Ingest Pipeline time,,0,s Total Ingest Pipeline failed,,0, ``` In the results file. We have done this on percolator, http-logs and a custom track of our own but everytime the result returns 0 for all 3 even though we know ingestion must be occuring. How do we fix tihs? ---
**Author:** ![Bradley\_Deam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley_deam/32/52991_2.png) [@Bradley\_Deam](https://discuss.elastic.co/u/Bradley_Deam)\ **Post date:** [June 14, 2022, 1:05am UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069/2 "2022-06-14T01:05:04Z")
Hi @raphaelli1, thanks for using Rally 🙂 These metrics are specifically related to the usage of the '[Ingest Pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html)' feature, see also the [`put-pipeline` operation](https://esrally.readthedocs.io/en/stable/track.html?highlight=ingest-pipeline#put-pipeline), and [`bulk` operation's `pipeline` parameter](https://esrally.readthedocs.io/en/stable/track.html?highlight=ingest-pipeline#bulk). If no Ingest Pipelines are used by the indexing/bulk operations, then these stats are expected to be `0`. To answer your track specific questions; - `percolator` does not use any ingest pipelines, - `http_logs` _can_ use Ingest Pipelines, but you must specify the track param `ingest_pipeline` ([see README here](https://github.com/elastic/rally-tracks/tree/master/http_logs#parameters)) - example invocation: ```auto esrally race --track=http_logs --target-hosts=http://xxx.xxx.xxx.xxx:xxxx/ --pipeline=benchmark-only --challenge="append-index-only-with-ingest-pipeline" --track-params="ingest_pipeline:baseline" --report-file=es_report_E4_32_actualgb.md --report-format=csv --kill-running-processes ``` - I can't comment on your own custom track, but if you're not using any ingest pipelines then seeing these values as `0` is expected. Thanks! ---
**Author:** ![raphaelli1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raphaelli1/32/106733_2.png) [@raphaelli1](https://discuss.elastic.co/u/raphaelli1)\ **Post date:** [June 14, 2022, 3:30pm UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069/3 "2022-06-14T15:30:16Z")
Hi, I tried the command you said; however: > Total Ingest Pipeline count,,0, > > Total Ingest Pipeline time,,0,s > > Total Ingest Pipeline failed,,0, I also got this error: > [WARNING] Error rate is 100.0 for operation 'index-append-with-ingest-baseline-pipeline'. Please check the logs. > > [WARNING] No throughput metrics available for [index-append-with-ingest-baseline-pipeline]. Likely cause: Error rate is 100.0%. Please check the logs. But when I saw the logs the only thing that was being printed out was: > 2022-06-14 05:59:40,164 -not-actor-/PID:9105 elasticsearch WARNING POST [http://xxx.xxx.xxx.xxx](http://xxx.xxx.xxx.xxx):xxxx/\_bulk?pipeline=http-log-baseline-pipeline [status:500 request:0.553s] This seems related to throughput metrics but ingestion metrics should still be there if only throughput metrics are not available. ---
**Author:** ![Bradley\_Deam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley_deam/32/52991_2.png) [@Bradley\_Deam](https://discuss.elastic.co/u/Bradley_Deam)\ **Post date:** [June 15, 2022, 2:53am UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069/4 "2022-06-15T02:53:43Z")
`status:500` typically indicates that the Elasticsearch server is erroring out. Given that you're already running an Elasticsearch server (`--pipeline=benchmark-only`), can you please check that ingest pipelines etc. are supported on that version? If you want to extract the full error details per bulk request, you'll need do one of the following: - Edit the track to set the `bulk` operation parameter `detailed: true`, e.g. and check the logs ```auto "operation": { "operation-type": "bulk", "detailed-results": true } ``` - Set the Elasticsearch client logger to `DEBUG` via editing the following section in `~/.rally/logging.json` and check the logs for failed requests (this will be noisy): ```auto [...] "loggers": { "elasticsearch": { "handlers": ["rally_log_handler"], "level": "DEBUG", "propagate": false }, [...] ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 13, 2022, 2:54am UTC](https://discuss.elastic.co/t/error-total-ingest-pipeline-count-time-failed-metrics-are-all-returning-0/307069/5 "2022-07-13T02:54:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.