# How to config esrally metric store so that "meta" of rally-metrics-\* can record "took" of each query

**URL:** <https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [November 24, 2021, 8:07am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024 "2021-11-24T08:07:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![kingswan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kingswan/32/139263_2.png) [@kingswan](https://discuss.elastic.co/u/kingswan)\
**Post date:** [November 24, 2021, 8:07am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/1 "2021-11-24T08:07:21Z")

</div>

I use metric store to save esrally results. Some of my results, the "meta" includes "took" of each query:

```auto
"meta": {
           "plugins": [
             "test_plugin"
           ],
           "attribute_xpack.installed": "true",
           "source_revision": "053779d",
           "distribution_version": "6.3.2",
           "distribution_flavor": "default",
           "index": "index_test",
           "took": 129,
           "success": true,
           "success-count": 50,
           "error-count": 0
},

```

However, some results are not, like this:

```auto
"meta" : {
            "plugins" : [
              "PLUGIN_TEST"
            ],
            "attribute_ml.machine_memory" : "33651015680",
            "attribute_xpack.installed" : "true",
            "attribute_ml.max_open_jobs" : "20",
            "attribute_ml.enabled" : "true",
            "source_revision" : "8453f77",
            "distribution_version" : "6.7.0",
            "distribution_flavor" : "default",
            "success" : true
          },

```

Which setting that I can configure for "took" ?

* * *

#Update 1  
Seems only "took" of bulk operation can be recorded.  
Is that possible to save "took" of query operation in metric store?  
Thanks!

---

<div class="post-metadata">

**Author:** ![json](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/json/32/4125_2.png) [@json](https://discuss.elastic.co/u/json)\
**Post date:** [November 24, 2021, 10:03pm UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/2 "2021-11-24T22:03:45Z")

</div>

Hi @kingswan, for `search` operations, have a look at setting `detailed-results` to `true` to return `took` time in meta-data.

[https://esrally.readthedocs.io/en/stable/track.html#id8](https://esrally.readthedocs.io/en/stable/track.html#id8)

---

<div class="post-metadata">

**Author:** ![kingswan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kingswan/32/139263_2.png) [@kingswan](https://discuss.elastic.co/u/kingswan)\
**Post date:** [November 25, 2021, 2:33am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/3 "2021-11-25T02:33:44Z")

</div>

Hi Jason,  
Thanks for your help!  
I changed my `search` operation with `detailed-results ` in this way:

```auto
{
      "name": "randomized-nested-queries",
      "operation-type": "search",
      "detailed-results": true,
      "param-source": "nested-query-source"
}

```

here is my challenge:

```auto
{
    "name": "queries-tp002",
    "description": "vector-plugin-p15w-nested-queries-tp2",
    "default": false,
    "schedule": [
      {
          "operation": "randomized-nested-queries",
          "clients": 2,
          "target-throughput": 2,
          "warmup-iterations": 5,
          "iterations": 10
      }
    ]
},

```

here is my track.json:

```auto
  "operations": [
    {{ rally.collect(parts="operations/*.json") }}
  ],
  "challenges": [
    {{ rally.collect(parts="challenges/*.json") }}
  ]

```

and this is the way I start a rally:

```auto
esrally race --track-path=track.json --challenge=queries-tp002 --target-hosts=remoteIP:9201 --pipeline=benchmark-only

```

However, my meta-data still not contains `took`:

```auto
"meta" : {
            "plugins" : [
              "GWSSI-KNN-PLUGIN"
            ],
            "attribute_ml.machine_memory" : "33651015680",
            "attribute_xpack.installed" : "true",
            "attribute_ml.max_open_jobs" : "20",
            "attribute_ml.enabled" : "true",
            "source_revision" : "8453f77",
            "distribution_version" : "6.7.0",
            "distribution_flavor" : "default",
            "success" : true
          },

```

is that relate to my plugin or maybe the cache?  
p.s. I use rally 2.3.0, my cluster version is es6.7.0, my metric store version is es7.4.0  
Best,  
Chester

---

<div class="post-metadata">

**Author:** ![Quentin\_Pradet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quentin_pradet/32/94192_2.png) [@Quentin\_Pradet](https://discuss.elastic.co/u/Quentin_Pradet)\
**Post date:** [November 25, 2021, 1:36pm UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/4 "2021-11-25T13:36:07Z")

</div>

I can reproduce the issue: while `detailed-results` works with normal searches, it does not have any effect with custom parameter sources. I opened [Custom parameter sources do not honor detailed-results · Issue #1390 · elastic/rally · GitHub](https://github.com/elastic/rally/issues/1390) to track this.

---

<div class="post-metadata">

**Author:** ![kingswan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kingswan/32/139263_2.png) [@kingswan](https://discuss.elastic.co/u/kingswan)\
**Post date:** [November 25, 2021, 1:47pm UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/5 "2021-11-25T13:47:24Z")

</div>

Hi Quentin, thanks for the feedback. Hope it can be fixed soon.

---

<div class="post-metadata">

**Author:** ![Quentin\_Pradet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quentin_pradet/32/94192_2.png) [@Quentin\_Pradet](https://discuss.elastic.co/u/Quentin_Pradet)\
**Post date:** [November 25, 2021, 4:48pm UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/6 "2021-11-25T16:48:40Z")

</div>

@kingswan Can you please try setting `detailed-results` to True in your customer parameter source code in track.py? I confirmed that works.

---

<div class="post-metadata">

**Author:** ![kingswan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kingswan/32/139263_2.png) [@kingswan](https://discuss.elastic.co/u/kingswan)\
**Post date:** [November 26, 2021, 2:33am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/7 "2021-11-26T02:33:21Z")

</div>

Yes, it works! Really appreciate for your help.

---

<div class="post-metadata">

**Author:** ![Quentin\_Pradet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quentin_pradet/32/94192_2.png) [@Quentin\_Pradet](https://discuss.elastic.co/u/Quentin_Pradet)\
**Post date:** [November 26, 2021, 3:02am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/8 "2021-11-26T03:02:32Z")

</div>

And I'm thankful for the help of @danielmitterdorfer who told me the solution in that GitHub issue. 🙂

Glad we could help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2021, 3:03am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024/9 "2021-12-24T03:03:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.