# How to customize the summary report?

**URL:** <https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [September 27, 2018, 7:25am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134 "2018-09-27T07:25:26Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [September 27, 2018, 7:25am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/1 "2018-09-27T07:25:26Z")

</div>

now, I have a pressure measurement scenarios with a set of tests, and each test (with different variable) should be run once to completion, one after another. So, the `trial-timestamp` of each test is different.when I analyse the summary report in kibana,I want to put all tests together at the same time point.

Can I add a key-value pari into the summary report,like `round_time: 20180927T070012Z` or Is there any other way to solve it ?

---

<div class="post-metadata">

**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\
**Post date:** [September 27, 2018, 7:29am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/2 "2018-09-27T07:29:55Z")

</div>

You can use the [--user-tag](https://esrally.readthedocs.io/en/stable/metrics.html?highlight=track%20params#meta) option to inject as many comma separated key value pairs.

e.g. if you specify `--user-tag="round_time:2222"` this will be available in every doc as `meta.round_time`.

---

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [September 27, 2018, 7:38am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/3 "2018-09-27T07:38:24Z")

</div>

If I don not misunderstand, `meta.round_time` will appear in the index `rally-metrics-*` ?  
But now in kibana, the index I need analyse is `rally-results-*`

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 27, 2018, 8:08am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/4 "2018-09-27T08:08:03Z")

</div>

The summary results are not very flexible as far as I know, so I usually analyse the raw metrics in Kibana. This is where the user tags are useful. I you want to compare different sections or runs over time, measured from the start, you should look at the `relative-time` field in the Rally metrics.

---

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [September 27, 2018, 8:41am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/5 "2018-09-27T08:41:35Z")

</div>

👌 Thanks.

hope that we can customize some values in summary report in the future.

---

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [September 28, 2018, 8:23am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/6 "2018-09-28T08:23:22Z")

</div>

Hi,

I found that the index `rally-metrics-*` does not include metric `error_rate`, but in my test scenario, I need focus on this metric.  
So, If I `git clone https://github.com/elastic/rally.git` and then add some code in `./esrally/reporter.py` like:

```auto
import os

class Stats:
    def __init__ (self, d=None):
        ...

        self.segment_count = self.v(d, "segment_count")

        self.round_time = os.environ['RALLY_ROUND_TIME']

```

the index `rally-results-*` will include `round_time: xxx` ?

If it works, I want to konw how to compile and install Rally with source code now ?

Thanks.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 28, 2018, 8:30am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/7 "2018-09-28T08:30:10Z")

</div>

The `rally-metrics-*` indices contain a couple of documents per operation, and there should be a field on it that indicates whether the operation was successful or not. You should be able to use this to report on the error rate.

---

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [September 30, 2018, 2:33am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/8 "2018-09-30T02:33:05Z")

</div>

Hi,

I use the VirtualEnv to install Rally and the version is `1.0.0`, the source code locates at `./lib/python3.4/site-packages/esrally`.

In the `./metrics.py`, I found the code that index the result to ES, So I set a breaking point:

```auto
class EsResultsStore:
    """
    Stores the results of a race in a format that is better suited for reporting with Kibana.
    """
    INDEX_PREFIX = "rally-results-"
    RESULTS_DOC_TYPE = "results"

    def __init__ (self, cfg, client_factory_class=EsClientFactory, index_template_provider_class=IndexTemplateProvider):
        """
        Creates a new results store.

        :param cfg: The config object. Mandatory.
        :param client_factory_class: This parameter is optional and needed for testing.
        :param index_template_provider_class: This parameter is optional and needed for testing.
        """
        self.cfg = cfg
        self.trial_timestamp = cfg.opts("system", "time.start")
        self.client = client_factory_class(cfg).create()
        self.index_template_provider = index_template_provider_class(cfg)

    def store_results(self, race):
        # always update the mapping to the latest version
        import pdb
        pdb.set_trtace()

        self.client.put_template("rally-results", self.index_template_provider.results_template())
        self.client.bulk_index(index=self.index_name(), doc_type=EsResultsStore.RESULTS_DOC_TYPE, items=race.to_result_dicts())

    def index_name(self):
        return "%s%04d-%02d" % (EsResultsStore.INDEX_PREFIX, self.trial_timestamp.year, self.trial_timestamp.month)

```

it does not reach this step, but the summary report has been indexed into ES.

And then,In the `./racecontrol.py`, I found that you use the `godaddy/Thespian` to achieve the actor model. I also set a breaking point:

```auto
def race(cfg, sources=False, build=False, distribution=False, external=False, docker=False):
    logger = logging.getLogger( __name__ )
    # at this point an actor system has to run and we should only join
    actor_system = actor.bootstrap_actor_system(try_join=True)
    benchmark_actor = actor_system.createActor(BenchmarkActor, targetActorRequirements={"coordinator": True})
    try:
        import pdb
        pdb.set_trtace()

        result = actor_system.ask(benchmark_actor, Setup(cfg, sources, build, distribution, external, docker))
        if isinstance(result, Success):
            logger.info("Benchmark has finished successfully.")
        # may happen if one of the load generators has detected that the user has cancelled the benchmark.
        elif isinstance(result, actor.BenchmarkCancelled):
            logger.info("User has cancelled the benchmark (detected by actor).")
        elif isinstance(result, actor.BenchmarkFailure):
            logger.error("A benchmark failure has occurred")
            raise exceptions.RallyError(result.message, result.cause)
        else:
            raise exceptions.RallyError("Got an unexpected result during benchmarking: [%s]." % str(result))
    except KeyboardInterrupt:
        logger.info("User has cancelled the benchmark (detected by race control).")
        # notify the coordinator so it can properly handle this state. Do it blocking so we don't have a race between this message
        # and the actor exit request.
        actor_system.ask(benchmark_actor, actor.BenchmarkCancelled())
    finally:
        logger.info("Telling benchmark actor to exit.")
        actor_system.tell(benchmark_actor, thespian.actors.ActorExitRequest())

```

when I input `n` in debug model, it just run and then return success!!

Can you tell me what happened? Why it seems to do not run the code that index the summary report to ES, but the report has been indexed in fact ?  
Now, I want to add some key-value paris, where should I change?

Thanks!

---

<div class="post-metadata">

**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\
**Post date:** [September 30, 2018, 10:11am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/9 "2018-09-30T10:11:45Z")

</div>

Hello,

I presume that you used `pdb.set_trace()` and not `pdb.set_trtace()` as the latter is not a valid attribute in the module.

As you correctly noticed Rally relies on the [Thespian actor system](https://thespianpy.com/doc/) and Rally uses [by default](https://github.com/elastic/rally/blob/9e5af36b945d7c813669427cb4c5a9dfab1b1804/esrally/actor.py#L180) the [multiprocTCPBase](https://thespianpy.com/doc/using.html#outline-container-org401f9a7) over IPv4/TCP port 1900. As a result of using an asynchronous distributed model with actors you won't be able to utilize debuggers like pdb/ipdb, as separate processes get launched by Rally and communication is done over TCP. If you insist on setting break points you can try to use the [simpleSystemBase](https://thespianpy.com/doc/using.html#outline-container-org401f9a7) for troubleshooting; this will run the actors synchronously and won't support running multiple Rally (load generator) processes on different machines.

The best, most reliable and fastest way to develop is to add corresponding test cases in [metrics\_test](https://github.com/elastic/rally/blob/master/tests/metrics_test.py). Depending on which index you want to add your additional metrics in, you'll need to adjust the test cases in either [EsMetricsTests](https://github.com/elastic/rally/blob/9e5af36b945d7c813669427cb4c5a9dfab1b1804/tests/metrics_test.py#L268-L719) class or [EsResultsStoreTests](https://github.com/elastic/rally/blob/9e5af36b945d7c813669427cb4c5a9dfab1b1804/tests/metrics_test.py#L836-L1007).

---

<div class="post-metadata">

**Author:** ![Todd](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@Todd](https://discuss.elastic.co/u/Todd)\
**Post date:** [October 8, 2018, 2:40am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/10 "2018-10-08T02:40:04Z")

</div>

Hi,

I am on holiday these days, I am deeply sorry for my late reply.

> [@dliappis](#):
>
> pdb.set\_trtace()

this is my spelling mistake when asking this question, in fact I use the `pdb.set_trace()`.

I found that modifying the source code may not be a good way to solve this problem, I will try other ways, thanks a lot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2018, 2:40am UTC](https://discuss.elastic.co/t/how-to-customize-the-summary-report/150134/11 "2018-11-05T02:40:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.