# Create track using datastreams

**URL:** <https://discuss.elastic.co/t/create-track-using-datastreams/289097>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [November 12, 2021, 7:49pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097 "2021-11-12T19:49:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kvmuralidhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvmuralidhar/32/45160_2.png) [@kvmuralidhar](https://discuss.elastic.co/u/kvmuralidhar)\
**Post date:** [November 12, 2021, 7:49pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097/1 "2021-11-12T19:49:07Z")

</div>

Hi,

I am trying to create a custom track using an data stream index and it's failing because of the datastream is an hidden index. What's the work around to create this custom track ?

Thanks  
Murali

---

<div class="post-metadata">

**Author:** ![Bradley\_Deam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bradley_deam/32/52991_2.png) [@Bradley\_Deam](https://discuss.elastic.co/u/Bradley_Deam)\
**Post date:** [November 15, 2021, 10:48pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097/2 "2021-11-15T22:48:41Z")

</div>

Hey @kvmuralidhar !

Unfortunately we don't have this functionality available 😞 - we are aware of it though: [Support data streams in create-track · Issue #1360 · elastic/rally · GitHub](https://github.com/elastic/rally/issues/1360)

However, you can 'hack' Rally for a temp workaround (I quickly tested this locally) by checking out the repo locally and modifying..

[Line 68](https://github.com/elastic/rally/blob/f4302fbbc38bf4e7c00cd43ac9b363f26e579196/esrally/tracker/index.py#L68):

```auto
# Original
response = client.indices.get(index_pattern)

# Modified to
response = client.indices.get(index_pattern, params={"expand_wildcards": "all"})

```

[Line 52-53](https://github.com/elastic/rally/blob/f4302fbbc38bf4e7c00cd43ac9b363f26e579196/esrally/tracker/index.py#L52-L53):

```auto
# Original
    if index_name.startswith("."):
        return False, f"Index [{index_name}] is hidden"
# Modified to (comment these lines out)
    # if index_name.startswith("."):
    # return False, f"Index [{index_name}] is hidden"

```

That should unblock you for the time being. I tested this with my own personal cluster:

```auto
$ esrally create-track --track=iis --target-hosts=https://my-es-cluster:9243 --client-options="timeout:320,use_ssl:true,verify_certs:true,basic_auth_user:'elastic',basic_auth_password:'not a real password'" --indices=".ds-metrics-windows.service-default-2021.05.21-000013" --output-path=.

    ________
   / __\____ _/ / /_ __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\ __,_/_/_/\__ , /
                / ____ /

[INFO] Connected to Elasticsearch cluster [instance-0000000068] version [7.12.0].

Extracting documents for index [.ds-metrics-windows.se... 1000/1000 docs [100.0% done]
Extracting documents for index [.ds-metrics-window... 216117/1947230 docs [11.1% done]

```

---

<div class="post-metadata">

**Author:** ![kvmuralidhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvmuralidhar/32/45160_2.png) [@kvmuralidhar](https://discuss.elastic.co/u/kvmuralidhar)\
**Post date:** [November 15, 2021, 11:33pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097/3 "2021-11-15T23:33:05Z")

</div>

@Bradley_Deam

It worked like a champ, thanks for the work around.

Murali

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 13, 2021, 11:34pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097/4 "2021-12-13T23:34:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.