# Write to TSDS index via python

**URL:** <https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001>\
**Category:** Elasticsearch\
**Created:** [June 6, 2024, 10:33pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001 "2024-06-06T22:33:57Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 6, 2024, 10:33pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/1 "2024-06-06T22:33:57Z")

</div>

I create ILM/Template and Index  
when I try to write via python it errors  
but same record works via dev tool

```auto
POST sachin_test/_doc
{
  "@timestamp": "2024-06-06T19:21:15.000Z", --> this part I did manually
  "system_type": "gpu", 
  "available": 1, "maintenance_status": 1, "cpu_core_inventory": 0, 
  "sum_cpu_core_inventory": 0, "sum_gpu_card_inventory": 0, 
  "count": 1, "system_prod": "yes", "center": "hou", 
  "host_status": "maintenance"
}

```

Here is python record looks like

`{'@timestamp': Timestamp('2024-06-06 13:45:00+0000', tz='UTC'), 'system_type': 'gpu', 'available':1, 'maintenance_status': 1, 'cpu_core_inventory': 0, 'sum_cpu_core_inventory': 0, 'sum_gpu_card_inventory': 0, 'count': 1, 'system_prod': 'yes', 'center': 'hou', 'host_status': 'maintenance', '_index': 'sachin_test', '_op_type': 'create'}`

it says  
error is 500 document(s) failed to index.  
An exception of type BulkIndexError occurred. Arguments:  
('500 document(s) failed to index.',)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 7, 2024, 5:31am UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/2 "2024-06-07T05:31:48Z")

</div>

> [@elasticforme](#):
>
> {'@timestamp': Timestamp('2024-06-06 13:45:00+0000', tz='UTC'),

Hmmm You should probaly share your python code.

Did you simply try

` 'timestamp': datetime.now(),`

Like the Example here to see if it works...

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 7, 2024, 1:23pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/3 "2024-06-07T13:23:26Z")

</div>

Basically I do push this using helpers.bulk

```auto
mylist = [{'@timestamp': Timestamp('2024-06-06 13:45:00+0000', tz='UTC'), 'system_type': 'gpu', 'available':1, 'maintenance_status': 1, 'cpu_core_inventory': 0, 'sum_cpu_core_inventory': 0, 'sum_gpu_card_inventory': 0, 'count': 1, 'system_prod': 'yes', 'center': 'hou', 'host_status': 'maintenance', '_index': 'sachin_test', '_op_type': 'create'}]

helpers.bulk(elastic_output, mylist)

```

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 7, 2024, 1:25pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/4 "2024-06-07T13:25:35Z")

</div>

> [@stephenb](#):
>
> `'timestamp': datetime.now(),`
> 
> Like the Example here to see if it works...

and I just try this. same error

{'@timestamp': Timestamp('2024-06-07 08:25:03.480489'),

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 7, 2024, 2:32pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/5 "2024-06-07T14:32:20Z")

</div>

Delete the index first and then try doing the python first.  
Perhaps there's something wrong with the initial doc you posted

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 7, 2024, 4:12pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/6 "2024-06-07T16:12:13Z")

</div>

if I write to different index all togather ( regular index) then it works.  
something with TSDS and timestamp for sure.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 7, 2024, 4:32pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/7 "2024-06-07T16:32:33Z")

</div>

Did you walk through this carefully?

> **[Set up a time series data stream (TSDS) | Elasticsearch Guide \[8.14\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-tsds.html)**

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 7, 2024, 4:39pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/8 "2024-06-07T16:39:52Z")

</div>

ok it works. Thanks @stephenb

basically I had time different between TSDS creation and @timestamp that I was inserting.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 7, 2024, 4:40pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/9 "2024-06-07T16:40:56Z")

</div>

> [@elasticforme](#):
>
> basically I had time different between TSDS creation and @timestamp that I was inserting.

Yes that is interesting can you show / describe a little more for anyone else that runs into the same?

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [June 7, 2024, 7:53pm UTC](https://discuss.elastic.co/t/write-to-tsds-index-via-python/361001/10 "2024-06-07T19:53:42Z")

</div>

I think when you create TSDS with index.mode = time\_series enable

then it has two value

index.time\_series.start\_time value set to `now - index.look_ahead_time`  
index.time\_series.end\_time value set to `now + index.look_ahead_time`

and you can only write between these two time.
