# Elasticsearch store time series data when there are many properties fields (about 60000 fields)

**URL:** <https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114>\
**Category:** Elasticsearch\
**Created:** [February 20, 2020, 7:52am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114 "2020-02-20T07:52:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![qinlin](https://avatars.discourse-cdn.com/v4/letter/q/8491ac/32.png) [@qinlin](https://discuss.elastic.co/u/qinlin)\
**Post date:** [February 20, 2020, 7:52am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/1 "2020-02-20T07:52:35Z")

</div>

store 60000 points value every 500 milliseconds with elasticsearch,there are two ways to design elasticsearch mapping:

1.one pointName/one timestamp in one doc,like this:

```auto
_id timestamp pointName value 
uuid1 1582130490000 p1 1 
uuid2 1582130490000 p2 2 
....
uuid 1582130490500 p1 x

```

2.one timestamp with 60000 pointName in one doc,like this:

```auto
_id timestamp p1 p2 p3 p4 ... p60000
uuid1 1582130490000 1 2 3 4 ... x
uuid2 1582130490500 1 2 3 4 ... x

```

i have tested write/query performance in two uper ways,no problem with aggregate query performance.but batch write performance is too slow,it takes about 1500 milliseconds to write 10000 points with 3 elasticsearch clusters.

is there a bad write performance over 1000 fields in elasticsearch? how should i to design elasticsearch mapping to store 60000 points every 500 milliseconds with good write performance?

thanks

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 20, 2020, 8:02am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/2 "2020-02-20T08:02:46Z")

</div>

What is the size of your Elasticsearch cluster? What is the specification of the nodes? What type of storage are you using? Have you followed the guidelines around tuning for indexing speed in the docs?

---

<div class="post-metadata">

**Author:** ![qinlin](https://avatars.discourse-cdn.com/v4/letter/q/8491ac/32.png) [@qinlin](https://discuss.elastic.co/u/qinlin)\
**Post date:** [February 20, 2020, 8:10am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/3 "2020-02-20T08:10:21Z")

</div>

thank you for your reply!  
I'm just testing the write speed of two mapping designs. I'm not sure whether it's better to store in column(one timestamp with 60000 pointName in one doc) or in row(1.one pointName/one timestamp in one doc). Can you give me some suggestions on storing 60000 point mapping design.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 20, 2020, 8:33am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/4 "2020-02-20T08:33:43Z")

</div>

As that is a lot of fields I would probably go with option 1. Many small documents may also spread the load better across the cluster.

---

<div class="post-metadata">

**Author:** ![qinlin](https://avatars.discourse-cdn.com/v4/letter/q/8491ac/32.png) [@qinlin](https://discuss.elastic.co/u/qinlin)\
**Post date:** [February 20, 2020, 8:38am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/5 "2020-02-20T08:38:21Z")

</div>

ok,thank you so much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2020, 8:41am UTC](https://discuss.elastic.co/t/elasticsearch-store-time-series-data-when-there-are-many-properties-fields-about-60000-fields/220114/6 "2020-03-19T08:41:25Z")

</div>

Closed on March 19, 2020, 8:41am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.