# Average text field length

**URL:** <https://discuss.elastic.co/t/average-text-field-length/170534>\
**Category:** Elasticsearch\
**Created:** [March 1, 2019, 4:32pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534 "2019-03-01T16:32:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bbking](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bbking/32/77563_2.png) [@bbking](https://discuss.elastic.co/u/bbking)\
**Post date:** [March 1, 2019, 4:32pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534/1 "2019-03-01T16:32:17Z")

</div>

Hi, is there way to get the average length of a text field? I got `"Fielddata is disabled on text fields by default"` exception when I try to use avg aggregation. In the following query, "reason" is a text field type.

```auto
  "size": 0,
  "aggs" : {
    "avg_size" : {
      "avg": {
         "script" : {
           "lang": "painless",
           "source": "doc['reason'].value.length()"
        }
      }
    }
  }

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 1, 2019, 5:34pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534/2 "2019-03-01T17:34:39Z")

</div>

I'd compute that at index time with a similar painless script that I'd put in a [Script Processor](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/script-processor.html).

---

<div class="post-metadata">

**Author:** ![bbking](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bbking/32/77563_2.png) [@bbking](https://discuss.elastic.co/u/bbking)\
**Post date:** [March 1, 2019, 6:36pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534/3 "2019-03-01T18:36:53Z")

</div>

I don't need the average length field stored in index. I mainly use it for test to get an idea how long the field is. Is there a way to do it using Search API?

I mainly want to figure out how and why Elasticsearch stores documents, the original sources, efficiently in terms of space. I understand that inverted docs efficiently store indexs. But how does the original documents compressed that ES shows much smaller space usage compared to MySQL DB? Do duplicate fields in different documents only share the same copy of data?

Thanks!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 2, 2019, 12:01pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534/4 "2019-03-02T12:01:27Z")

</div>

> [@bbking](#):
>
> I mainly want to figure out how and why Elasticsearch stores documents, the original sources, efficiently in terms of space.

That's a totally different story. There are so many data structures involved depending on what you are doing ie `text` vs `keyword`, `text analyzers`, `store`, `compression algorithm`, number of segments, number of shards...

Comparing MySQL storage (not speaking about MySQL indices) vs Elasticsearch storage is like to me comparing oranges and apples.

If we just speak of storing `_source`, which mean basically disabling all indexation on all fields, then you might be able to compare...  
Compression on stored fields is done per Lucene segment. A shard may have multiple segments unless you call the `_forcemerge` API. An index may have multiple shards.  
So if you want to "compare", you may want to create one index with one single shard, disable all indexing for all fields in the mapping, and once the operation is done, call the `_forcemerge` API to have only one segments.

Then you can more or less check the size on disk by doing:

```auto
du -s data/nodes/0/indices/vj8heygETPqlLrBQUZsR1w/0/index

```

Note that `vj8heygETPqlLrBQUZsR1w` depends on your index id, which you can find with the `_cat/indices` API:

```auto
GET /_cat/indices/yourindexname?v

```

HTH

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2019, 12:01pm UTC](https://discuss.elastic.co/t/average-text-field-length/170534/5 "2019-03-30T12:01:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.