# Elasticsearch Indices and Space

**URL:** <https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536>\
**Category:** Elasticsearch\
**Created:** [December 7, 2015, 2:07pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536 "2015-12-07T14:07:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jackal9301](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jackal9301/32/5748_2.png) [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\
**Post date:** [December 7, 2015, 2:07pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/1 "2015-12-07T14:07:05Z")

</div>

I'm starting an elasticsearch instance and i'm importing logs from logstash. Kibana is how I am viewing them after. My question is how does elasticsearch index the data? I create fields in logstash I want to search by however those fields don't seem to work when i search them in kibana. If I have more fields does it take up more space to index? I'd really like to understand what takes up the most space when it comes to storage.

Thanks in advance for any insight!!!

Jack West

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 7, 2015, 3:37pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/2 "2015-12-07T15:37:35Z")

</div>

May be this is good to read? [https://www.elastic.co/guide/en/elasticsearch/guide/current/inverted-index.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/inverted-index.html)

Hope this helps

---

<div class="post-metadata">

**Author:** ![Jackal9301](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jackal9301/32/5748_2.png) [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\
**Post date:** [December 7, 2015, 3:42pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/3 "2015-12-07T15:42:10Z")

</div>

Thanks, This is a start. I'm trying to understand why my indices moving from 1.7 - 2.1 have jumped a huge size and how I can fix it.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 7, 2015, 3:54pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/4 "2015-12-07T15:54:33Z")

</div>

The main reason I can see is `doc_values`.  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html)

See also this thread: [Index Size: Elasticsearch 1.4 -\> 2.1](https://discuss.elastic.co/t/index-size-elasticsearch-1-4-2-1/36253)

---

<div class="post-metadata">

**Author:** ![Jackal9301](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jackal9301/32/5748_2.png) [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\
**Post date:** [December 9, 2015, 3:13pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/5 "2015-12-09T15:13:11Z")

</div>

I'm not very familiar with database logic. However I'm looking at my fields area in my index using **_Sense_**. What do I need to change in here? From what I can tell I don't have any mappings.

```
    "_index": ".kibana",
    "_type": "index-pattern",
    "_id": "logstash-*",
    "_score": 1,
    "_source": {
      "title": "logstash-*",
      "timeFieldName": "@timestamp",
      "fields": "[{\"name\":\"_index\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"geoip.ip\",\"type\":\"ip\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"@timestamp\",\"type\":\"date\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.location\",\"type\":\"geo_point\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":false},{\"name\":\"@version\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.latitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"_source\",\"type\":\"_source\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"geoip.longitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"timestamp.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"Source_Port.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"host\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Error_id.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"LCEPRM\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"LCE_time.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"Source_IP\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Log_Type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"tags.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"Log\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Destination_IP\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Destination_IP.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"tags\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Message_Data\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"Event_ID.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"LCE_log_num.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"header_type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"User.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"Error_id\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"User\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"header_type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"Log_Type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"@message.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"LCEPRM.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},
    }

```

I apologize if there is a better way to format that! My goal is to take as little disk space as possible even if I have to sacrifice memory

---

<div class="post-metadata">

**Author:** ![Jackal9301](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jackal9301/32/5748_2.png) [@Jackal9301](https://discuss.elastic.co/u/Jackal9301)\
**Post date:** [December 9, 2015, 3:17pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/6 "2015-12-09T15:17:29Z")

</div>

Also I see all of my variables I created in logstash but what are the .raw extentions? it looks like it doubles my fields

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:32pm UTC](https://discuss.elastic.co/t/elasticsearch-indices-and-space/36536/7 "2017-07-05T23:32:17Z")

</div>