Same space usage when disabling storing the source document

Elastic Stack Elasticsearch
1

Hi,

I've ingested 120GB of data into an elasticsearch index, and ~115GB of that data is used for storing the source document. Considering I don't need the source document and I only need the IDs of the documents that match an ES query, I tried disabling storing the source document. However, the space usage didn't go down, apparently due to an ES feature called soft delete:

Indices with "_source.enabled: false" same size as indices with "_source.enabled: true" · Issue #41628 · elastic/elasticsearch · GitHub)

A few questions:

  1. I bulk-ingested into a new index with no delete operations. Why are the source documents still being retained?
  2. What are the implications of disabling soft delete?

Thanks,
Ahmed

2

This seems weird, how did you got to those numbers? Can you share how?

Quick question, do you use discover in Kibana to see your data? If you use it you cannot disable the _source field, it is required by DIscover.

3

Hey Leandro,

I appreciate the response! I'm using the analyze disk usage API to determine the space used by storing the source document:

Analyze index disk usage API | Elasticsearch Guide [8.13] | Elastic

I don't use Kibana -- Elasticsearch only.