# Difference between df and du in linux
**URL:** https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287
**Category:** Elasticsearch
**Created:** [April 25, 2011, 3:24pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287 "2011-04-25T15:24:02Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)
#### Post date: [April 25, 2011, 3:24pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/1 "2011-04-25T15:24:02Z")
Hi,
In one of our production clusters we saw that there is a big difference
between df and du disk size reports. df shows 100% and du shows as 70%. This
remains as it is while cluster is running. Do you have any ideas about the
reason of this problem?
--
Mustafa Sener
[www.ifountain.com](http://www.ifountain.com)
---
### Author:  [@Joaquin\_Cuenca\_Abela](https://discuss.elastic.co/u/Joaquin_Cuenca_Abela)
#### Post date: [April 25, 2011, 4:48pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/2 "2011-04-25T16:48:05Z")
df shows disk space usage on your volume, du disk used by the list
that it can access. The remaining 30% is in files that du is not
seeing. Assuming there are no obvious mistakes, this usually comes
from big files that have been deleted from the file system but that
are still mapped by some process. For instance, maybe you deleted a
big log file, but forgot to restart / reload the server that created
this log file.
On Mon, Apr 25, 2011 at 5:24 PM, Mustafa Sener [mustafa.sener@gmail.com](mailto:mustafa.sener@gmail.com) wrote:
> Hi,
> In one of our production clusters we saw that there is a big difference
> between df and du disk size reports. df shows 100% and du shows as 70%. This
> remains as it is while cluster is running. Do you have any ideas about the
> reason of this problem?
>
> --
> Mustafa Sener
> [www.ifountain.com](http://www.ifountain.com)
--
Joaquin Cuenca Abela -- [presspeople.com](http://presspeople.com): Fuentes de prensa y comunicados
---
### Author:  [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)
#### Post date: [April 25, 2011, 6:27pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/3 "2011-04-25T18:27:17Z")
That is the reason why I report this situation. I am trying to investigate
whether this may caused by ES or not. We come across this situation twice in
our production ES cluster. ES gave an exception about disk space previously.
When we checked file system we saw that df and du give different values. We
restarted servers and our data is corrupted because of this. We restarted
them and repopulated whole data again and after nearly two months later same
situation appeared again. We use ES version 0.15.2. When I searched about
this problem, I saw that other lucene based products face with same problem
if any IndexReader is not closed. may this be a problem in ES too?
On Mon, Apr 25, 2011 at 7:48 PM, Joaquin Cuenca Abela \<
[joaquin@cuencaabela.com](mailto:joaquin@cuencaabela.com)\> wrote:
> df shows disk space usage on your volume, du disk used by the list
> that it can access. The remaining 30% is in files that du is not
> seeing. Assuming there are no obvious mistakes, this usually comes
> from big files that have been deleted from the file system but that
> are still mapped by some process. For instance, maybe you deleted a
> big log file, but forgot to restart / reload the server that created
> this log file.
>
> On Mon, Apr 25, 2011 at 5:24 PM, Mustafa Sener [mustafa.sener@gmail.com](mailto:mustafa.sener@gmail.com)
> wrote:
>
> > Hi,
> > In one of our production clusters we saw that there is a big difference
> > between df and du disk size reports. df shows 100% and du shows as 70%.
> > This
> > remains as it is while cluster is running. Do you have any ideas about
> > the
> > reason of this problem?
> >
> > --
> > Mustafa Sener
> > [www.ifountain.com](http://www.ifountain.com)
>
> --
> Joaquin Cuenca Abela -- [presspeople.com](http://presspeople.com): Fuentes de prensa y comunicados
--
Mustafa Sener
[www.ifountain.com](http://www.ifountain.com)
---
### Author:  [@Yeroc1](https://discuss.elastic.co/u/Yeroc1)
#### Post date: [April 25, 2011, 8:34pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/4 "2011-04-25T20:34:51Z")
You can use a command like "lsof | grep deleted" to get a listing of
the deleted files which have open file handles along with the pid that
is holding them. At least you'll know the names of the files and can
validate that they are indeed being held open by elasticsearch. If
you provide the filenames it may at least help narrow down the cause
of the issue.
On Apr 25, 12:27 pm, Mustafa Sener [mustafa.se...@gmail.com](mailto:mustafa.se...@gmail.com) wrote:
> That is the reason why I report this situation. I am trying to investigate
> whether this may caused by ES or not. We come across this situation twice in
> our production ES cluster. ES gave an exception about disk space previously.
> When we checked file system we saw that df and du give different values. We
> restarted servers and our data is corrupted because of this. We restarted
> them and repopulated whole data again and after nearly two months later same
> situation appeared again. We use ES version 0.15.2. When I searched about
> this problem, I saw that other lucene based products face with same problem
> if any IndexReader is not closed. may this be a problem in ES too?
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [April 26, 2011, 5:45pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/5 "2011-04-26T17:45:38Z")
Which version are you using? You might hit this: [Concurrent calls to refresh might result in "dangling" searchers · Issue #823 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/823) which was fixed in 0.16.
On Monday, April 25, 2011 at 11:34 PM, Yeroc wrote:
> You can use a command like "lsof | grep deleted" to get a listing of
> the deleted files which have open file handles along with the pid that
> is holding them. At least you'll know the names of the files and can
> validate that they are indeed being held open by elasticsearch. If
> you provide the filenames it may at least help narrow down the cause
> of the issue.
>
> On Apr 25, 12:27 pm, Mustafa Sener [mustafa.se...@gmail.com](mailto:mustafa.se...@gmail.com) wrote:
>
> > That is the reason why I report this situation. I am trying to investigate
> > whether this may caused by ES or not. We come across this situation twice in
> > our production ES cluster. ES gave an exception about disk space previously.
> > When we checked file system we saw that df and du give different values. We
> > restarted servers and our data is corrupted because of this. We restarted
> > them and repopulated whole data again and after nearly two months later same
> > situation appeared again. We use ES version 0.15.2. When I searched about
> > this problem, I saw that other lucene based products face with same problem
> > if any IndexReader is not closed. may this be a problem in ES too?
---
### Author:  [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)
#### Post date: [April 26, 2011, 6:11pm UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/6 "2011-04-26T18:11:56Z")
I am using version 0.15.2
On Tue, Apr 26, 2011 at 8:45 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> Which version are you using? You might hit this:
> [Concurrent calls to refresh might result in "dangling" searchers · Issue #823 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/823) which was fixed
> in 0.16.
>
> On Monday, April 25, 2011 at 11:34 PM, Yeroc wrote:
>
> You can use a command like "lsof | grep deleted" to get a listing of
> the deleted files which have open file handles along with the pid that
> is holding them. At least you'll know the names of the files and can
> validate that they are indeed being held open by elasticsearch. If
> you provide the filenames it may at least help narrow down the cause
> of the issue.
>
> On Apr 25, 12:27 pm, Mustafa Sener [mustafa.se...@gmail.com](mailto:mustafa.se...@gmail.com) wrote:
>
> That is the reason why I report this situation. I am trying to investigate
> whether this may caused by ES or not. We come across this situation twice
> in
> our production ES cluster. ES gave an exception about disk space
> previously.
> When we checked file system we saw that df and du give different values. We
> restarted servers and our data is corrupted because of this. We restarted
> them and repopulated whole data again and after nearly two months later
> same
> situation appeared again. We use ES version 0.15.2. When I searched about
> this problem, I saw that other lucene based products face with same problem
> if any IndexReader is not closed. may this be a problem in ES too?
--
Mustafa Sener
[www.ifountain.com](http://www.ifountain.com)
---
### Author:  [@Sourav\_Gulati](https://discuss.elastic.co/u/Sourav_Gulati)
#### Post date: [August 8, 2012, 6:39am UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/7 "2012-08-08T06:39:38Z")
Memory used by all the processes which are open will be added in df not in du. you can run lsof and count the momory used by open processes and add it in the output of du , you will get the same result as given in df .
For example, if df shows the file system which is mounted on /tmp consuming 100% memory then try to run lsof | grep /tmp and count the memory used . The result of df and du will match
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:17am UTC](https://discuss.elastic.co/t/difference-between-df-and-du-in-linux/4287/8 "2017-07-06T03:17:18Z")