# 3 nodes to a cluster and 1 using a lot more disk space

**URL:** https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366
**Category:** Elasticsearch
**Created:** [November 12, 2013, 7:26pm UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366 "2013-11-12T19:26:36Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Mike\_Zupan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_zupan/32/1981_2.png) [@Mike\_Zupan](https://discuss.elastic.co/u/Mike_Zupan)
#### Post date: [November 12, 2013, 7:26pm UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/1 "2013-11-12T19:26:36Z")

</div>

I have 3 nodes in a cluster and 1 (the primary right now) is using a lot  
more disk space then the other 2. Is this normal?

/dev/mapper/data--group-data--vol 50G 32G 15G 69% /data

vs

/dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
/dev/mapper/data--group-data--vol 50G 19G 29G 40% /data

There is nothing else using this partition except for elasticsearch.

Thanks

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [November 13, 2013, 3:48am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/2 "2013-11-13T03:48:48Z")

</div>

Out of curiosity, under /dev/mapper/data--group-data--vol/data/${  
cluster.name}/nodes, how many directories are there? How about just under  
/dev/mapper/data--group-data--vol/data/? Perhaps you have some old data  
under a different cluster name, or you started two instances on the same  
node resulting in two directories.

Cheers,

Ivan

On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan [mike@recurly.com](mailto:mike@recurly.com) wrote:

> I have 3 nodes in a cluster and 1 (the primary right now) is using a lot  
> more disk space then the other 2. Is this normal?
> 
> /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> 
> vs
> 
> /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> 
> There is nothing else using this partition except for elasticsearch.
> 
> Thanks
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Mike\_Zupan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_zupan/32/1981_2.png) [@Mike\_Zupan](https://discuss.elastic.co/u/Mike_Zupan)
#### Post date: [November 13, 2013, 4:36am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/3 "2013-11-13T04:36:23Z")

</div>

Ivan,

Thanks for the reply.. here they are for the 2 that are the same for  
/data/cluster.name/nodes

$ ls  
0  
$  
0

Then for the one larger

$ ls  
0 1

So it seems like you are correct.

Then on the larger one

# du -h --max-depth=1

2.1G ./1  
14G ./0

So it seems there is an extra 1 that might have a lot of data.

On Tuesday, November 12, 2013 7:48:48 PM UTC-8, Ivan Brusic wrote:

> Out of curiosity, under /dev/mapper/data--group-data--vol/data/${  
> cluster.name}/nodes, how many directories are there? How about just under  
> /dev/mapper/data--group-data--vol/data/? Perhaps you have some old data  
> under a different cluster name, or you started two instances on the same  
> node resulting in two directories.
> 
> Cheers,
> 
> Ivan
> 
> On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan \<[mi...@recurly.com](mailto:mi...@recurly.com)\<javascript:\>
> 
> > wrote:
> 
> > I have 3 nodes in a cluster and 1 (the primary right now) is using a lot  
> > more disk space then the other 2. Is this normal?
> > 
> > /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> > 
> > vs
> > 
> > /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> > /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> > 
> > There is nothing else using this partition except for elasticsearch.
> > 
> > Thanks
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Mike\_Zupan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_zupan/32/1981_2.png) [@Mike\_Zupan](https://discuss.elastic.co/u/Mike_Zupan)
#### Post date: [November 13, 2013, 4:38am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/4 "2013-11-13T04:38:07Z")

</div>

So it seems the data is all in

16G ./elasticsearch/nodes/0

That is under ./data/

On Tuesday, November 12, 2013 8:36:23 PM UTC-8, Mike Zupan wrote:

> Ivan,
> 
> Thanks for the reply.. here they are for the 2 that are the same for /data/  
> cluster.name/nodes
> 
> $ ls  
> 0  
> $  
> 0
> 
> Then for the one larger
> 
> $ ls  
> 0 1
> 
> So it seems like you are correct.
> 
> Then on the larger one
> 
> # du -h --max-depth=1
> 
> 2.1G ./1  
> 14G ./0
> 
> So it seems there is an extra 1 that might have a lot of data.
> 
> On Tuesday, November 12, 2013 7:48:48 PM UTC-8, Ivan Brusic wrote:
> 
> > Out of curiosity, under /dev/mapper/data--group-data--vol/data/${  
> > cluster.name}/nodes, how many directories are there? How about just  
> > under /dev/mapper/data--group-data--vol/data/? Perhaps you have some old  
> > data under a different cluster name, or you started two instances on the  
> > same node resulting in two directories.
> > 
> > Cheers,
> > 
> > Ivan
> > 
> > On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan [mi...@recurly.com](mailto:mi...@recurly.com) wrote:
> > 
> > > I have 3 nodes in a cluster and 1 (the primary right now) is using a lot  
> > > more disk space then the other 2. Is this normal?
> > > 
> > > /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> > > 
> > > vs
> > > 
> > > /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> > > /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> > > 
> > > There is nothing else using this partition except for elasticsearch.
> > > 
> > > Thanks
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [November 13, 2013, 5:19am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/5 "2013-11-13T05:19:18Z")

</div>

So at one point you must have started two instances on the same node. Try  
using the server wrapper or other method (rpm, etc..) that uses a pid to  
only run one instance at a time.

--  
Ivan

On Tue, Nov 12, 2013 at 8:38 PM, Mike Zupan [mike@recurly.com](mailto:mike@recurly.com) wrote:

> So it seems the data is all in
> 
> 16G ./elasticsearch/nodes/0
> 
> That is under ./data/
> 
> On Tuesday, November 12, 2013 8:36:23 PM UTC-8, Mike Zupan wrote:
> 
> > Ivan,
> > 
> > Thanks for the reply.. here they are for the 2 that are the same for  
> > /data/cluster.name/nodes
> > 
> > $ ls  
> > 0  
> > $  
> > 0
> > 
> > Then for the one larger
> > 
> > $ ls  
> > 0 1
> > 
> > So it seems like you are correct.
> > 
> > Then on the larger one
> > 
> > # du -h --max-depth=1
> > 
> > 2.1G ./1  
> > 14G ./0
> > 
> > So it seems there is an extra 1 that might have a lot of data.
> > 
> > On Tuesday, November 12, 2013 7:48:48 PM UTC-8, Ivan Brusic wrote:
> > 
> > > Out of curiosity, under /dev/mapper/data--group-data--vol/data/${  
> > > cluster.name}/nodes, how many directories are there? How about just  
> > > under /dev/mapper/data--group-data--vol/data/? Perhaps you have some  
> > > old data under a different cluster name, or you started two instances on  
> > > the same node resulting in two directories.
> > > 
> > > Cheers,
> > > 
> > > Ivan
> > > 
> > > On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan [mi...@recurly.com](mailto:mi...@recurly.com) wrote:
> > > 
> > > > I have 3 nodes in a cluster and 1 (the primary right now) is using a  
> > > > lot more disk space then the other 2. Is this normal?
> > > > 
> > > > /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> > > > 
> > > > vs
> > > > 
> > > > /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> > > > /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> > > > 
> > > > There is nothing else using this partition except for elasticsearch.
> > > > 
> > > > Thanks
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups "elasticsearch" group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Mike\_Zupan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_zupan/32/1981_2.png) [@Mike\_Zupan](https://discuss.elastic.co/u/Mike_Zupan)
#### Post date: [November 13, 2013, 5:21am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/6 "2013-11-13T05:21:01Z")

</div>

Is there a safe way to clean it?

--  
Mike Zupan

On Tuesday, November 12, 2013 at 9:19 PM, Ivan Brusic wrote:

> So at one point you must have started two instances on the same node. Try using the server wrapper or other method (rpm, etc..) that uses a pid to only run one instance at a time.
> 
> --  
> Ivan
> 
> On Tue, Nov 12, 2013 at 8:38 PM, Mike Zupan \<[mike@recurly.com](mailto:mike@recurly.com) ([mailto:mike@recurly.com](mailto:mike@recurly.com))\> wrote:
> 
> > So it seems the data is all in
> > 
> > 16G ./elasticsearch/nodes/0
> > 
> > That is under ./data/
> > 
> > On Tuesday, November 12, 2013 8:36:23 PM UTC-8, Mike Zupan wrote:
> > 
> > > Ivan,
> > > 
> > > Thanks for the reply.. here they are for the 2 that are the same for /data/cluster.name/nodes ([http://cluster.name/nodes](http://cluster.name/nodes))
> > > 
> > > $ ls  
> > > 0  
> > > $  
> > > 0
> > > 
> > > Then for the one larger
> > > 
> > > $ ls  
> > > 0 1
> > > 
> > > So it seems like you are correct.
> > > 
> > > Then on the larger one
> > > 
> > > # du -h --max-depth=1
> > > 
> > > 2.1G ./1  
> > > 14G ./0
> > > 
> > > So it seems there is an extra 1 that might have a lot of data.  
> > > On Tuesday, November 12, 2013 7:48:48 PM UTC-8, Ivan Brusic wrote:
> > > 
> > > > Out of curiosity, under /dev/mapper/data--group-data--vol/data/${cluster.name ([http://cluster.name](http://cluster.name))}/nodes, how many directories are there? How about just under /dev/mapper/data--group-data--vol/data/? Perhaps you have some old data under a different cluster name, or you started two instances on the same node resulting in two directories.
> > > > 
> > > > Cheers,
> > > > 
> > > > Ivan
> > > > 
> > > > On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan [mi...@recurly.com](mailto:mi...@recurly.com) wrote:
> > > > 
> > > > > I have 3 nodes in a cluster and 1 (the primary right now) is using a lot more disk space then the other 2. Is this normal?
> > > > > 
> > > > > /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> > > > > 
> > > > > vs
> > > > > 
> > > > > /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> > > > > /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> > > > > 
> > > > > There is nothing else using this partition except for elasticsearch.
> > > > > 
> > > > > Thanks
> > > > > 
> > > > > --  
> > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) (mailto:elasticsearch%2Bunsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/\_EY3rkka4r4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_EY3rkka4r4/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) ([mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com)).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [November 13, 2013, 7:34am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/7 "2013-11-13T07:34:10Z")

</div>

If you are sure that the files are not in used (timestamp?), you can delete  
them directly in the OS.

--  
Ivan

On Tue, Nov 12, 2013 at 9:21 PM, Mike Zupan [mike@recurly.com](mailto:mike@recurly.com) wrote:

> Is there a safe way to clean it?
> 
> --  
> Mike Zupan
> 
> On Tuesday, November 12, 2013 at 9:19 PM, Ivan Brusic wrote:
> 
> So at one point you must have started two instances on the same node. Try  
> using the server wrapper or other method (rpm, etc..) that uses a pid to  
> only run one instance at a time.
> 
> --  
> Ivan
> 
> On Tue, Nov 12, 2013 at 8:38 PM, Mike Zupan [mike@recurly.com](mailto:mike@recurly.com) wrote:
> 
> So it seems the data is all in
> 
> 16G ./elasticsearch/nodes/0
> 
> That is under ./data/
> 
> On Tuesday, November 12, 2013 8:36:23 PM UTC-8, Mike Zupan wrote:
> 
> Ivan,
> 
> Thanks for the reply.. here they are for the 2 that are the same for /data/  
> cluster.name/nodes
> 
> $ ls  
> 0  
> $  
> 0
> 
> Then for the one larger
> 
> $ ls  
> 0 1
> 
> So it seems like you are correct.
> 
> Then on the larger one
> 
> # du -h --max-depth=1
> 
> 2.1G ./1  
> 14G ./0
> 
> So it seems there is an extra 1 that might have a lot of data.
> 
> On Tuesday, November 12, 2013 7:48:48 PM UTC-8, Ivan Brusic wrote:
> 
> Out of curiosity, under /dev/mapper/data--group-data--vol/data/${  
> cluster.name}/nodes, how many directories are there? How about just under  
> /dev/mapper/data--group-data--vol/data/? Perhaps you have some old data  
> under a different cluster name, or you started two instances on the same  
> node resulting in two directories.
> 
> Cheers,
> 
> Ivan
> 
> On Tue, Nov 12, 2013 at 11:26 AM, Mike Zupan [mi...@recurly.com](mailto:mi...@recurly.com) wrote:
> 
> I have 3 nodes in a cluster and 1 (the primary right now) is using a lot  
> more disk space then the other 2. Is this normal?
> 
> /dev/mapper/data--group-data--vol 50G 32G 15G 69% /data
> 
> vs
> 
> /dev/mapper/data--group-data--vol 50G 18G 30G 38% /data  
> /dev/mapper/data--group-data--vol 50G 19G 29G 40% /data
> 
> There is nothing else using this partition except for elasticsearch.
> 
> Thanks
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/\_EY3rkka4r4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/_EY3rkka4r4/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> 
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:07am UTC](https://discuss.elastic.co/t/3-nodes-to-a-cluster-and-1-using-a-lot-more-disk-space/14366/8 "2017-07-06T02:07:35Z")

</div>


