# Path.data v0.18.1 **URL:** \ **Category:** Elasticsearch\ **Created:** [October 27, 2011, 10:08pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712 "2011-10-27T22:08:54Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 27, 2011, 10:08pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/1 "2011-10-27T22:08:54Z")
Hello, I'm using version 0.18.1 and configuring path.data as follows: path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 After 65,000 index doc, I see the size of each path and I was a little surprised the lengths of each path were not similar. Below the size of each path found. gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index / | grep "total" total 350M gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index / | grep "total" 34m total gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index / | grep "total" total 60M gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index / | grep "total" total 35M You can observe that the size of this data4 path much larger than the other paths. Is that correct? I thought that the sizes of the path would be the same or very close. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 27, 2011, 11:12pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/2 "2011-10-27T23:12:37Z")
Is that on the same drive, or different drives? On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > Hello, > I'm using version 0.18.1 and configuring path.data as follows: > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > After 65,000 index doc, I see the size of each path and I was a little > surprised the lengths of each path were not similar. Below the size of > each path found. > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > / | grep "total" > total 350M > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > / | grep "total" > 34m total > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > / | grep "total" > total 60M > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > / | grep "total" > total 35M > > You can observe that the size of this data4 path much larger than the > other paths. Is that correct? I thought that the sizes of the path > would be the same or very close. ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 27, 2011, 11:28pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/3 "2011-10-27T23:28:34Z")
Is the same Driver. 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > Is that on the same drive, or different drives? > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > Hello, > > I'm using version 0.18.1 and configuring path.data as follows: > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > After 65,000 index doc, I see the size of each path and I was a little > > surprised the lengths of each path were not similar. Below the size of > > each path found. > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 350M > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > 34m total > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 60M > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 35M > > > > You can observe that the size of this data4 path much larger than the > > other paths. Is that correct? I thought that the sizes of the path > > would be the same or very close. -- Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 27, 2011, 11:32pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/4 "2011-10-27T23:32:15Z")
I'm using the same drive. 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > Is that on the same drive, or different drives? > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > Hello, > > I'm using version 0.18.1 and configuring path.data as follows: > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > After 65,000 index doc, I see the size of each path and I was a little > > surprised the lengths of each path were not similar. Below the size of > > each path found. > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 350M > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > 34m total > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 60M > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > / | grep "total" > > total 35M > > > > You can observe that the size of this data4 path much larger than the > > other paths. Is that correct? I thought that the sizes of the path > > would be the same or very close. -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 27, 2011, 11:33pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/5 "2011-10-27T23:33:05Z")
The check for the size of where to place the data is based on the free space available per "drive", so having multiple data locations on the same drive makes little sense. On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > Is the same Driver. > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > Is that on the same drive, or different drives? > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > > > Hello, > > > I'm using version 0.18.1 and configuring path.data as follows: > > > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > After 65,000 index doc, I see the size of each path and I was a little > > > surprised the lengths of each path were not similar. Below the size of > > > each path found. > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > / | grep "total" > > > total 350M > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > / | grep "total" > > > 34m total > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > / | grep "total" > > > total 60M > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > / | grep "total" > > > total 35M > > > > > > You can observe that the size of this data4 path much larger than the > > > other paths. Is that correct? I thought that the sizes of the path > > > would be the same or very close. > > -- > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 27, 2011, 11:40pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/6 "2011-10-27T23:40:29Z")
OK. If I have 4 drives with the same size (150GB each drive). When I index 1million documents all drives will have the same size? I'm doing tests on my machine and then indexing at amazon. 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > The check for the size of where to place the data is based on the free > space available per "drive", so having multiple data locations on the same > drive makes little sense. > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > wrote: > > > Is the same Driver. > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > Is that on the same drive, or different drives? > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > Hello, > > > > I'm using version 0.18.1 and configuring path.data as follows: path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > After 65,000 index doc, I see the size of each path and I was a little > > > > surprised the lengths of each path were not similar. Below the size of > > > > each path found. > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > / | grep "total" > > > > total 350M > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > / | grep "total" > > > > 34m total > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > / | grep "total" > > > > total 60M > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > / | grep "total" > > > > total 35M > > > > > > > > You can observe that the size of this data4 path much larger than the > > > > other paths. Is that correct? I thought that the sizes of the path > > > > would be the same or very close. > > > > -- > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 28, 2011, 2:48am UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/7 "2011-10-28T02:48:58Z")
The following test was done using the 0.18.0 version of the ES. Was set up multiple path, in multiple drives. Below is the sizes of the data path. 1. /dev/sdb 414G 199M 393G 1% /mnt 2) /dev/sdc 414G 199M 393G 1% /media/dev1 3) /dev/sdd 414G 15G 378G 4% /media/dev2 4) /dev/sde 414G 199M 393G 1% /media/dev3 This distribution of indices that correct or is it a bug that was fixed in version0.18.1? Earlier I was talking about a test with the version 0.18.1, using the same drive with multiple path. 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > OK. If I have 4 drives with the same size (150GB each drive). When > I index 1million documents all drives will have the same size? > > I'm doing tests on my machine and then indexing at amazon. > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > The check for the size of where to place the data is based on the free > > space available per "drive", so having multiple data locations on the same > > drive makes little sense. > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > wrote: > > > > > Is the same Driver. > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > Is that on the same drive, or different drives? > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia \< > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > Hello, > > > > > I'm using version 0.18.1 and configuring path.data as follows: path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > After 65,000 index doc, I see the size of each path and I was a > > > > > little > > > > > surprised the lengths of each path were not similar. Below the size > > > > > of > > > > > each path found. > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > / | grep "total" > > > > > total 350M > > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > / | grep "total" > > > > > 34m total > > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > / | grep "total" > > > > > total 60M > > > > > gustavo @ gustavo-linux: ~ $ ls-lh /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > / | grep "total" > > > > > total 35M > > > > > > > > > > You can observe that the size of this data4 path much larger than the > > > > > other paths. Is that correct? I thought that the sizes of the path > > > > > would be the same or very close. > > > > > > -- > > > Gustavo Maia > > -- > Gustavo Maia -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 31, 2011, 5:43pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/8 "2011-10-31T17:43:42Z")
Strange..., nothing really fixed. I added some logging to 0.18 branch that might help, can you try with a custom build of it, and set: env to TRACE? On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > The following test was done using the 0.18.0 version of the ES. > Was set up multiple path, in multiple drives. > Below is the sizes of the data path. > > 1. > > /dev/sdb 414G 199M 393G 1% /mnt > 2) > /dev/sdc 414G 199M 393G 1% /media/dev1 > 3) > /dev/sdd 414G 15G 378G 4% /media/dev2 > 4) > /dev/sde 414G 199M 393G 1% /media/dev3 > > This distribution of indices that correct or is it a bug that was fixed in > version0.18.1? > > Earlier I was talking about a test with the version 0.18.1, using the same > drive with multiple path. > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > OK. If I have 4 drives with the same size (150GB each drive). When > > I index 1million documents all drives will have the same size? > > > > I'm doing tests on my machine and then indexing at amazon. > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > The check for the size of where to place the data is based on the free > > > space available per "drive", so having multiple data locations on the same > > > drive makes little sense. > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > Is the same Driver. > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia \< > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > > > Hello, > > > > > > I'm using version 0.18.1 and configuring path.data as follows: > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > After 65,000 index doc, I see the size of each path and I was a > > > > > > little > > > > > > surprised the lengths of each path were not similar. Below the size > > > > > > of > > > > > > each path found. > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > total 350M > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > 34m total > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > total 60M > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > total 35M > > > > > > > > > > > > You can observe that the size of this data4 path much larger than > > > > > > the > > > > > > other paths. Is that correct? I thought that the sizes of the path > > > > > > would be the same or very close. > > > > > > > > -- > > > > Gustavo Maia > > > > -- > > Gustavo Maia > > -- > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [October 31, 2011, 7:53pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/9 "2011-10-31T19:53:29Z")
hi, Shay Yes i can do it. gateway: DEBUG org.apache: WARN discovery: TRACE Only put it in the config/logging.yml ?? 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > Strange..., nothing really fixed. I added some logging to 0.18 branch that might help, can you try with a custom build of it, and set: env to TRACE? > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > The following test was done using the 0.18.0 version of the ES. > > Was set up multiple path, in multiple drives. > > Below is the sizes of the data path. > > > > 1. > > > > /dev/sdb 414G 199M 393G 1% /mnt > > 2) > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > 3) > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > 4) > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > This distribution of indices that correct or is it a bug that was fixed in version0.18.1? > > > > Earlier I was talking about a test with the version 0.18.1, using the same drive with multiple path. > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > OK. If I have 4 drives with the same size (150GB each drive). When I index 1million documents all drives will have the same size? > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > The check for the size of where to place the data is based on the free space available per "drive", so having multiple data locations on the same drive makes little sense. > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > Is the same Driver. > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > Hello, > > > > > > > I'm using version 0.18.1 and configuring path.data as follows: > > > > > > > > > > > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > > > > > > After 65,000 index doc, I see the size of each path and I was a little > > > > > > > surprised the lengths of each path were not similar. Below the size of > > > > > > > each path found. > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > > total 350M > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > > 34m total > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > > total 60M > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > / | grep "total" > > > > > > > total 35M > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger than the > > > > > > > other paths. Is that correct? I thought that the sizes of the path > > > > > > > would be the same or very close. > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia > > > > -- > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 1, 2011, 5:35pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/10 "2011-11-01T17:35:04Z")
yes, but you will need to build a version of elasticsearch... . Should be simple, download / clone the 0.18 branch, and run "gradlew release". On Mon, Oct 31, 2011 at 9:53 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > hi, Shay > > Yes i can do it. > > gateway: DEBUG > org.apache: WARN > discovery: TRACE > > Only put it in the config/logging.yml ?? > > 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > Strange..., nothing really fixed. I added some logging to 0.18 branch > > that might help, can you try with a custom build of it, and set: env to > > TRACE? > > > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > wrote: > > > > > The following test was done using the 0.18.0 version of the ES. > > > Was set up multiple path, in multiple drives. > > > Below is the sizes of the data path. > > > > > > 1. > > > > > > /dev/sdb 414G 199M 393G 1% /mnt > > > 2) > > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > > 3) > > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > > 4) > > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > > > This distribution of indices that correct or is it a bug that was fixed > > > in version0.18.1? > > > > > > Earlier I was talking about a test with the version 0.18.1, using the > > > same drive with multiple path. > > > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > > OK. If I have 4 drives with the same size (150GB each drive). When > > > > I index 1million documents all drives will have the same size? > > > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > The check for the size of where to place the data is based on the > > > > > free space available per "drive", so having multiple data locations on the > > > > > same drive makes little sense. > > > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia \< > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > > > Is the same Driver. > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia \< > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > I'm using version 0.18.1 and configuring path.data as follows: > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > After 65,000 index doc, I see the size of each path and I was a > > > > > > > > little > > > > > > > > surprised the lengths of each path were not similar. Below the > > > > > > > > size of > > > > > > > > each path found. > > > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > total 350M > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > 34m total > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > total 60M > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > total 35M > > > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger > > > > > > > > than the > > > > > > > > other paths. Is that correct? I thought that the sizes of the > > > > > > > > path > > > > > > > > would be the same or very close. > > > > > > > > > > > > -- > > > > > > Gustavo Maia > > > > > > > > -- > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia > > -- > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [November 1, 2011, 8:48pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/11 "2011-11-01T20:48:32Z")
Please check, if it is correct. My logging.yml file: rootLogger: INFO, console, file logger: # log action execution errors for easier debugging action: DEBUG # reduce the logging for aws, too much is logged under the default INFO com.amazonaws: WARN # gateway gateway: DEBUG index.gateway: DEBUG # peer shard recovery indices.recovery: DEBUG # discovery discovery: TRACE appender: console: type: console layout: type: consolePattern conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" file: type: dailyRollingFile file: ${path.logs}/${cluster.name}.log datePattern: "'.'yyyy-MM-dd" layout: type: pattern conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" Below my log, but i only see log about save in the s3. [2011-11-01 18:43:51,141][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [335ms] index : version [1319483349133], number\_of\_files [1] with total\_size [5.4kb], took [42ms] translog : id [1319482375161], number\_of\_operations [649], took [72ms] [2011-11-01 18:43:59,124][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:43:59,715][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [591ms] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [57ms] translog : id [1319482375057], number\_of\_operations [555], took [286ms] [2011-11-01 18:44:01,141][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:01,715][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [574ms] index : version [1319483349133], number\_of\_files [1] with total\_size [5.4kb], took [51ms] translog : id [1319482375161], number\_of\_operations [517], took [95ms] [2011-11-01 18:44:09,715][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:44:10,293][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [577ms] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [49ms] translog : id [1319482375057], number\_of\_operations [344], took [74ms] [2011-11-01 18:44:11,715][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:12,039][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [323ms] index : version [1319483349133], number\_of\_files [1] with total\_size [5.4kb], took [52ms] translog : id [1319482375161], number\_of\_operations [293], took [48ms] [2011-11-01 18:44:20,294][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:44:20,865][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [571ms] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [276ms] translog : id [1319482375057], number\_of\_operations [524], took [117ms] [2011-11-01 18:44:22,039][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:22,354][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [313ms] index : version [1319483349133], number\_of\_files [1] with total\_size [5.4kb], took [32ms] translog : id [1319482375161], number\_of\_operations [610], took [76ms] [2011-11-01 18:44:30,865][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:44:31,204][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [338ms] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [38ms] translog : id [1319482375057], number\_of\_operations [655], took [109ms] [2011-11-01 18:44:32,354][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:35,822][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [3.4s] index : version [1319483349493], number\_of\_files [98] with total\_size [5.8mb], took [1.3s] translog : id [1319482375162], number\_of\_operations [49], took [61ms] [2011-11-01 18:44:41,205][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:44:41,694][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [489ms] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [45ms] translog : id [1319482375057], number\_of\_operations [421], took [112ms] [2011-11-01 18:44:45,822][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:46,438][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [616ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [117ms] translog : id [1319482375162], number\_of\_operations [500], took [63ms] [2011-11-01 18:44:51,695][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:44:53,314][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [1.6s] index : version [1319483347414], number\_of\_files [1] with total\_size [6kb], took [38ms] translog : id [1319482375057], number\_of\_operations [413], took [64ms] [2011-11-01 18:44:56,438][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:44:56,812][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [373ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [41ms] translog : id [1319482375162], number\_of\_operations [551], took [94ms] [2011-11-01 18:45:03,315][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:45:06,813][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:45:07,186][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [373ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [42ms] translog : id [1319482375162], number\_of\_operations [450], took [74ms] [2011-11-01 18:45:11,812][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [8.4s] index : version [1319483347731], number\_of\_files [114] with total\_size [17.9mb], took [2s] translog : id [1319482375058], number\_of\_operations [284], took [90ms] [2011-11-01 18:45:17,186][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:45:17,637][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [450ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [44ms] translog : id [1319482375162], number\_of\_operations [458], took [158ms] [2011-11-01 18:45:21,813][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:45:22,795][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [982ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [48ms] translog : id [1319482375058], number\_of\_operations [743], took [90ms] [2011-11-01 18:45:27,638][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:45:28,053][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [415ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [76ms] translog : id [1319482375162], number\_of\_operations [392], took [98ms] [2011-11-01 18:45:32,796][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:45:33,144][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [348ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [39ms] translog : id [1319482375058], number\_of\_operations [510], took [91ms] [2011-11-01 18:45:38,054][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:45:39,410][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [1.3s] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [42ms] translog : id [1319482375162], number\_of\_operations [564], took [88ms] [2011-11-01 18:45:43,145][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:45:43,666][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [521ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [59ms] translog : id [1319482375058], number\_of\_operations [568], took [83ms] [2011-11-01 18:45:49,410][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:45:50,025][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [615ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [55ms] translog : id [1319482375162], number\_of\_operations [587], took [308ms] [2011-11-01 18:45:53,666][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:45:54,098][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [431ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [41ms] translog : id [1319482375058], number\_of\_operations [545], took [88ms] [2011-11-01 18:46:00,025][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:46:00,535][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [509ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [85ms] translog : id [1319482375162], number\_of\_operations [498], took [81ms] [2011-11-01 18:46:04,098][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:46:04,424][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [325ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [38ms] translog : id [1319482375058], number\_of\_operations [379], took [83ms] [2011-11-01 18:46:10,535][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] ... [2011-11-01 18:46:11,168][DEBUG][index.gateway] [Black Fox] [topicoindex][1] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], took [632ms] index : version [1319483349493], number\_of\_files [1] with total\_size [6.3kb], took [40ms] translog : id [1319482375162], number\_of\_operations [302], took [188ms] [2011-11-01 18:46:14,424][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] ... [2011-11-01 18:46:15,062][DEBUG][index.gateway] [Black Fox] [topicoindex][0] snapshot (scheduled) completed to s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], took [637ms] index : version [1319483347731], number\_of\_files [1] with total\_size [5.9kb], took [139ms] translog : id [1319482375058], number\_of\_operations [399], took [81ms] 2011/11/1 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com): > yes, but you will need to build a version of elasticsearch... . Should be > simple, download / clone the 0.18 branch, and run "gradlew release". > > On Mon, Oct 31, 2011 at 9:53 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > wrote: > > > hi, Shay > > > > Yes i can do it. > > > > gateway: DEBUG > > org.apache: WARN > > discovery: TRACE > > > > Only put it in the config/logging.yml ?? > > > > 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > Strange..., nothing really fixed. I added some logging to 0.18 branch > > > that might help, can you try with a custom build of it, and set: env to > > > TRACE? > > > > > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > The following test was done using the 0.18.0 version of the ES. > > > > Was set up multiple path, in multiple drives. > > > > Below is the sizes of the data path. > > > > > > > > 1. > > > > > > > > /dev/sdb 414G 199M 393G 1% /mnt > > > > 2) > > > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > > > 3) > > > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > > > 4) > > > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > > > > > This distribution of indices that correct or is it a bug that was fixed > > > > in version0.18.1? > > > > > > > > Earlier I was talking about a test with the version 0.18.1, using the > > > > same drive with multiple path. > > > > > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > > > > OK. If I have 4 drives with the same size (150GB each drive). When > > > > > I index 1million documents all drives will have the same size? > > > > > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > The check for the size of where to place the data is based on the > > > > > > free space available per "drive", so having multiple data locations on the > > > > > > same drive makes little sense. > > > > > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > Is the same Driver. > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > I'm using version 0.18.1 and configuring path.data as follows: > > > > > > > > > > > > > > > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > > > > > > > > > > After 65,000 index doc, I see the size of each path and I was a > > > > > > > > > little > > > > > > > > > surprised the lengths of each path were not similar. Below the > > > > > > > > > size of > > > > > > > > > each path found. > > > > > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > > total 350M > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > > 34m total > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > > total 60M > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > / | grep "total" > > > > > > > > > total 35M > > > > > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger > > > > > > > > > than the > > > > > > > > > other paths. Is that correct? I thought that the sizes of the > > > > > > > > > path > > > > > > > > > would be the same or very close. > > > > > > > > > > > > > > -- > > > > > > > Gustavo Maia > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > > > -- > > > > Gustavo Maia > > > > -- > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 1, 2011, 10:06pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/12 "2011-11-01T22:06:57Z")
CAn you set in the logging.yml the "env" component to TRACE, and leave all the rest with teh default values, and then gist hte log output? On Tue, Nov 1, 2011 at 10:48 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > Please check, if it is correct. > > My logging.yml file: > > rootLogger: INFO, console, file > logger: > > # log action execution errors for easier debugging > > action: DEBUG > > # reduce the logging for aws, too much is logged under the default INFO > > com.amazonaws: WARN > > # gateway > > gateway: DEBUG > index.gateway: DEBUG > > # peer shard recovery > > indices.recovery: DEBUG > > # discovery > > discovery: TRACE > > appender: > console: > type: console > layout: > type: consolePattern > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > file: > type: dailyRollingFile > file: ${path.logs}/${cluster.name}.log > datePattern: "'.'yyyy-MM-dd" > layout: > type: pattern > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > Below my log, but i only see log about save in the s3. > > [2011-11-01 18:43:51,141][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [335ms] > index : version [1319483349133], number\_of\_files [1] with > total\_size [5.4kb], took [42ms] > translog : id [1319482375161], number\_of\_operations [649], took > [72ms] > [2011-11-01 18:43:59,124][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:43:59,715][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [591ms] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [57ms] > translog : id [1319482375057], number\_of\_operations [555], took > [286ms] > [2011-11-01 18:44:01,141][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:01,715][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [574ms] > index : version [1319483349133], number\_of\_files [1] with > total\_size [5.4kb], took [51ms] > translog : id [1319482375161], number\_of\_operations [517], took > [95ms] > [2011-11-01 18:44:09,715][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:44:10,293][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [577ms] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [49ms] > translog : id [1319482375057], number\_of\_operations [344], took > [74ms] > [2011-11-01 18:44:11,715][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:12,039][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [323ms] > index : version [1319483349133], number\_of\_files [1] with > total\_size [5.4kb], took [52ms] > translog : id [1319482375161], number\_of\_operations [293], took > [48ms] > [2011-11-01 18:44:20,294][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:44:20,865][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [571ms] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [276ms] > translog : id [1319482375057], number\_of\_operations [524], took > [117ms] > [2011-11-01 18:44:22,039][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:22,354][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [313ms] > index : version [1319483349133], number\_of\_files [1] with > total\_size [5.4kb], took [32ms] > translog : id [1319482375161], number\_of\_operations [610], took > [76ms] > [2011-11-01 18:44:30,865][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:44:31,204][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [338ms] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [38ms] > translog : id [1319482375057], number\_of\_operations [655], took > [109ms] > [2011-11-01 18:44:32,354][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:35,822][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [3.4s] > index : version [1319483349493], number\_of\_files [98] with > total\_size [5.8mb], took [1.3s] > translog : id [1319482375162], number\_of\_operations [49], took > [61ms] > [2011-11-01 18:44:41,205][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:44:41,694][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [489ms] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [45ms] > translog : id [1319482375057], number\_of\_operations [421], took > [112ms] > [2011-11-01 18:44:45,822][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:46,438][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [616ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [117ms] > translog : id [1319482375162], number\_of\_operations [500], took > [63ms] > [2011-11-01 18:44:51,695][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:44:53,314][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [1.6s] > index : version [1319483347414], number\_of\_files [1] with > total\_size [6kb], took [38ms] > translog : id [1319482375057], number\_of\_operations [413], took > [64ms] > [2011-11-01 18:44:56,438][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:44:56,812][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [373ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [41ms] > translog : id [1319482375162], number\_of\_operations [551], took > [94ms] > [2011-11-01 18:45:03,315][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:45:06,813][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:45:07,186][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [373ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [42ms] > translog : id [1319482375162], number\_of\_operations [450], took > [74ms] > [2011-11-01 18:45:11,812][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [8.4s] > index : version [1319483347731], number\_of\_files [114] with > total\_size [17.9mb], took [2s] > translog : id [1319482375058], number\_of\_operations [284], took > [90ms] > [2011-11-01 18:45:17,186][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:45:17,637][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [450ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [44ms] > translog : id [1319482375162], number\_of\_operations [458], took > [158ms] > [2011-11-01 18:45:21,813][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:45:22,795][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [982ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [48ms] > translog : id [1319482375058], number\_of\_operations [743], took > [90ms] > [2011-11-01 18:45:27,638][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:45:28,053][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [415ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [76ms] > translog : id [1319482375162], number\_of\_operations [392], took > [98ms] > [2011-11-01 18:45:32,796][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:45:33,144][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [348ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [39ms] > translog : id [1319482375058], number\_of\_operations [510], took > [91ms] > [2011-11-01 18:45:38,054][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:45:39,410][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [1.3s] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [42ms] > translog : id [1319482375162], number\_of\_operations [564], took > [88ms] > [2011-11-01 18:45:43,145][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:45:43,666][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [521ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [59ms] > translog : id [1319482375058], number\_of\_operations [568], took > [83ms] > [2011-11-01 18:45:49,410][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:45:50,025][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [615ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [55ms] > translog : id [1319482375162], number\_of\_operations [587], took > [308ms] > [2011-11-01 18:45:53,666][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:45:54,098][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [431ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [41ms] > translog : id [1319482375058], number\_of\_operations [545], took > [88ms] > [2011-11-01 18:46:00,025][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:46:00,535][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [509ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [85ms] > translog : id [1319482375162], number\_of\_operations [498], took > [81ms] > [2011-11-01 18:46:04,098][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:46:04,424][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [325ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [38ms] > translog : id [1319482375058], number\_of\_operations [379], took > [83ms] > [2011-11-01 18:46:10,535][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > ... > [2011-11-01 18:46:11,168][DEBUG][index.gateway] [Black > Fox] [topicoindex][1] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > took [632ms] > index : version [1319483349493], number\_of\_files [1] with > total\_size [6.3kb], took [40ms] > translog : id [1319482375162], number\_of\_operations [302], took > [188ms] > [2011-11-01 18:46:14,424][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > ... > [2011-11-01 18:46:15,062][DEBUG][index.gateway] [Black > Fox] [topicoindex][0] snapshot (scheduled) completed to > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > took [637ms] > index : version [1319483347731], number\_of\_files [1] with > total\_size [5.9kb], took [139ms] > translog : id [1319482375058], number\_of\_operations [399], took > [81ms] > > 2011/11/1 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com): > > > yes, but you will need to build a version of elasticsearch... . Should be > > simple, download / clone the 0.18 branch, and run "gradlew release". > > > > On Mon, Oct 31, 2011 at 9:53 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > wrote: > > > > > hi, Shay > > > > > > Yes i can do it. > > > > > > gateway: DEBUG > > > org.apache: WARN > > > discovery: TRACE > > > > > > Only put it in the config/logging.yml ?? > > > > > > 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > Strange..., nothing really fixed. I added some logging to 0.18 branch > > > > that might help, can you try with a custom build of it, and set: env > > > > to > > > > TRACE? > > > > > > > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia \< > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> > > > > wrote: > > > > > > > > > The following test was done using the 0.18.0 version of the ES. > > > > > Was set up multiple path, in multiple drives. > > > > > Below is the sizes of the data path. > > > > > > > > > > 1. > > > > > > > > > > /dev/sdb 414G 199M 393G 1% /mnt > > > > > 2) > > > > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > > > > 3) > > > > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > > > > 4) > > > > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > > > > > > > This distribution of indices that correct or is it a bug > > > > > that was fixed > > > > > in version0.18.1? > > > > > > > > > > Earlier I was talking about a test with the version 0.18.1, using the > > > > > same drive with multiple path. > > > > > > > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > > > > > > OK. If I have 4 drives with the same size (150GB each drive). When > > > > > > I index 1million documents all drives will have the same size? > > > > > > > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > The check for the size of where to place the data is based on the > > > > > > > free space available per "drive", so having multiple data > > > > > > > locations on the > > > > > > > same drive makes little sense. > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > Is the same Driver. > > > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia > > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > I'm using version 0.18.1 and configuring path.data as follows: > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > > > After 65,000 index doc, I see the size of each path and I was > > > > > > > > > > a > > > > > > > > > > little > > > > > > > > > > surprised the lengths of each path were not similar. Below the > > > > > > > > > > size of > > > > > > > > > > each path found. > > > > > > > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > / | grep "total" > > > > > > > > > > total 350M > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > / | grep "total" > > > > > > > > > > 34m total > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > / | grep "total" > > > > > > > > > > total 60M > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > / | grep "total" > > > > > > > > > > total 35M > > > > > > > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger > > > > > > > > > > than the > > > > > > > > > > other paths. Is that correct? I thought that the sizes of the > > > > > > > > > > path > > > > > > > > > > would be the same or very close. > > > > > > > > > > > > > > > > -- > > > > > > > > Gustavo Maia > > > > > > > > > > > > -- > > > > > > Gustavo Maia > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia > > -- > Gustavo Maia ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 2, 2011, 9:46pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/13 "2011-11-02T21:46:56Z")
Heya, found the problem and fixed it in 0.18 and master branch. Will release 0.18.3 sometime early next week with the fix. On Wed, Nov 2, 2011 at 12:06 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > CAn you set in the logging.yml the "env" component to TRACE, and leave all > the rest with teh default values, and then gist hte log output? > > On Tue, Nov 1, 2011 at 10:48 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)wrote: > > > Please check, if it is correct. > > > > My logging.yml file: > > > > rootLogger: INFO, console, file > > logger: > > > > # log action execution errors for easier debugging > > > > action: DEBUG > > > > # reduce the logging for aws, too much is logged under the default INFO > > > > com.amazonaws: WARN > > > > # gateway > > > > gateway: DEBUG > > index.gateway: DEBUG > > > > # peer shard recovery > > > > indices.recovery: DEBUG > > > > # discovery > > > > discovery: TRACE > > > > appender: > > console: > > type: console > > layout: > > type: consolePattern > > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > > > file: > > type: dailyRollingFile > > file: ${path.logs}/${cluster.name}.log > > datePattern: "'.'yyyy-MM-dd" > > layout: > > type: pattern > > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > > > Below my log, but i only see log about save in the s3. > > > > [2011-11-01 18:43:51,141][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [335ms] > > index : version [1319483349133], number\_of\_files [1] with > > total\_size [5.4kb], took [42ms] > > translog : id [1319482375161], number\_of\_operations [649], took > > [72ms] > > [2011-11-01 18:43:59,124][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:43:59,715][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [591ms] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [57ms] > > translog : id [1319482375057], number\_of\_operations [555], took > > [286ms] > > [2011-11-01 18:44:01,141][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:01,715][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [574ms] > > index : version [1319483349133], number\_of\_files [1] with > > total\_size [5.4kb], took [51ms] > > translog : id [1319482375161], number\_of\_operations [517], took > > [95ms] > > [2011-11-01 18:44:09,715][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:44:10,293][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [577ms] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [49ms] > > translog : id [1319482375057], number\_of\_operations [344], took > > [74ms] > > [2011-11-01 18:44:11,715][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:12,039][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [323ms] > > index : version [1319483349133], number\_of\_files [1] with > > total\_size [5.4kb], took [52ms] > > translog : id [1319482375161], number\_of\_operations [293], took > > [48ms] > > [2011-11-01 18:44:20,294][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:44:20,865][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [571ms] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [276ms] > > translog : id [1319482375057], number\_of\_operations [524], took > > [117ms] > > [2011-11-01 18:44:22,039][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:22,354][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [313ms] > > index : version [1319483349133], number\_of\_files [1] with > > total\_size [5.4kb], took [32ms] > > translog : id [1319482375161], number\_of\_operations [610], took > > [76ms] > > [2011-11-01 18:44:30,865][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:44:31,204][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [338ms] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [38ms] > > translog : id [1319482375057], number\_of\_operations [655], took > > [109ms] > > [2011-11-01 18:44:32,354][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:35,822][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [3.4s] > > index : version [1319483349493], number\_of\_files [98] with > > total\_size [5.8mb], took [1.3s] > > translog : id [1319482375162], number\_of\_operations [49], took > > [61ms] > > [2011-11-01 18:44:41,205][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:44:41,694][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [489ms] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [45ms] > > translog : id [1319482375057], number\_of\_operations [421], took > > [112ms] > > [2011-11-01 18:44:45,822][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:46,438][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [616ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [117ms] > > translog : id [1319482375162], number\_of\_operations [500], took > > [63ms] > > [2011-11-01 18:44:51,695][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:44:53,314][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [1.6s] > > index : version [1319483347414], number\_of\_files [1] with > > total\_size [6kb], took [38ms] > > translog : id [1319482375057], number\_of\_operations [413], took > > [64ms] > > [2011-11-01 18:44:56,438][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:44:56,812][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [373ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [41ms] > > translog : id [1319482375162], number\_of\_operations [551], took > > [94ms] > > [2011-11-01 18:45:03,315][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:45:06,813][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:45:07,186][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [373ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [42ms] > > translog : id [1319482375162], number\_of\_operations [450], took > > [74ms] > > [2011-11-01 18:45:11,812][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [8.4s] > > index : version [1319483347731], number\_of\_files [114] with > > total\_size [17.9mb], took [2s] > > translog : id [1319482375058], number\_of\_operations [284], took > > [90ms] > > [2011-11-01 18:45:17,186][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:45:17,637][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [450ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [44ms] > > translog : id [1319482375162], number\_of\_operations [458], took > > [158ms] > > [2011-11-01 18:45:21,813][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:45:22,795][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [982ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [48ms] > > translog : id [1319482375058], number\_of\_operations [743], took > > [90ms] > > [2011-11-01 18:45:27,638][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:45:28,053][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [415ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [76ms] > > translog : id [1319482375162], number\_of\_operations [392], took > > [98ms] > > [2011-11-01 18:45:32,796][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:45:33,144][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [348ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [39ms] > > translog : id [1319482375058], number\_of\_operations [510], took > > [91ms] > > [2011-11-01 18:45:38,054][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:45:39,410][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [1.3s] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [42ms] > > translog : id [1319482375162], number\_of\_operations [564], took > > [88ms] > > [2011-11-01 18:45:43,145][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:45:43,666][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [521ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [59ms] > > translog : id [1319482375058], number\_of\_operations [568], took > > [83ms] > > [2011-11-01 18:45:49,410][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:45:50,025][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [615ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [55ms] > > translog : id [1319482375162], number\_of\_operations [587], took > > [308ms] > > [2011-11-01 18:45:53,666][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:45:54,098][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [431ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [41ms] > > translog : id [1319482375058], number\_of\_operations [545], took > > [88ms] > > [2011-11-01 18:46:00,025][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:46:00,535][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [509ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [85ms] > > translog : id [1319482375162], number\_of\_operations [498], took > > [81ms] > > [2011-11-01 18:46:04,098][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:46:04,424][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [325ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [38ms] > > translog : id [1319482375058], number\_of\_operations [379], took > > [83ms] > > [2011-11-01 18:46:10,535][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > ... > > [2011-11-01 18:46:11,168][DEBUG][index.gateway] [Black > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > took [632ms] > > index : version [1319483349493], number\_of\_files [1] with > > total\_size [6.3kb], took [40ms] > > translog : id [1319482375162], number\_of\_operations [302], took > > [188ms] > > [2011-11-01 18:46:14,424][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > ... > > [2011-11-01 18:46:15,062][DEBUG][index.gateway] [Black > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > took [637ms] > > index : version [1319483347731], number\_of\_files [1] with > > total\_size [5.9kb], took [139ms] > > translog : id [1319482375058], number\_of\_operations [399], took > > [81ms] > > > > 2011/11/1 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com): > > > > > yes, but you will need to build a version of elasticsearch... . Should > > > be > > > simple, download / clone the 0.18 branch, and run "gradlew release". > > > > > > On Mon, Oct 31, 2011 at 9:53 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > wrote: > > > > > > > hi, Shay > > > > > > > > Yes i can do it. > > > > > > > > gateway: DEBUG > > > > org.apache: WARN > > > > discovery: TRACE > > > > > > > > Only put it in the config/logging.yml ?? > > > > > > > > 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > Strange..., nothing really fixed. I added some logging to 0.18 branch > > > > > that might help, can you try with a custom build of it, and set: env > > > > > to > > > > > TRACE? > > > > > > > > > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia \< > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com)\> > > > > > wrote: > > > > > > > > > > > The following test was done using the 0.18.0 version of the ES. > > > > > > Was set up multiple path, in multiple drives. > > > > > > Below is the sizes of the data path. > > > > > > > > > > > > 1. > > > > > > > > > > > > /dev/sdb 414G 199M 393G 1% /mnt > > > > > > 2) > > > > > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > > > > > 3) > > > > > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > > > > > 4) > > > > > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > > > > > > > > > This distribution of indices that correct or is it a bug > > > > > > that was fixed > > > > > > in version0.18.1? > > > > > > > > > > > > Earlier I was talking about a test with > > > > > > the version 0.18.1, using the > > > > > > same drive with multiple path. > > > > > > > > > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > > > > > > > > OK. If I have 4 drives with the same size (150GB each drive). When > > > > > > > I index 1million documents all drives will have the same size? > > > > > > > > > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > The check for the size of where to place the data is based on > > > > > > > > the > > > > > > > > free space available per "drive", so having multiple data > > > > > > > > locations on the > > > > > > > > same drive makes little sense. > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > Is the same Driver. > > > > > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia > > > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > I'm using version 0.18.1 and configuring path.data as > > > > > > > > > > > follows: > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > > > > > After 65,000 index doc, I see the size of each path and I > > > > > > > > > > > was a > > > > > > > > > > > little > > > > > > > > > > > surprised the lengths of each path were not similar. Below > > > > > > > > > > > the > > > > > > > > > > > size of > > > > > > > > > > > each path found. > > > > > > > > > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > total 350M > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > 34m total > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > total 60M > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > total 35M > > > > > > > > > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger > > > > > > > > > > > than the > > > > > > > > > > > other paths. Is that correct? I thought that the sizes of > > > > > > > > > > > the > > > > > > > > > > > path > > > > > > > > > > > would be the same or very close. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Gustavo Maia > > > > > > > > > > > > > > -- > > > > > > > Gustavo Maia > > > > > > > > > > > > -- > > > > > > Gustavo Maia > > > > > > > > -- > > > > Gustavo Maia > > > > -- > > Gustavo Maia ---
**Author:** ![Gustavo\_Maia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gustavo_maia/32/2183_2.png) [@Gustavo\_Maia](https://discuss.elastic.co/u/Gustavo_Maia)\ **Post date:** [November 2, 2011, 11:53pm UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/14 "2011-11-02T23:53:04Z")
Great..... thanks so much. I will download and test, if i have some problem i tell you. \thanks... 2011/11/2 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com): > Heya, found the problem and fixed it in 0.18 and master branch. Will release > 0.18.3 sometime early next week with the fix. > > On Wed, Nov 2, 2011 at 12:06 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > CAn you set in the logging.yml the "env" component to TRACE, and leave all > > the rest with teh default values, and then gist hte log output? > > > > On Tue, Nov 1, 2011 at 10:48 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > wrote: > > > > > Please check, if it is correct. > > > > > > My logging.yml file: > > > > > > rootLogger: INFO, console, file > > > logger: > > > > > > # log action execution errors for easier debugging > > > > > > action: DEBUG > > > > > > # reduce the logging for aws, too much is logged under the default INFO > > > > > > com.amazonaws: WARN > > > > > > # gateway > > > > > > gateway: DEBUG > > > index.gateway: DEBUG > > > > > > # peer shard recovery > > > > > > indices.recovery: DEBUG > > > > > > # discovery > > > > > > discovery: TRACE > > > > > > appender: > > > console: > > > type: console > > > layout: > > > type: consolePattern > > > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > > > > > file: > > > type: dailyRollingFile > > > file: ${path.logs}/${cluster.name}.log > > > datePattern: "'.'yyyy-MM-dd" > > > layout: > > > type: pattern > > > conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n" > > > > > > Below my log, but i only see log about save in the s3. > > > > > > [2011-11-01 18:43:51,141][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [335ms] > > > index : version [1319483349133], number\_of\_files [1] with > > > total\_size [5.4kb], took [42ms] > > > translog : id [1319482375161], number\_of\_operations [649], took > > > [72ms] > > > [2011-11-01 18:43:59,124][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:43:59,715][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [591ms] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [57ms] > > > translog : id [1319482375057], number\_of\_operations [555], took > > > [286ms] > > > [2011-11-01 18:44:01,141][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:01,715][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [574ms] > > > index : version [1319483349133], number\_of\_files [1] with > > > total\_size [5.4kb], took [51ms] > > > translog : id [1319482375161], number\_of\_operations [517], took > > > [95ms] > > > [2011-11-01 18:44:09,715][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:44:10,293][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [577ms] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [49ms] > > > translog : id [1319482375057], number\_of\_operations [344], took > > > [74ms] > > > [2011-11-01 18:44:11,715][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:12,039][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [323ms] > > > index : version [1319483349133], number\_of\_files [1] with > > > total\_size [5.4kb], took [52ms] > > > translog : id [1319482375161], number\_of\_operations [293], took > > > [48ms] > > > [2011-11-01 18:44:20,294][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:44:20,865][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [571ms] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [276ms] > > > translog : id [1319482375057], number\_of\_operations [524], took > > > [117ms] > > > [2011-11-01 18:44:22,039][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:22,354][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [313ms] > > > index : version [1319483349133], number\_of\_files [1] with > > > total\_size [5.4kb], took [32ms] > > > translog : id [1319482375161], number\_of\_operations [610], took > > > [76ms] > > > [2011-11-01 18:44:30,865][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:44:31,204][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [338ms] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [38ms] > > > translog : id [1319482375057], number\_of\_operations [655], took > > > [109ms] > > > [2011-11-01 18:44:32,354][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:35,822][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [3.4s] > > > index : version [1319483349493], number\_of\_files [98] with > > > total\_size [5.8mb], took [1.3s] > > > translog : id [1319482375162], number\_of\_operations [49], took > > > [61ms] > > > [2011-11-01 18:44:41,205][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:44:41,694][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [489ms] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [45ms] > > > translog : id [1319482375057], number\_of\_operations [421], took > > > [112ms] > > > [2011-11-01 18:44:45,822][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:46,438][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [616ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [117ms] > > > translog : id [1319482375162], number\_of\_operations [500], took > > > [63ms] > > > [2011-11-01 18:44:51,695][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:44:53,314][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [1.6s] > > > index : version [1319483347414], number\_of\_files [1] with > > > total\_size [6kb], took [38ms] > > > translog : id [1319482375057], number\_of\_operations [413], took > > > [64ms] > > > [2011-11-01 18:44:56,438][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:44:56,812][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [373ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [41ms] > > > translog : id [1319482375162], number\_of\_operations [551], took > > > [94ms] > > > [2011-11-01 18:45:03,315][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:45:06,813][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:45:07,186][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [373ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [42ms] > > > translog : id [1319482375162], number\_of\_operations [450], took > > > [74ms] > > > [2011-11-01 18:45:11,812][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [8.4s] > > > index : version [1319483347731], number\_of\_files [114] with > > > total\_size [17.9mb], took [2s] > > > translog : id [1319482375058], number\_of\_operations [284], took > > > [90ms] > > > [2011-11-01 18:45:17,186][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:45:17,637][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [450ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [44ms] > > > translog : id [1319482375162], number\_of\_operations [458], took > > > [158ms] > > > [2011-11-01 18:45:21,813][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:45:22,795][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [982ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [48ms] > > > translog : id [1319482375058], number\_of\_operations [743], took > > > [90ms] > > > [2011-11-01 18:45:27,638][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:45:28,053][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [415ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [76ms] > > > translog : id [1319482375162], number\_of\_operations [392], took > > > [98ms] > > > [2011-11-01 18:45:32,796][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:45:33,144][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [348ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [39ms] > > > translog : id [1319482375058], number\_of\_operations [510], took > > > [91ms] > > > [2011-11-01 18:45:38,054][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:45:39,410][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [1.3s] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [42ms] > > > translog : id [1319482375162], number\_of\_operations [564], took > > > [88ms] > > > [2011-11-01 18:45:43,145][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:45:43,666][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [521ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [59ms] > > > translog : id [1319482375058], number\_of\_operations [568], took > > > [83ms] > > > [2011-11-01 18:45:49,410][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:45:50,025][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [615ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [55ms] > > > translog : id [1319482375162], number\_of\_operations [587], took > > > [308ms] > > > [2011-11-01 18:45:53,666][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:45:54,098][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [431ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [41ms] > > > translog : id [1319482375058], number\_of\_operations [545], took > > > [88ms] > > > [2011-11-01 18:46:00,025][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:46:00,535][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [509ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [85ms] > > > translog : id [1319482375162], number\_of\_operations [498], took > > > [81ms] > > > [2011-11-01 18:46:04,098][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:46:04,424][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [325ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [38ms] > > > translog : id [1319482375058], number\_of\_operations [379], took > > > [83ms] > > > [2011-11-01 18:46:10,535][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1] > > > ... > > > [2011-11-01 18:46:11,168][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][1] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][1], > > > took [632ms] > > > index : version [1319483349493], number\_of\_files [1] with > > > total\_size [6.3kb], took [40ms] > > > translog : id [1319482375162], number\_of\_operations [302], took > > > [188ms] > > > [2011-11-01 18:46:14,424][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0] > > > ... > > > [2011-11-01 18:46:15,062][DEBUG][index.gateway] [Black > > > Fox] [topicoindex][0] snapshot (scheduled) completed to > > > s3://jusbrasil-index/[jusbrasil-search-cluster][indices][topicoindex][0], > > > took [637ms] > > > index : version [1319483347731], number\_of\_files [1] with > > > total\_size [5.9kb], took [139ms] > > > translog : id [1319482375058], number\_of\_operations [399], took > > > [81ms] > > > > > > 2011/11/1 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com): > > > > > > > yes, but you will need to build a version of elasticsearch... . Should > > > > be > > > > simple, download / clone the 0.18 branch, and run "gradlew release". > > > > > > > > On Mon, Oct 31, 2011 at 9:53 PM, Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > wrote: > > > > > > > > > hi, Shay > > > > > > > > > > Yes i can do it. > > > > > > > > > > gateway: DEBUG > > > > > org.apache: WARN > > > > > discovery: TRACE > > > > > > > > > > Only put it in the config/logging.yml ?? > > > > > > > > > > 2011/10/31 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > Strange..., nothing really fixed. I added some logging to 0.18 > > > > > > branch > > > > > > that might help, can you try with a custom build of it, and set: env > > > > > > to > > > > > > TRACE? > > > > > > > > > > > > On Fri, Oct 28, 2011 at 4:48 AM, Gustavo Maia > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > The following test was done using the 0.18.0 version of the ES. > > > > > > > Was set up multiple path, in multiple drives. > > > > > > > Below is the sizes of the data path. > > > > > > > > > > > > > > 1. > > > > > > > > > > > > > > /dev/sdb 414G 199M 393G 1% /mnt > > > > > > > 2) > > > > > > > /dev/sdc 414G 199M 393G 1% /media/dev1 > > > > > > > 3) > > > > > > > /dev/sdd 414G 15G 378G 4% /media/dev2 > > > > > > > 4) > > > > > > > /dev/sde 414G 199M 393G 1% /media/dev3 > > > > > > > > > > > > > > This distribution of indices that correct or is it a bug > > > > > > > that was fixed > > > > > > > in version0.18.1? > > > > > > > > > > > > > > Earlier I was talking about a test with > > > > > > > the version 0.18.1, using the > > > > > > > same drive with multiple path. > > > > > > > > > > > > > > 2011/10/27 Gustavo Maia [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) > > > > > > > > > > > > > > > OK. If I have 4 drives with the same size (150GB each > > > > > > > > drive). When > > > > > > > > I index 1million documents all drives will have the same size? > > > > > > > > > > > > > > > > I'm doing tests on my machine and then indexing at amazon. > > > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > > > The check for the size of where to place the data is based on > > > > > > > > > the > > > > > > > > > free space available per "drive", so having multiple data > > > > > > > > > locations on the > > > > > > > > > same drive makes little sense. > > > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 1:28 AM, Gustavo Maia > > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > > > Is the same Driver. > > > > > > > > > > > > > > > > > > > > 2011/10/27 Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) > > > > > > > > > > > > > > > > > > > > > Is that on the same drive, or different drives? > > > > > > > > > > > > > > > > > > > > > > On Fri, Oct 28, 2011 at 12:08 AM, Gustavo Maia > > > > > > > > > > > [gustavobbmaia@gmail.com](mailto:gustavobbmaia@gmail.com) wrote: > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > I'm using version 0.18.1 and configuring path.data as > > > > > > > > > > > > follows: > > > > > > > > > > > > > > > > > > > > > > > > path.data:/apps/elasticsearch-0.18.1/data/data1,/apps/elasticsearch-0.18.1/data/data2,/apps/elasticsearch-0.18.1/data/data3,/apps/elasticsearch-0.18.1/data/data4 > > > > > > > > > > > > > > > > > > > > > > > > After 65,000 index doc, I see the size of each path and I > > > > > > > > > > > > was a > > > > > > > > > > > > little > > > > > > > > > > > > surprised the lengths of each path were not similar. Below > > > > > > > > > > > > the > > > > > > > > > > > > size of > > > > > > > > > > > > each path found. > > > > > > > > > > > > > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data4/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > > total 350M > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data3/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > > 34m total > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data2/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > > total 60M > > > > > > > > > > > > gustavo @ gustavo-linux: ~ $ ls-lh > > > > > > > > > > > > > > > > > > > > > > > > /apps/elasticsearch-0.18.1/data/data1/gustavo-index/nodes/0/indices/noticiaindex/0/index > > > > > > > > > > > > / | grep "total" > > > > > > > > > > > > total 35M > > > > > > > > > > > > > > > > > > > > > > > > You can observe that the size of this data4 path much larger > > > > > > > > > > > > than the > > > > > > > > > > > > other paths. Is that correct? I thought that the sizes of > > > > > > > > > > > > the > > > > > > > > > > > > path > > > > > > > > > > > > would be the same or very close. > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Gustavo Maia > > > > > > > > > > > > > > > > -- > > > > > > > > Gustavo Maia > > > > > > > > > > > > > > -- > > > > > > > Gustavo Maia > > > > > > > > > > -- > > > > > Gustavo Maia > > > > > > -- > > > Gustavo Maia -- Gustavo Maia ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:50am UTC](https://discuss.elastic.co/t/path-data-v0-18-1/5712/15 "2017-07-06T03:50:01Z")