Just because this saga is annoying me (no offense), I ran a perl one-liner
to sum up the file sizes. Here's the total sizes by extension:
ES:
fdt: 7,964,999,668
fdx: 880,736,240
fnm: 992
frq: 3,705,120,837
gen: 20
nrm: 110,092,142
prx: 1,610,367,435
tii: 26,441,783
tis: 2,749,100,849
Lucene:
fdt: 4,343,895,958
fdx: 883,151,808
fnm: 336
frq: 3,104,536,899
gen: 20
nrm: 110,394,025
prx: 506,303,552
tii: 538,856
tis: 38,773,091
I suspect the sizes after optimize will be pretty much the same.
On Wed, May 29, 2013 at 7:36 PM, Matt Weber matt.weber@gmail.com wrote:
In lucene use forceMerge(1) or optimize(1) and in ES use
/_optimize?max_num_segments=1.Thanks,
Matt WeberOn Wed, May 29, 2013 at 9:19 AM, Shlomi Vaknin shlomivaknin@gmail.comwrote:
yeah, i actually noticed that and optimized, but then it turned it to a
CFS format.. so i am rerunning it now. ill have the new listing tomorrow.although, in the code i have a call to writer.optimize(); so i am not
sure if there will be different results.On Wed, May 29, 2013 at 7:16 PM, Matt Weber matt.weber@gmail.com wrote:
I think Simon asked for a completely optimized listing of each index
(max segments = 1).On Wed, May 29, 2013 at 9:12 AM, Shlomi Vaknin shlomivaknin@gmail.comwrote:
Thanks Jörg,
Ok, i disabled CFS in lucene, and ran it again. it didnt seem to make
much difference:ls -ltra
total 8777268
drwxrwxrwt 30 root root 20480 May 29 18:30 ..
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:39 _34.fnm
-rw-rw-r-- 1 shlomiv shlomiv 263247124 May 29 18:39 _34.fdx
-rw-rw-r-- 1 shlomiv shlomiv 1295797937 May 29 18:39 _34.fdt
-rw-rw-r-- 1 shlomiv shlomiv 8154272 May 29 18:40 _34.tis
-rw-rw-r-- 1 shlomiv shlomiv 114159 May 29 18:40 _34.tii
-rw-rw-r-- 1 shlomiv shlomiv 151801202 May 29 18:40 _34.prx
-rw-rw-r-- 1 shlomiv shlomiv 925753058 May 29 18:40 _34.frq
-rw-rw-r-- 1 shlomiv shlomiv 32905894 May 29 18:40 _34.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:47 _66.fnm
-rw-rw-r-- 1 shlomiv shlomiv 244284132 May 29 18:47 _66.fdx
-rw-rw-r-- 1 shlomiv shlomiv 1200444042 May 29 18:47 _66.fdt
-rw-rw-r-- 1 shlomiv shlomiv 9156383 May 29 18:48 _66.tis
-rw-rw-r-- 1 shlomiv shlomiv 127850 May 29 18:48 _66.tii
-rw-rw-r-- 1 shlomiv shlomiv 140138826 May 29 18:48 _66.prx
-rw-rw-r-- 1 shlomiv shlomiv 862107109 May 29 18:48 _66.frq
-rw-rw-r-- 1 shlomiv shlomiv 30535520 May 29 18:48 _66.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:56 _99.fnm
-rw-rw-r-- 1 shlomiv shlomiv 254267076 May 29 18:56 _99.fdx
-rw-rw-r-- 1 shlomiv shlomiv 1245411536 May 29 18:56 _99.fdt
-rw-rw-r-- 1 shlomiv shlomiv 8429835 May 29 18:57 _99.tis
-rw-rw-r-- 1 shlomiv shlomiv 117255 May 29 18:57 _99.tii
-rw-rw-r-- 1 shlomiv shlomiv 144073653 May 29 18:57 _99.prx
-rw-rw-r-- 1 shlomiv shlomiv 891520868 May 29 18:57 _99.frq
-rw-rw-r-- 1 shlomiv shlomiv 31783388 May 29 18:57 _99.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:57 _9j.fnm
-rw-rw-r-- 1 shlomiv shlomiv 27082292 May 29 18:57 _9j.fdx
-rw-rw-r-- 1 shlomiv shlomiv 135824217 May 29 18:57 _9j.fdt
-rw-rw-r-- 1 shlomiv shlomiv 2030268 May 29 18:57 _9j.tis
-rw-rw-r-- 1 shlomiv shlomiv 28317 May 29 18:57 _9j.tii
-rw-rw-r-- 1 shlomiv shlomiv 15952977 May 29 18:57 _9j.prx
-rw-rw-r-- 1 shlomiv shlomiv 94820220 May 29 18:57 _9j.frq
-rw-rw-r-- 1 shlomiv shlomiv 3385290 May 29 18:57 _9j.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:58 _9u.fnm
-rw-rw-r-- 1 shlomiv shlomiv 27505988 May 29 18:58 _9u.fdx
-rw-rw-r-- 1 shlomiv shlomiv 136384808 May 29 18:58 _9u.fdt
-rw-rw-r-- 1 shlomiv shlomiv 1674541 May 29 18:58 _9u.tis
-rw-rw-r-- 1 shlomiv shlomiv 23094 May 29 18:58 _9u.tii
-rw-rw-r-- 1 shlomiv shlomiv 16176340 May 29 18:58 _9u.prx
-rw-rw-r-- 1 shlomiv shlomiv 96098118 May 29 18:58 _9u.frq
-rw-rw-r-- 1 shlomiv shlomiv 3438252 May 29 18:58 _9u.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:59 _a5.fnm
-rw-rw-r-- 1 shlomiv shlomiv 24784676 May 29 18:59 _a5.fdx
-rw-rw-r-- 1 shlomiv shlomiv 120749640 May 29 18:59 _a5.fdt
-rw-rw-r-- 1 shlomiv shlomiv 2457010 May 29 18:59 _a5.tis
-rw-rw-r-- 1 shlomiv shlomiv 33481 May 29 18:59 _a5.tii
-rw-rw-r-- 1 shlomiv shlomiv 13850257 May 29 18:59 _a5.prx
-rw-rw-r-- 1 shlomiv shlomiv 86453688 May 29 18:59 _a5.frq
-rw-rw-r-- 1 shlomiv shlomiv 3098088 May 29 18:59 _a5.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 18:59 _ag.fnm
-rw-rw-r-- 1 shlomiv shlomiv 26705412 May 29 18:59 _ag.fdx
-rw-rw-r-- 1 shlomiv shlomiv 131405660 May 29 18:59 _ag.fdt
-rw-rw-r-- 1 shlomiv shlomiv 1204948 May 29 19:00 _ah.fdx
-rw-rw-r-- 1 shlomiv shlomiv 5911795 May 29 19:00 _ah.fdt
-rw-rw-r-- 1 shlomiv shlomiv 330832 May 29 19:00 _ah.tis
-rw-rw-r-- 1 shlomiv shlomiv 4639 May 29 19:00 _ah.tii
-rw-rw-r-- 1 shlomiv shlomiv 696420 May 29 19:00 _ah.prx
-rw-rw-r-- 1 shlomiv shlomiv 150622 May 29 19:00 _ah.nrm
-rw-rw-r-- 1 shlomiv shlomiv 4183072 May 29 19:00 _ah.frq
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _ah.fnm
-rw-rw-r-- 1 shlomiv shlomiv 2569983 May 29 19:00 _ag.tis
-rw-rw-r-- 1 shlomiv shlomiv 35436 May 29 19:00 _ag.tii
-rw-rw-r-- 1 shlomiv shlomiv 15571391 May 29 19:00 _ag.prx
-rw-rw-r-- 1 shlomiv shlomiv 94078982 May 29 19:00 _ag.frq
-rw-rw-r-- 1 shlomiv shlomiv 3338180 May 29 19:00 _ag.nrm
-rw-rw-r-- 1 shlomiv shlomiv 3123508 May 29 19:00 _ai.fdx
-rw-rw-r-- 1 shlomiv shlomiv 15925878 May 29 19:00 _ai.fdt
-rw-rw-r-- 1 shlomiv shlomiv 614128 May 29 19:00 _ai.tis
-rw-rw-r-- 1 shlomiv shlomiv 8534 May 29 19:00 _ai.tii
-rw-rw-r-- 1 shlomiv shlomiv 1811166 May 29 19:00 _ai.prx
-rw-rw-r-- 1 shlomiv shlomiv 390442 May 29 19:00 _ai.nrm
-rw-rw-r-- 1 shlomiv shlomiv 10911623 May 29 19:00 _ai.frq
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _ai.fnm
-rw-rw-r-- 1 shlomiv shlomiv 3217708 May 29 19:00 _aj.fdx
-rw-rw-r-- 1 shlomiv shlomiv 15914211 May 29 19:00 _aj.fdt
-rw-rw-r-- 1 shlomiv shlomiv 577853 May 29 19:00 _aj.tis
-rw-rw-r-- 1 shlomiv shlomiv 7902 May 29 19:00 _aj.tii
-rw-rw-r-- 1 shlomiv shlomiv 1874837 May 29 19:00 _aj.prx
-rw-rw-r-- 1 shlomiv shlomiv 11189671 May 29 19:00 _aj.frq
-rw-rw-r-- 1 shlomiv shlomiv 402217 May 29 19:00 _aj.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _aj.fnm
-rw-rw-r-- 1 shlomiv shlomiv 671412 May 29 19:00 _ak.fdx
-rw-rw-r-- 1 shlomiv shlomiv 3394019 May 29 19:00 _ak.fdt
-rw-rw-r-- 1 shlomiv shlomiv 232124 May 29 19:00 _ak.tis
-rw-rw-r-- 1 shlomiv shlomiv 3258 May 29 19:00 _ak.tii
-rw-rw-r-- 1 shlomiv shlomiv 390852 May 29 19:00 _ak.prx
-rw-rw-r-- 1 shlomiv shlomiv 83930 May 29 19:00 _ak.nrm
-rw-rw-r-- 1 shlomiv shlomiv 2324403 May 29 19:00 _ak.frq
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _ak.fnm
-rw-rw-r-- 1 shlomiv shlomiv 2320804 May 29 19:00 _al.fdx
-rw-rw-r-- 1 shlomiv shlomiv 11751843 May 29 19:00 _al.fdt
-rw-rw-r-- 1 shlomiv shlomiv 919429 May 29 19:00 _al.tis
-rw-rw-r-- 1 shlomiv shlomiv 12504 May 29 19:00 _al.tii
-rw-rw-r-- 1 shlomiv shlomiv 1279214 May 29 19:00 _al.prx
-rw-rw-r-- 1 shlomiv shlomiv 8255522 May 29 19:00 _al.frq
-rw-rw-r-- 1 shlomiv shlomiv 290104 May 29 19:00 _al.nrm
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _al.fnm
-rw-rw-r-- 1 shlomiv shlomiv 2432020 May 29 19:00 _am.fdx
-rw-rw-r-- 1 shlomiv shlomiv 12765052 May 29 19:00 _am.fdt
-rw-rw-r-- 1 shlomiv shlomiv 885852 May 29 19:00 _am.tis
-rw-rw-r-- 1 shlomiv shlomiv 12305 May 29 19:00 _am.tii
-rw-rw-r-- 1 shlomiv shlomiv 1360925 May 29 19:00 _am.prx
-rw-rw-r-- 1 shlomiv shlomiv 304006 May 29 19:00 _am.nrm
-rw-rw-r-- 1 shlomiv shlomiv 8638946 May 29 19:00 _am.frq
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _am.fnm
-rw-rw-r-- 1 shlomiv shlomiv 2304708 May 29 19:00 _an.fdx
-rw-rw-r-- 1 shlomiv shlomiv 12215320 May 29 19:00 _an.fdt
-rw-rw-r-- 1 shlomiv shlomiv 740581 May 29 19:00 _an.tis
-rw-rw-r-- 1 shlomiv shlomiv 10122 May 29 19:00 _an.tii
-rw-rw-r-- 1 shlomiv shlomiv 1325492 May 29 19:00 _an.prx
-rw-rw-r-- 1 shlomiv shlomiv 288092 May 29 19:00 _an.nrm
-rw-rw-r-- 1 shlomiv shlomiv 8201619 May 29 19:00 _an.frq
-rw-rw-r-- 1 shlomiv shlomiv 24 May 29 19:00 _an.fnm
-rw-rw-r-- 1 shlomiv shlomiv 20 May 29 19:00 segments.gen
-rw-rw-r-- 1 shlomiv shlomiv 2888 May 29 19:00 segments_37
drwxrwxr-x 2 shlomiv shlomiv 4096 May 29 19:04 .what do you think i should try next?
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/6j0E-2pTbWg/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.