# Memory problems with larger synonyms file **URL:** \ **Category:** Elasticsearch\ **Created:** [May 31, 2011, 9:38am UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503 "2011-05-31T09:38:40Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Yannick\_Smits\_2](https://avatars.discourse-cdn.com/v4/letter/y/a4c791/32.png) [@Yannick\_Smits\_2](https://discuss.elastic.co/u/Yannick_Smits_2)\ **Post date:** [May 31, 2011, 9:38am UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503/1 "2011-05-31T09:38:40Z")
With this gist: [https://gist.github.com/1000235](https://gist.github.com/1000235) [https://gist.github.com/1000235](https://gist.github.com/1000235), when I run the index configuration the java.exe process goes to 30% CPU and memory usage grows to 400+MB's, the index never gets created and it keeps the high CPU. The ES window tells me this kind of messages: [2011-05-31 11:34:14,808][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][25] took [5.7s]/[82ms], reclaimed [17.7mb], leaving [217.8mb] used, max [1gb] [2011-05-31 11:34:32,865][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][26] took [5.7s]/[85ms], reclaimed [30.4mb], leaving [221.3mb] used, max [1gb] [2011-05-31 11:34:52,007][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][27] took [5.7s]/[88ms], reclaimed [18.7mb], leaving [244.6mb] used, max [1gb] When I try with smaller synonyms files it seems to work and create the index without problem. Thanks, Yannick ---
**Author:** ![Yannick\_Smits\_2](https://avatars.discourse-cdn.com/v4/letter/y/a4c791/32.png) [@Yannick\_Smits\_2](https://discuss.elastic.co/u/Yannick_Smits_2)\ **Post date:** [June 4, 2011, 4:21pm UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503/2 "2011-06-04T16:21:18Z")
Anyone been able to reproduce this one? Still unresolved. Running on Windows 7 with Java 6 update 24 btw. Thanks, Yannick From: Yannick Smits [[mailto:mailinglists@goyaweb.nl](mailto:mailinglists@goyaweb.nl)] Sent: dinsdag 31 mei 2011 11:39 To: [users@elasticsearch.com](mailto:users@elasticsearch.com) Subject: memory problems with larger synonyms file With this gist: [https://gist.github.com/1000235](https://gist.github.com/1000235) [https://gist.github.com/1000235](https://gist.github.com/1000235), when I run the index configuration the java.exe process goes to 30% CPU and memory usage grows to 400+MB's, the index never gets created and it keeps the high CPU. The ES window tells me this kind of messages: [2011-05-31 11:34:14,808][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][25] took [5.7s]/[82ms], reclaimed [17.7mb], leaving [217.8mb] used, max [1gb] [2011-05-31 11:34:32,865][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][26] took [5.7s]/[85ms], reclaimed [30.4mb], leaving [221.3mb] used, max [1gb] [2011-05-31 11:34:52,007][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][27] took [5.7s]/[88ms], reclaimed [18.7mb], leaving [244.6mb] used, max [1gb] When I try with smaller synonyms files it seems to work and create the index without problem. Thanks, Yannick ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 4, 2011, 4:34pm UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503/3 "2011-06-04T16:34:39Z")
Heya, Yea, I managed to simulate this. It does take time to process the synonym file. The other problem is that it is being processed twice, which I already fixed in master. Moreover, something that needs to be added is caching of the same synonym file for difference indices if they use the same one. -shay.banon On Saturday, June 4, 2011 at 7:21 PM, Yannick Smits wrote: > Anyone been able to reproduce this one? Still unresolved. > Running on Windows 7 with Java 6 update 24 btw. > > Thanks, > Yannick > > From: Yannick Smits [[mailto:mailinglists@goyaweb.nl](mailto:mailinglists@goyaweb.nl)] > Sent: dinsdag 31 mei 2011 11:39 > To: [users@elasticsearch.com](mailto:users@elasticsearch.com) ([mailto:users@elasticsearch.com](mailto:users@elasticsearch.com)) > Subject: memory problems with larger synonyms file > > With this gist: [1000235’s gists · GitHub](https://gist.github.com/1000235), when I run the index configuration the java.exe process goes to 30% CPU and memory usage grows to 400+MB’s, the index never gets created and it keeps the high CPU. The ES window tells me this kind of messages: > > [2011-05-31 11:34:14,808][INFO][monitor.jvm] [test] [ > gc][ConcurrentMarkSweep][25] took [5.7s]/[82ms], reclaimed [17.7mb], leaving [217.8mb] used, max [1gb] > [2011-05-31 11:34:32,865][INFO][monitor.jvm] [test] [ > gc][ConcurrentMarkSweep][26] took [5.7s]/[85ms], reclaimed [30.4mb], leaving [221.3mb] used, max [1gb] > [2011-05-31 11:34:52,007][INFO][monitor.jvm] [test] [ > gc][ConcurrentMarkSweep][27] took [5.7s]/[88ms], reclaimed [18.7mb], leaving [244.6mb] used, max [1gb] > > When I try with smaller synonyms files it seems to work and create the index without problem. > > Thanks, > Yannick ---
**Author:** ![Yannick\_Smits\_2](https://avatars.discourse-cdn.com/v4/letter/y/a4c791/32.png) [@Yannick\_Smits\_2](https://discuss.elastic.co/u/Yannick_Smits_2)\ **Post date:** [June 22, 2011, 8:52pm UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503/4 "2011-06-22T20:52:51Z")
I tried to add a 1MB synonym file to an index which took more than 13 hours, after which I had to kill it because I couldn’t wait anymore (high cpu was bringing other services down). 1MB seems very small to me so I really can’t understand why it would take over 13 hours… Hope something can be done about that or you have other tips to solve adding larger synonyms files. Thanks, Yannick From: Shay Banon [[mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)] Sent: zaterdag 4 juni 2011 18:35 To: [users@elasticsearch.com](mailto:users@elasticsearch.com) Subject: Re: memory problems with larger synonyms file Heya, Yea, I managed to simulate this. It does take time to process the synonym file. The other problem is that it is being processed twice, which I already fixed in master. Moreover, something that needs to be added is caching of the same synonym file for difference indices if they use the same one. -shay.banon On Saturday, June 4, 2011 at 7:21 PM, Yannick Smits wrote: Anyone been able to reproduce this one? Still unresolved. Running on Windows 7 with Java 6 update 24 btw. Thanks, Yannick From: Yannick Smits [[mailto:mailinglists@goyaweb.nl](mailto:mailinglists@goyaweb.nl)] Sent: dinsdag 31 mei 2011 11:39 To: [users@elasticsearch.com](mailto:users@elasticsearch.com) Subject: memory problems with larger synonyms file With this gist: [https://gist.github.com/1000235](https://gist.github.com/1000235) [https://gist.github.com/1000235](https://gist.github.com/1000235), when I run the index configuration the java.exe process goes to 30% CPU and memory usage grows to 400+MB’s, the index never gets created and it keeps the high CPU. The ES window tells me this kind of messages: [2011-05-31 11:34:14,808][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][25] took [5.7s]/[82ms], reclaimed [17.7mb], leaving [217.8mb] used, max [1gb] [2011-05-31 11:34:32,865][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][26] took [5.7s]/[85ms], reclaimed [30.4mb], leaving [221.3mb] used, max [1gb] [2011-05-31 11:34:52,007][INFO][monitor.jvm] [test] [ gc][ConcurrentMarkSweep][27] took [5.7s]/[88ms], reclaimed [18.7mb], leaving [244.6mb] used, max [1gb] When I try with smaller synonyms files it seems to work and create the index without problem. Thanks, Yannick ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:02am UTC](https://discuss.elastic.co/t/memory-problems-with-larger-synonyms-file/4503/5 "2017-07-06T04:02:55Z")