I recently replicated our elasticsearch server and noticed a weird process after starting elasticsearch.
110 is the UID for elasticsearch user
110 6772 0.0 0.0 24612 1848 ? S 20:19 0:00 wget -O /tmp/MMPP http://222.186.21.166:1857/MMPP
Whois info for the IP:
whois 222.186.21.166
% [whois.apnic.net]
% Whois data copyright terms APNIC Whois Database Acceptable Use Agreement – APNIC
% Information related to '222.184.0.0 - 222.191.255.255'
inetnum: 222.184.0.0 - 222.191.255.255
netname: CHINANET-JS
descr: CHINANET jiangsu province network
descr: China Telecom
descr: A12,Xin-Jie-Kou-Wai Street
descr: Beijing 100088
country: CN
admin-c: CH93-AP
tech-c: CJ186-AP
mnt-by: APNIC-HM
mnt-lower: MAINT-CHINANET-JS
mnt-routes: MAINT-CHINANET-JS
remarks: This object can only modify by APNIC hostmaster
remarks: If you wish to modify this object details please
remarks: send email to hostmaster@apnic.net with your
remarks: organisation account name in the subject line.
status: ALLOCATED PORTABLE
source: APNIC
mnt-irt: IRT-CHINANET-CN
changed: hm-changed@apnic.net 20040223
Several of these commands were run, downloading a small handful of files. When I killed the following java process being run by elasticsearch the wgets stopped.
110 12841 2.8 14.1 9903064 546236 ? Sl 09:38 9:32 /usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
I noticed this in the elasticsearch log as well
[2015-11-04 20:21:11,377][DEBUG][action.search.type ] [Big Wheel] [402] Failed to execute fetch phase
org.elasticsearch.script.groovy.GroovyScriptExecutionException: IOException[Cannot run program "/tmp/MMPP": error=26, Text file busy]; nested: IOException[error=26, Text file busy];
at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:257)
at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:460)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:410)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:407)
at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
[2015-11-04 20:25:53,435][DEBUG][action.search.type ] [Big Wheel] [737] Failed to execute fetch phase
org.elasticsearch.script.groovy.GroovyScriptExecutionException: IOException[Cannot run program "/tmp/NALIn": error=26, Text file busy]; nested: IOException[error=26, Text file busy];
at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:257)
at org.elasticsearch.search.fetch.script.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:74)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:211)
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:460)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:410)
at org.elasticsearch.search.action.SearchServiceTransportAction$17.call(SearchServiceTransportAction.java:407)
at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Why would elasticsearch be downloading files from a Chinese server and trying to run them? Is this normal behavior? What is the purpose?