Thanks Mark,
auth.log doesnt show any login or sudo at the time of the elastic
stopping...
nothing else is running on that machine - it is a dedicated ES server.
what i did find in the auth log is that someone is trying to hack into the
system, yet i dont see how it got to do with elastic stopping?
On Sunday, August 24, 2014 4:35:41 AM UTC+3, Mark Walkom wrote:
Something is stopping the service.
If you are on linux check the auth log, if anyone is using sudo to stop it
then you will see that logged. Otherwise, what else runs on the machine?Regards,
Mark WalkomInfrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.comOn 24 August 2014 06:15, Eitan Vesely <eita...@gmail.com <javascript:>>
wrote:Hi Guys,
i've installed ES a month ago and its working just fine.today, for some reason, ES just went down for no visible reason:
here is what i see in the log file :
[2014-08-23 16:47:11,272][DEBUG][action.search.type ] [Plunderer]
[g30nm0bi2j663tgu6ud][1], node[Vc4xSuh1S1qQOvQdv-wD_A], [P], s[STARTED]:
Failed to execute [org.elasticsearch.action.search.SearchRequest@5531dfad]
lastShard [true]
org.elasticsearch.search.SearchParseException: [g30nm0bi2j663tgu6ud][1]:
from[-1],size[-1]: Parse Failure [Failed to parse source
[{"facets":{"0":{"date_histogram":{"key_field":"@timestamp","value_field":"user_count","interval":"1h"},"global":true,"facet_filter":{"fquery":{"query":{"filtered":{"query":{"query_string":{"query":""}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1407602785182,"to":1408812385182}}},{"range":{"@timestamp":{"from":1408516424602,"to":1408811520255}}}]}}}}}}}},"size":0}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:511)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203)
at
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.search.facet.FacetPhaseExecutionException:
Facet [0]: (key) field [@timestamp] not found
at
org.elasticsearch.search.facet.datehistogram.DateHistogramFacetParser.parse(DateHistogramFacetParser.java:160)
at
org.elasticsearch.search.facet.FacetParseElement.parse(FacetParseElement.java:93)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)
... 9 more
[2014-08-23 16:47:11,273][DEBUG][action.search.type ] [Plunderer]
[g30nm0bi2j663tgu6ud][0], node[Vc4xSuh1S1qQOvQdv-wD_A], [P], s[STARTED]:
Failed to execute [org.elasticsearch.action.search.SearchRequest@5531dfad]
org.elasticsearch.search.SearchParseException: [g30nm0bi2j663tgu6ud][0]:
from[-1],size[-1]: Parse Failure [Failed to parse source
[{"facets":{"0":{"date_histogram":{"key_field":"@timestamp","value_field":"user_count","interval":"1h"},"global":true,"facet_filter":{"fquery":{"query":{"filtered":{"query":{"query_string":{"query":""}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1407602785182,"to":1408812385182}}},{"range":{"@timestamp":{"from":1408516424602,"to":1408811520255}}}]}}}}}}}},"size":0}]]
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:649)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:511)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:483)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:206)
at
org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:203)
at
org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.search.facet.FacetPhaseExecutionException:
Facet [0]: (key) field [@timestamp] not found
at
org.elasticsearch.search.facet.datehistogram.DateHistogramFacetParser.parse(DateHistogramFacetParser.java:160)
at
org.elasticsearch.search.facet.FacetParseElement.parse(FacetParseElement.java:93)
at
org.elasticsearch.search.SearchService.parseSource(SearchService.java:633)
... 9 more
[2014-08-23 16:49:05,401][INFO ][node ] [Plunderer]
stopping ...
[2014-08-23 16:49:06,737][INFO ][node ] [Plunderer]
stopped
[2014-08-23 16:49:06,737][INFO ][node ] [Plunderer]
closing ...
[2014-08-23 16:49:06,772][INFO ][node ] [Plunderer]
closed
[2014-08-23 20:02:09,693][WARN ][common.jna ] Unable to
lock JVM memory (ENOMEM). This can result in part of the JVM being swapped
out. Increase RLIMIT_MEMLOCK or run elasticsearch as root.
[2014-08-23 20:02:09,921][INFO ][node ] [Ahmet Abdol]
version[1.2.2], pid[2715], build[9902f08/2014-07-09T12:02:32Z]
[2014-08-23 20:02:09,921][INFO ][node ] [Ahmet Abdol]
initializing ...
[2014-08-23 20:02:09,940][INFO ][plugins ] [Ahmet Abdol]
loaded , sites [kopf]
[2014-08-23 20:02:14,692][INFO ][node ] [Ahmet Abdol]
initializedany ideas??
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/55571116-a9ac-4517-b530-fc2e0f13501f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/55571116-a9ac-4517-b530-fc2e0f13501f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d87b8ca2-8f11-4f5c-b0f1-d661d5bf2f3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.