# Elastic crash while import

**URL:** <https://discuss.elastic.co/t/elastic-crash-while-import/277593>\
**Category:** Elasticsearch\
**Created:** [July 1, 2021, 7:52pm UTC](https://discuss.elastic.co/t/elastic-crash-while-import/277593 "2021-07-01T19:52:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vladimir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vladimir/32/91042_2.png) [@Vladimir](https://discuss.elastic.co/u/Vladimir)\
**Post date:** [July 1, 2021, 7:52pm UTC](https://discuss.elastic.co/t/elastic-crash-while-import/277593/1 "2021-07-01T19:52:18Z")

</div>

I have Elasticsearch installed so I am playing with it. I made a bash script which generates simple documents with name, title, perex and text fields (text is 100 sentences of lorem ipsum) and send it via \_bulk api to Elasticsearch. I run 5 parallel scripts to do it and one bulk contains 250 documents. I had about 100 000 items indexed when Elasticsearch crashed and I need to manually restart the process. The index definition is here:

```auto
PUT /test2
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": { 
    "properties": {
      "name" : {
        "type" : "text",
        "fields" : {
          "raw" : {
            "type" : "keyword"
            }
        }
      },
      "title" : {
        "type" : "text",
        "fields" : {
          "raw" : {
            "type" : "keyword"
          }
        }
      },
      "perex" : {
        "type" : "text"
      },
      "text" : {
        "type" : "text"
      }
    }    
  }
} 

```

elasticsearch.log file contains this:

```auto
[2021-07-01T20:58:45,424][INFO][o.e.t.NettyAllocator] [vmi503579.contaboserver.net] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2021-07-01T20:58:45,561][INFO][o.e.d.DiscoveryModule] [vmi503579.contaboserver.net] using discovery type [zen] and seed hosts providers [settings]
[2021-07-01T20:58:46,533][INFO][o.e.g.DanglingIndicesState] [vmi503579.contaboserver.net] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2021-07-01T20:58:47,422][INFO][o.e.n.Node] [vmi503579.contaboserver.net] initialized
[2021-07-01T20:58:47,424][INFO][o.e.n.Node] [vmi503579.contaboserver.net] starting ...
[2021-07-01T20:58:47,447][INFO][o.e.x.s.c.f.PersistentCache] [vmi503579.contaboserver.net] persistent cache index loaded
[2021-07-01T20:58:47,699][INFO][o.e.t.TransportService] [vmi503579.contaboserver.net] publish_address {localhost/127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2021-07-01T20:58:48,587][WARN][o.e.b.BootstrapChecks] [vmi503579.contaboserver.net] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2021-07-01T20:58:48,589][INFO][o.e.c.c.Coordinator] [vmi503579.contaboserver.net] cluster UUID [QR6cWTx6ToSKgVEAqs34TQ]
[2021-07-01T20:58:48,605][INFO][o.e.c.c.ClusterBootstrapService] [vmi503579.contaboserver.net] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2021-07-01T20:58:48,719][INFO][o.e.c.s.MasterService] [vmi503579.contaboserver.net] elected-as-master ([1] nodes joined)[{vmi503579.contaboserver.net}{Wp9M32ZRR_mNKO29Ttmaew}{ssVTNtcLSraJu-hzJiUzNA}{localhost}{127.0.0.1:9300}{cdfhilmrstw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 8, version: 215, delta: master node changed {previous [], current [{vmi503579.contaboserver.net}{Wp9M32ZRR_mNKO29Ttmaew}{ssVTNtcLSraJu-hzJiUzNA}{localhost}{127.0.0.1:9300}{cdfhilmrstw}]}
[2021-07-01T20:58:48,880][INFO][o.e.c.s.ClusterApplierService] [vmi503579.contaboserver.net] master node changed {previous [], current [{vmi503579.contaboserver.net}{Wp9M32ZRR_mNKO29Ttmaew}{ssVTNtcLSraJu-hzJiUzNA}{localhost}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 8, version: 215, reason: Publication{term=8, version=215}
[2021-07-01T20:58:48,956][INFO][o.e.h.AbstractHttpServerTransport] [vmi503579.contaboserver.net] publish_address {localhost/127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2021-07-01T20:58:48,957][INFO][o.e.n.Node] [vmi503579.contaboserver.net] started
[2021-07-01T20:58:49,874][INFO][o.e.l.LicenseService] [vmi503579.contaboserver.net] license [f35cccd9-e198-4bb4-baeb-1c7acf80cea4] mode [basic] - valid
[2021-07-01T20:58:49,876][INFO][o.e.x.s.s.SecurityStatusChangeListener] [vmi503579.contaboserver.net] Active license is now [BASIC]; Security is disabled
[2021-07-01T20:58:49,877][WARN][o.e.x.s.s.SecurityStatusChangeListener] [vmi503579.contaboserver.net] Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
[2021-07-01T20:58:49,883][INFO][o.e.g.GatewayService] [vmi503579.contaboserver.net] recovered [8] indices into cluster_state
[2021-07-01T20:58:51,368][WARN][r.suppressed] [vmi503579.contaboserver.net] path: /.kibana_task_manager/_search, params: {ignore_unavailable=true, index=.kibana_task_manager, track_total_hits=true}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:661) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:384) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:694) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:467) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.access$000(AbstractSearchAsyncAction.java:62) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:316) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:66) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:48) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:400) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TransportService$5.handleException(TransportService.java:738) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1283) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1392) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1366) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:50) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:45) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:40) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.ActionRunnable.onFailure(ActionRunnable.java:77) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:720) [elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:28) [elasticsearch-7.13.2.jar:7.13.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:831) [?:?]
Caused by: org.elasticsearch.action.NoShardAvailableActionException: [vmi503579.contaboserver.net][127.0.0.1:9300][indices:data/read/search[phase/query]]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:500) ~[elasticsearch-7.13.2.jar:7.13.2]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:449) [elasticsearch-7.13.2.jar:7.13.2]
	... 18 more

```

Here are the links to log files:  
[https://drive.google.com/file/d/1kQQyF6d7HX\_UmYpgVplTTcDREwHLpRPN/view?usp=sharing](https://drive.google.com/file/d/1kQQyF6d7HX_UmYpgVplTTcDREwHLpRPN/view?usp=sharing)  
[https://drive.google.com/file/d/1QgbRSJA67\_2XM4WfJgLBfBksSfwaGTje/view?usp=sharing](https://drive.google.com/file/d/1QgbRSJA67_2XM4WfJgLBfBksSfwaGTje/view?usp=sharing)  
[https://drive.google.com/file/d/1iyzSCJ-\_UqmCbHS60JvL4VaZ7mcz79y1/view?usp=sharing](https://drive.google.com/file/d/1iyzSCJ-_UqmCbHS60JvL4VaZ7mcz79y1/view?usp=sharing)

Does anybody know what is the problem? I would like to fix it, but I am only a beginner in this.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 5, 2021, 2:20am UTC](https://discuss.elastic.co/t/elastic-crash-while-import/277593/2 "2021-07-05T02:20:51Z")

</div>

I am not seeing any crash in the log files, just Elasticsearch running or restarting, trying to recover and then something restarting Elasticsearch.

---

<div class="post-metadata">

**Author:** ![Vladimir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vladimir/32/91042_2.png) [@Vladimir](https://discuss.elastic.co/u/Vladimir)\
**Post date:** [July 5, 2021, 8:10am UTC](https://discuss.elastic.co/t/elastic-crash-while-import/277593/3 "2021-07-05T08:10:03Z")

</div>

I have a run small cron script every minute to check if Elasticsearch in running.

```auto
#!/bin/bash

# -c returns number of lines in grep result
isActive=$(systemctl status elasticsearch | grep "active (running)" -c)

if (( $isActive == 0 )) 
then

  systemctl start elasticsearch
  
  timestamp=$(date +"%Y-%m-%d %H-%M-%S") 
  touch /root/custom-scripts/elasticsearch/start.log
  echo "${timestamp} Elasticsearch service has to be restarted by /root/custom-scripts/elasticsearch/start-elasticsearch.sh because service status was not active." >> /root/custom-scripts/elasticsearch/start.log

fi

```

Is it possible the condition `active (running)` is not correct? I see this script starts Elastic on average three times per hour.

Here are today logs.  
[Elasticsearch.log](https://drive.google.com/file/d/1lZ5X32wS_09sX1erhF9nqH_Iew48BjJ6/view?usp=sharing)  
and [gc.log](https://drive.google.com/file/d/13_0WR8O4rFnt1U2XVFytqQgNGQJVoOGC/view?usp=sharing)

May be there are not enough resources. It is VPS with 6vCPU 8GB ram 400GB SSD. There is only one node and few indices. One index with 950 000 items.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2021, 8:10am UTC](https://discuss.elastic.co/t/elastic-crash-while-import/277593/4 "2021-08-02T08:10:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
