Loading Sample Data Error unavailable_shards_exception


(Harshal Yeole) #1

Hi,

ELK Stack version: 6.2.4

I am using Loading Sample Data..and I have managed to create the mapping using curls calls provided but when I hit

`curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json`
`curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/doc/_bulk?pretty' --data-binary @shakespeare_6.0.json`
`curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/_bulk?pretty' --data-binary @logs.jsonl`

I am getting lots of errors like

`{
  "index" : {
    "_index" : "shakespeare",
    "_type" : "doc",
    "_id" : "111393",
    "status" : 503,
    "error" : {
      "type" : "unavailable_shards_exception",
      "reason" : "[shakespeare][4] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[shakespeare][4]] containing [22430] requests]"
    }
  }
},
{
  "index" : {
    "_index" : "shakespeare",
    "_type" : "doc",
    "_id" : "111394",
    "status" : 503,
    "error" : {
      "type" : "unavailable_shards_exception",
      "reason" : "[shakespeare][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[shakespeare][0]] containing [22040] requests]"
    }
  }
},
{
  "index" : {
    "_index" : "shakespeare",
    "_type" : "doc",
    "_id" : "111395",
    "status" : 503,
    "error" : {
      "type" : "unavailable_shards_exception",
      "reason" : "[shakespeare][1] primary shard is not active Timeout: [1m], request:             [BulkShardRequest [[shakespeare][1]] containing [22454] requests]"
        }
      }
    }
  ]
}

`

Can anyone help?


(Vikas) #2

Can you post the logs after starting elasticsearch? As it says shards started successfully.I see your command is correct taken from Lynda.:slight_smile:


(Harshal Yeole) #3

Hi,
Thanks for replying.
I am very new to elasticsearch.
How can I check logs?
Where to look for them?


(Vikas) #4

there is a file in elasticsearch/logs . check that


(Harshal Yeole) #5

I have installed it on the ubuntu machine.
I am not able to find the file. :frowning:


(Harshal Yeole) #6

Found it, it was in /var/log/ directory:

[2018-05-15T17:41:53,212][INFO ][o.e.n.Node               ] [] initializing ...
[2018-05-15T17:41:53,617][INFO ][o.e.e.NodeEnvironment    ] [D89gJz4] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [393.6gb], $
[2018-05-15T17:41:53,620][INFO ][o.e.e.NodeEnvironment    ] [D89gJz4] heap size [990.7mb], compressed ordinary object pointers [true]
[2018-05-15T17:41:53,965][INFO ][o.e.n.Node               ] node name [D89gJz4] derived from node ID [D89gJz4VQ8uGWiIqv8ERqA]; set [node.name] $
[2018-05-15T17:41:53,966][INFO ][o.e.n.Node               ] version[6.2.4], pid[16372], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/3.$
[2018-05-15T17:41:53,966][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyF$
[2018-05-15T17:42:01,254][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [aggs-matrix-stats]
[2018-05-15T17:42:01,255][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [analysis-common]
[2018-05-15T17:42:01,256][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [ingest-common]
[2018-05-15T17:42:01,256][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [lang-expression]
[2018-05-15T17:42:01,256][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [lang-mustache]
[2018-05-15T17:42:01,256][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [lang-painless]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [mapper-extras]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [parent-join]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [percolator]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [rank-eval]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [reindex]
[2018-05-15T17:42:01,257][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [repository-url]
[2018-05-15T17:42:01,258][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [transport-netty4]
[2018-05-15T17:42:01,258][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded module [tribe]
[2018-05-15T17:42:01,265][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-core]
[2018-05-15T17:42:01,265][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-deprecation]
[2018-05-15T17:42:01,265][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-graph]
[2018-05-15T17:42:01,266][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-logstash]
[2018-05-15T17:42:01,267][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-ml]
[2018-05-15T17:42:01,267][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-monitoring]
[2018-05-15T17:42:01,268][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-security]
[2018-05-15T17:42:01,269][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-upgrade]
[2018-05-15T17:42:01,269][INFO ][o.e.p.PluginsService     ] [D89gJz4] loaded plugin [x-pack-watcher]

(Vikas) #7

is that all the log u have?,did it never say that elasticsearch node started?


(Harshal Yeole) #8

No, this was it.


(Vikas) #9

what command you run to start elasticsearch?


(Harshal Yeole) #10
sudo service elasticsearch start

(Harshal Yeole) #11

I am sorry but elastic search works fine, I can see the cluster info on localhost:9200.

It's just that, I cant load the data for the kibana.


(Vikas) #12

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json
are you running this command in kibana.Please give this PUT /swiss in kibana.it should create an index named swiss.


(Harshal Yeole) #13

I am running it on ubuntu terminal.
Is that correct?


(Vikas) #14

running the above on ubunto terminal is correct.


(Harshal Yeole) #15

Yes.


(Harshal Yeole) #16

Check this indices info please.

In the link for loading the data this CURL CALL should return some docs count and details but I can see docs for only bank collection and for others its nothing there.


(Vikas) #17

yup you are right.But the thing is did you send any data to other index except bank?if yes than its problem otherwise I dnt think so it is prblm. As I can see u run the command for Shakespeare index.so Shakespeare.json file is present in ur machine rght.Actually for me it worked.I will check into this issue.


(Harshal Yeole) #18

Yes, I have this files in my machine.


(Vikas) #19

Can you delete the index Shakespeare and rerun the command.Before running the command run this: using curl
PUT /shakespeare/_settings
{
"index" : {
"number_of_shards":3
"number_of_replicas" : 2

}

}


(Harshal Yeole) #20

Hi @Dvikas ,
I tried your solution it didi not work.