After putting index setting, cannot add document

lemon_soft · April 17, 2019, 2:06pm

I failed to create my ElasticSearch index, again. Here are my steps:

created index with name clef2018, number of shard=1, number of replicas=0
closed index
put settings of clef2018 (website)
put mappings of clef2018 (website)
opened index
checked that no document indexed
cannot added document with bulk API or python
also search failed

Is it an ElasticSearch issue? My python script is:

from elasticsearch import Elasticsearch
title="title"
body="body"
es = Elasticsearch([{"host": "10.0.0.1", "port": 9200}])
contentt = {'content': title +"\n"+ body}
res = ""
idd=idd+1
try:
	res = es.index(index='clef2018', doc_type='_doc', id=idd, body=contentt)
except:
	print("error:" + str(res))

Error is:

{
    "error": {
        "root_cause": [
            {
                "type": "unavailable_shards_exception",
                "reason": "[example][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[example][0]] containing [index {[example][_doc][1], source[{\n    \"content\": \"Hello world\"\n}]}]]"
            }
        ],
        "type": "unavailable_shards_exception",
        "reason": "[example][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[example][0]] containing [index {[example][_doc][1], source[{\n    \"content\": \"Hello world\"\n}]}]]"
    },
    "status": 503
}

gabriel_tessier · April 18, 2019, 1:55am

Hi,
you don't need to close the index to put settings and mapping.

Here the documentation about how to create an index and set settings and mapping: https://www.elastic.co/guide/en/elasticsearch/reference/7.0/indices-create-index.html

Can you first try with curl something like to add a document:

curl -XPOST 10.0.0.1:9200/clef2018/_doc/1 -d '{"content": "some content"}'

You're suppose to get your indexed document with curl -XGET 10.0.0.1:9200/clef2018/_doc/1

If it's work it mean that elastic is ok and there's a problem with your script but so far it look correct.

When you write "checked that no document indexed" how do you check?

DavidTurner · April 18, 2019, 5:16am

[example][0] primary shard is not active

I think this means your cluster health will be red. Here is an article about how to work out why:

lemon_soft · April 18, 2019, 7:40am

Hi,
Thank you for your reply. I followed your comment and instructions, now indexing is ok. I gave settings while creating index. I fixed a little problem in mapping string. Settings and mappings are succeeded:

{"settings":{"number_of_shards":1,"number_of_replicas":0,"analysis":{"analyzer":{"my_english":{"tokenizer":"standard","filter":["lowercase","terrier_stopwords","porter_stem"]}},"filter":{"terrier_stopwords":{"type":"stop","stopwords_path":"stopwords/terrier-stop.txt"}}},"similarity":{"bm25_content":{"type":"BM25","b":0.75,"k1":1.2}}},"mappings":{"docType":{"_source":{"enabled":true},"properties":{"content":{"type":"text","similarity":"bm25_content","analyzer":"my_english"},"is_html":{"type":"boolean"}}}}}

{"acknowledged":true,"shards_acknowledged":true,"index":"clef2018"}

I added doc with id=1. I checked "count" and saw nothing:

curl http://10.0.0.1:9200/clef2018/_doc/_count

{"count":0,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}

The result showed id=1 is stored:

curl -XGET 10.0.0.1:9200/clef2018/_doc/1

{"_index":"clef2018","_type":"_doc","_id":"1","_version":9,"_seq_no":74,"_primary_term":2,"found":true,"_source":{ "content": "Hello world"}}

Search returned nothing

curl http://10.0.0.1:9200/clef2018/_doc/_search?pretty=true

{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 0, "max_score" : null, "hits" : [ ] } }

What should I check?
Thanks.

gabriel_tessier · April 18, 2019, 8:58am

Hi,

Do you launch your command from a script?
If so you can add a refresh call after index to force the refresh and make the document available for search and count.

curl http://10.0.0.1:9200/clef2018/_refresh

lemon_soft · April 18, 2019, 9:18am

Hi,
Sorry, refresh command didn't succeed. Still count is zero. I deleted index and created again, but nothing worked

gabriel_tessier · April 18, 2019, 9:26am

There's something strange in your test

{"_index":"clef2018","_type":"_doc","_id":"1","_version":9,"_seq_no":74,"_primary_term":2,"found":true,"_source":{ "content": "Hello world"}}

Here you have _version == 9 which mean that you update your doc with id 1, 9 times and _seq_no ==74 which mean that you also insert/delete/update 74 (-9 for the document 1) times other documents???

the command you write are really the one you run? if so there's something strange with the meta fields.

lemon_soft · April 18, 2019, 10:00am

Yes. I ran it repeatedly. Here is fresh result after deleting index and creating again:

{"_index":"clef2018","_type":"_doc","_id":"1","_version":1,"_seq_no":0,"_primary_term":2,"found":true,"_source":{ "content": "Hello world" }}

Should I upgrade 6.7 to 7.0?

gabriel_tessier · April 18, 2019, 10:31am

I tried with your data there's a problem in your settings definition actually you wrote

,"mappings":{"docType": <--- but you insert in _doc which make confusion...

Here the command I run and I have result for count and search. I only removed the call for terrier_stopwords as I don't have the file.

Can you confirm that it's work for you. I tested on a 6.5.x

PUT clef2018
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "analysis": {
      "analyzer": {
        "my_english": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "porter_stem"
          ]
        }
      }
    },
    "similarity": {
      "bm25_content": {
        "type": "BM25",
        "b": 0.75,
        "k1": 1.2
      }
    }
  },
  "mappings": {
    "docType": {
      "_source": {
        "enabled": true
      },
      "properties": {
        "content": {
          "type": "text",
          "similarity": "bm25_content",
          "analyzer": "my_english"
        },
        "is_html": {
          "type": "boolean"
        }
      }
    }
  }
}

POST clef2018/docType/1
{ "content": "Hello world"}

GET /clef2018/docType/1

GET /clef2018/docType/_count

GET /clef2018/docType/_search

lemon_soft · April 18, 2019, 10:49am

You are exactly right. The problem is in setting string. I checked the website which gives the settings and changed docType.

Thanks for all your efforts @gabriel_tessier

adding worked:

PUT _doc/1
{"content": "Hello world"}

searching worked:

GET _doc/_search?pretty=true
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 1.0, "hits" : [ { "_index" : "example", "_type" : "_doc", "_id" : "1", "_score" : 1.0, "_source" : { "content" : "Hello world" } } ] } }

system · May 16, 2019, 10:49am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
UnavailableShardsException When Creating Index With Settings Elasticsearch	6	1714	July 6, 2017
Unavailable_shards_exception, reason=index_xyz 0 primary shard is not active Timeout: 1m Elasticsearch	1	598	December 25, 2019
Raise bulkindexerror Elasticsearch language-clients	1	579	January 9, 2022
Unavailable_shards_exception Elasticsearch elastic-stack-alerting	1	245	January 11, 2024
Create mapping error！——primary shard is not active Timeout Elasticsearch	2	8307	July 7, 2017

After putting index setting, cannot add document

Related topics