After putting index setting, cannot add document

I failed to create my ElasticSearch index, again. Here are my steps:

  • created index with name clef2018, number of shard=1, number of replicas=0
  • closed index
  • put settings of clef2018 (website)
  • put mappings of clef2018 (website)
  • opened index
  • checked that no document indexed
  • cannot added document with bulk API or python
  • also search failed

Is it an ElasticSearch issue? My python script is:

from elasticsearch import Elasticsearch
title="title"
body="body"
es = Elasticsearch([{"host": "10.0.0.1", "port": 9200}])
contentt = {'content': title +"\n"+ body}
res = ""
idd=idd+1
try:
	res = es.index(index='clef2018', doc_type='_doc', id=idd, body=contentt)
except:
	print("error:" + str(res))

Error is:

{
    "error": {
        "root_cause": [
            {
                "type": "unavailable_shards_exception",
                "reason": "[example][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[example][0]] containing [index {[example][_doc][1], source[{\n    \"content\": \"Hello world\"\n}]}]]"
            }
        ],
        "type": "unavailable_shards_exception",
        "reason": "[example][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[example][0]] containing [index {[example][_doc][1], source[{\n    \"content\": \"Hello world\"\n}]}]]"
    },
    "status": 503
}

Hi,
you don't need to close the index to put settings and mapping.

Here the documentation about how to create an index and set settings and mapping: https://www.elastic.co/guide/en/elasticsearch/reference/7.0/indices-create-index.html

Can you first try with curl something like to add a document:

curl -XPOST 10.0.0.1:9200/clef2018/_doc/1 -d '{"content": "some content"}'

You're suppose to get your indexed document with curl -XGET 10.0.0.1:9200/clef2018/_doc/1

If it's work it mean that elastic is ok and there's a problem with your script but so far it look correct.

When you write "checked that no document indexed" how do you check?

1 Like

[example][0] primary shard is not active

I think this means your cluster health will be red. Here is an article about how to work out why:

2 Likes

Hi,
Thank you for your reply. I followed your comment and instructions, now indexing is ok. I gave settings while creating index. I fixed a little problem in mapping string. Settings and mappings are succeeded:

{"settings":{"number_of_shards":1,"number_of_replicas":0,"analysis":{"analyzer":{"my_english":{"tokenizer":"standard","filter":["lowercase","terrier_stopwords","porter_stem"]}},"filter":{"terrier_stopwords":{"type":"stop","stopwords_path":"stopwords/terrier-stop.txt"}}},"similarity":{"bm25_content":{"type":"BM25","b":0.75,"k1":1.2}}},"mappings":{"docType":{"_source":{"enabled":true},"properties":{"content":{"type":"text","similarity":"bm25_content","analyzer":"my_english"},"is_html":{"type":"boolean"}}}}}

{"acknowledged":true,"shards_acknowledged":true,"index":"clef2018"}

I added doc with id=1. I checked "count" and saw nothing:

curl http://10.0.0.1:9200/clef2018/_doc/_count

{"count":0,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}}

The result showed id=1 is stored:

curl -XGET 10.0.0.1:9200/clef2018/_doc/1

{"_index":"clef2018","_type":"_doc","_id":"1","_version":9,"_seq_no":74,"_primary_term":2,"found":true,"_source":{ "content": "Hello world"}}

Search returned nothing :frowning:

curl http://10.0.0.1:9200/clef2018/_doc/_search?pretty=true

{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 0, "max_score" : null, "hits" : [ ] } }

What should I check?
Thanks.

Hi,

Do you launch your command from a script?
If so you can add a refresh call after index to force the refresh and make the document available for search and count.

curl http://10.0.0.1:9200/clef2018/_refresh

Hi,
Sorry, refresh command didn't succeed. Still count is zero. I deleted index and created again, but nothing worked :frowning:

There's something strange in your test

{"_index":"clef2018","_type":"_doc","_id":"1","_version":9,"_seq_no":74,"_primary_term":2,"found":true,"_source":{ "content": "Hello world"}}

Here you have _version == 9 which mean that you update your doc with id 1, 9 times and _seq_no ==74 which mean that you also insert/delete/update 74 (-9 for the document 1) times other documents???

the command you write are really the one you run? if so there's something strange with the meta fields.

Yes. I ran it repeatedly. Here is fresh result after deleting index and creating again:

{"_index":"clef2018","_type":"_doc","_id":"1","_version":1,"_seq_no":0,"_primary_term":2,"found":true,"_source":{ "content": "Hello world" }}

Should I upgrade 6.7 to 7.0?

I tried with your data there's a problem in your settings definition actually you wrote

,"mappings":{"docType": <--- but you insert in _doc which make confusion...

Here the command I run and I have result for count and search. I only removed the call for terrier_stopwords as I don't have the file.

Can you confirm that it's work for you. I tested on a 6.5.x

PUT clef2018
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "analysis": {
      "analyzer": {
        "my_english": {
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "porter_stem"
          ]
        }
      }
    },
    "similarity": {
      "bm25_content": {
        "type": "BM25",
        "b": 0.75,
        "k1": 1.2
      }
    }
  },
  "mappings": {
    "docType": {
      "_source": {
        "enabled": true
      },
      "properties": {
        "content": {
          "type": "text",
          "similarity": "bm25_content",
          "analyzer": "my_english"
        },
        "is_html": {
          "type": "boolean"
        }
      }
    }
  }
}

POST clef2018/docType/1
{ "content": "Hello world"}

GET /clef2018/docType/1

GET /clef2018/docType/_count

GET /clef2018/docType/_search
2 Likes

You are exactly right. The problem is in setting string. I checked the website which gives the settings and changed docType.

Thanks for all your efforts :relaxed: @gabriel_tessier

adding worked:

PUT _doc/1
{"content": "Hello world"}

searching worked:

GET _doc/_search?pretty=true
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 1.0, "hits" : [ { "_index" : "example", "_type" : "_doc", "_id" : "1", "_score" : 1.0, "_source" : { "content" : "Hello world" } } ] } }

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.