ES 1.7 completion suggester throws zero results

[deltafarce]

I'm aware of 1.7 reaching EOL but I have to make this work on it.

What I defined in the mapping:

"mappings": {
        "questions": {
            "_id": {"store": "yes", "path": "id"},
            "_source": {"enabled": True},
            "properties": {
                "id": {"type": "integer"},
                "question": {
                    "type": "string",
                    "search_analyzer": "query_analyzer",
                    "index_analyzer": "ngram_analyzer",
                    "fields": {
                        "raw": {"type": "string", "index": "analyzed"}
                    }
                },
                "answer": {"type": "string"},
                "category": {
                    "type": "nested",
                    "properties": {
                        "id": {"type": "integer"},
                        "name": {"type": "string"}
                    }
                },
                "suggest": {
                    "type": "completion",
                    "index_analyzer": "simple",
                    "search_analyzer": "simple",
                    "payloads": True
                }
            }
        }
    }

What I'm indexing:

{
  'id': 504, 
  'question': 'What all Government Jobs can I target?', 
  'answer': '1. Public Prosecutor\n2. Standing Counsel\n3. Legal experts\n4. Judge\nand many more.', 
  'category': [{'id': 13, 'name': 'General'}], 
  'suggest': {
    'input': ['government', 'jobs', 'target'], 
    'output': 'What all Government Jobs can I target?', 
    'payload': {'id': 504}, 
    'weight': 1
   }
}

Note that I'm using the elasticsearch-py Python library to do the mapping and indexing. The mapping code is as such:

    self.base_index_name = 'faqs'
    self.doc_type = 'question'
    self.index_client = IndicesClient(self.es)
    self.index_name = self._create_lmscq_index() 
    qs = Questions.objects.all().distinct()
    itemcount = 0
    for question in qs:
        try:
            print("Indexing",itemcount,"/",qs.count(),end="\r")
            itemcount += 1
            question_dict = self.prep_q(question) # returns the document above in Python dictionary format
            self.index_q(question_dict, delete_elastic_cache=False)
            sys.stdout.flush()
        except Exception as ex:
            print("reindex_error", ex)
            import traceback
            traceback.print_tb(ex.__traceback__)

    print(itemcount,"FAQs indexed")
    self._create_alias()
    self.index_client.clear_cache()

The function self.index_q() is defined as such:

def index_q(self, q_body, delete_elastic_cache=True):

    self.es.index(
        index=self.index_name,
        doc_type=self.doc_type,
        id=q_body['id'],
        body=q_body,
        timeout=20
    )

    if delete_elastic_cache:
        self.index_client = IndicesClient(self.es)
        self.index_client.clear_cache(id=q_body['id'])

And finally, what I'm querying:

POST http://localhost:9200/faqs/_suggest -d "

{
"question_suggest": {
	"text": "government jobs target",
	"completion": {
		"field": "suggest"
			}
	}	
}
"

This is all I'm getting from Elasticsearch:

    {
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "question_suggest": [
        {
            "text": "government jobs target",
            "offset": 0,
            "length": 22,
            "options": []
        }
    ]
}

Why is ES throwing blanks to me?

[spinscale (Alexander Reelsen)]

I'll say this only once, but it needs to be said. Do not run software that was released four years ago. A lot of bugs have been fixed in the meantime.

The completion suggester is a prefix suggester, so a term like government jobs target also requires a similar input and not three different ones

[deltafarce]

@spinscale

a term like government jobs target also requires a similar input and not three different ones

From my code,

@deltafarce

'suggest': 
    { 
        'input': ['government', 'jobs', 'target'], 
        'output': 'What all Government Jobs can I target?', 
        'payload': {'id': 504}, 
        'weight': 1 
    }

I am sending three separate words to the input field, so it should essentially appear if I type even a part of the words government, jobs or targetalone rather than all together. Irrespective, the results do not show up.

To check if the suggest module is working on our ES instance, I tried the example here. It works perfectly, so this is a weird case.

@spinscale

The completion suggester is a prefix suggester

Okay, so by the looks of it this is probably not the best way to achieve what I need. I'll put it here just in case anybody has a better idea.

I'm basically enabling ''search-as-you-type" ability for a set of frequently asked questions, which should search my index based on some keywords. I'm using a stem, token and stop analyzer in both indexing and querying to ensure only the exact keywords are matched - however, it throws back almost the entire index (400+ hits in an index of 500), and after the first 5-10 the other results have nothing to do with the text typed at all.

e.g. "How do I prepare for the GRE?" becomes ["prepar", "gre"] in both the index and query analyzer. It should only give questions related to preparing for the GRE, but instead it also throws up unrelated exams as well as FAQs that have nothing to do with the concern. I had set up completion suggesters to tackle this very problem.

@spinscale

I'll say this only once, but it needs to be said. Do not run software that was released four years ago. A lot of bugs have been fixed in the meantime.

Trust me, this isn't my first rodeo. I've been refused to.