Dear fellow user,
To evaluate the search functions by certain queries and the documents, I have a dataset with relevances of documents, related to a certain query.
I have filled it in in the Kibana console, having the following structure:
GET /financialtimes/_rank_eval
{
"requests": [
{
"id": "Query_Number_351 ",
"request": {
"query": { "match": { "text": " Falkland petroleum exploration " }}
},
"ratings": [
{ "_index": "financialtimes", "_id": "FT911-1098", "rating": 0 },
[ ... ]
{ "_index": "financialtimes", "_id": "FT911-1", "rating": 0 }
]}],
"metric": {
"precision": {
"k" : 10,
"relevant_rating_threshold": 1,
"ignore_unlabeled": false
}
}
}
The above code structure was run in one go to execute 10 different query requests to get the precisions. The console has successfully given the output, containing the precision of each query and the documents the search algorithm had retrieved.
However, after four successful executions of the command, the console gave no output at all. My educated guess goes no further than a crash of the Ranking Evaluation API, though I could not manage to confirm this theory, nor solve the issue.
After the console not seeming to handle our command, I tried to run the same command for one query.
The same applies to this: for some commands, it works, though it seems that executing the queries to fast in succession causes problems.
Command adaptions tried:
Adapting k-value of the precision metrics:
Each value from 10 to 1.
Results:
It could happen that the command did not do anything at all (No console output)
It could also happen that there were no problems at all.
It could happen the console stops working at a middle value (e.g., for k values k=10, k=9, k=8 it was a success. After that no commands on the specific query could be executed, also not for k=10, k=9 or k=8)
Adapting Command by adapting query and ratings:
Same results, so:
It could happen that the command did not do anything at all (No console output)
It could also happen that there were no problems at all.
It could happen the console stops working at a middle value (e.g., for k values k=10, k=9, k=8 it was a success. After that no commands on the specific query could be executed, also not for k=10, k=9 or k=8)
I tried:
Restarting the server by shutting down the Ubuntu command screens it is running on.
Ubuntu from Windows Store. Version: 1804.2018.817.0
Anyone has an idea how to solve this issue?