Our application programmatically creates search engines per environment/client. Just recently we started getting errors because the 'engine name is already taken'. Within our account such a duplicate engine name does not exist. Is the engine name supposed to be globally unique across all elastic users? Even so, our naming convention is already quite unique so we're doubtful that someone else really has the same name (e.g. np-local-search-np-lt-11-brisbane). Is this a bug or are we doing something wrong?
I believe engine name indeed should be unique within the Elasticsearch deployment that App Search is connecting to.
All I could think of is that there is already an engine in your Elasticsearch deployment created with the specified name.
Can you check in the interface, that such engine does not exist for your deployment? If so, can you add it using App Search UI, or the problem persists only for the API?
Do you think it's possible to provide the steps (for example, API requests/responses) that lead to such behaviour?
thanks for replying, there is no existing engine and the issue occurs even when I try adding via UI. Its difficult to replicate because most of the engines create fine its just randomly we've had 2 that 'already exist'. The only thing I can say is that we originally attempted to create via API and if it did not work then we would try via the UI.
Do you still have a setup that you can use to reproduce the problem with specific index name?
Things that I'm interested about are:
- What is the name of the engine that is unable to be created?
- How many engines do you have in the deployment?
- Can you share an example API request that triggers it?
- (the hardest one) Can you share the list of engine names that already exist in the deployment?
I checked the code and haven't found anything suspicious for now - the code just checks the uniqueness of name field for the index that stores engines.
Hi
Name of engine unable to be created is np-local-search-np-lt-11-brisbane
12 engines in the deployment
here is list of engines in UI
here is response from list engines API
example create request
I see, I did some investigation and found the following:
The logic of uniqueness for some field values is using additional indices to store its data. E.g. if you create an engine, there's another record in another hidden index that will be used later for the code that enforces the uniqueness constraint.
It's possible sometimes to have an issue when you delete an engine from App Search, but the index that contains uniqueness constraint logic is not properly updated.
You can try to check the index that enforces uniqueness constraints to see if there's a problem with it.
As I'm not sure what development version you are running, I'll give you approximate steps how to check it:
- Go to Kibana Dev Tools -> Console
- Execute the following request to see the indices that store the engines and uniqueness constraints:
GET _cat/indices/.ent-search-actastic-engines*
For example, you'll see the following response:
green open .ent-search-actastic-engines_v26-loco_moco_account_id-slug-unique-constraint _XQAVSvXRPKMsgtVqSeiYg 1 0 5 1 16.2kb 16.2kb
green open .ent-search-actastic-engines_v26 qHYXjr8ySpexuvUZ7f_MNA 1 0 5 0 63kb 63kb
The third column is the one that is interesting to us.
- Pick the shorter name from two and execute the request:
GET .ent-search-actastic-engines_v26/_search # Here you might have a different value after *engines_v, it's okay
{
"size": 2000
}
Check the results and verify, that you see the same entries that you see in App Search UI
- Now execute the request for uniqueness constraint:
GET .ent-search-actastic-engines_v26-loco_moco_account_id-slug-unique-constraint/_search # You might also have a slightly different name, specifically the v26 part
{
"size": 2000
}
Check the results and see if there's any record that indicates that a constraint should be applied, but points to non-existing engine.
If it's the case, you might need to manually delete the entry that is problematic.
Please let me know if it helps and we'll dig deeper into this if it won't help.
Seems that I have multiple versions of indexes, is that normal? Should I try to delete all but the latest (v25) ?
green open .ent-search-actastic-engines_v20-loco_moco_account_id-slug-unique-constraint fetIo--kQqyOnEQQGjAWFw 1 1 1 0 7.5kb 3.7kb
green open .ent-search-actastic-engines_v25 MiQYvQMYSFWpuokod2koRg 1 1 12 2 107.6kb 53.7kb
green open .ent-search-actastic-engines_v18 CW-mshFuQiWCklBb5XU_GQ 1 1 1 0 23.1kb 11.5kb
green open .ent-search-actastic-engines_v18-loco_moco_account_id-slug-unique-constraint eLMvxG5AQImIwaoT1DnffA 1 1 1 0 7.4kb 3.7kb
green open .ent-search-actastic-engines_v19 bnoYSZKyQsuYFfQS0ko75w 1 1 1 0 24.5kb 12.2kb
green open .ent-search-actastic-engines_v23 59KzJ_YJS5uvFTW7t4yg6A 1 1 3 0 21.7kb 10.8kb
green open .ent-search-actastic-engines_v17-loco_moco_account_id-slug-unique-constraint KWsCaz5kQJa8SYtOuk4a6g 1 1 1 0 7.4kb 3.7kb
green open .ent-search-actastic-engines_v20 lX3sgE09QRW5PB5RBGQD1A 1 1 1 0 22.7kb 11.3kb
green open .ent-search-actastic-engines_v25-loco_moco_account_id-slug-unique-constraint UO0hbT0vTwi_ZhDeTseIeQ 1 1 12 2 22.2kb 11.1kb
green open .ent-search-actastic-engines_v15 vaImhRotQr2p2FZelgOPCg 1 1 1 9 28.9kb 14.4kb
green open .ent-search-actastic-engines_v16-loco_moco_account_id-slug-unique-constraint 6mZGwhgUSgypfk8xMPKRhg 1 1 1 0 7.4kb 3.7kb
green open .ent-search-actastic-engines_v24 IlYSNglSSFuesEDlNa0DNw 1 1 3 0 18.8kb 9.4kb
green open .ent-search-actastic-engines_v22 t04mqTxMQ9-MHbP2aPGW_w 1 1 3 9 51.1kb 25.5kb
green open .ent-search-actastic-engines_v16 _mproe__Sz-w5gXEmvsduw 1 1 1 0 18.4kb 9.2kb
green open .ent-search-actastic-engines_v15-account_id-loco_moco_account_id-slug-unique-constraint mhj66ovzQlG_A0aTWNDmRQ 1 1 1 0 7.5kb 3.7kb
green open .ent-search-actastic-engines_v17 RW4yCESURB-f2FVdvZib6g 1 1 1 0 19.2kb 9.6kb
green open .ent-search-actastic-engines_v24-loco_moco_account_id-slug-unique-constraint QRJUVkaRQ0WuzXfl9hASdQ 1 1 3 0 8.4kb 4.2kb
green open .ent-search-actastic-engines_v21 pjDb_JPyTcKh3qN1DMG2Bw 1 1 1 0 22.6kb 11.3kb
green open .ent-search-actastic-engines_v21-loco_moco_account_id-slug-unique-constraint vrENKRNPTFeUUuVPqQiIUQ 1 1 1 0 7.5kb 3.7kb
green open .ent-search-actastic-engines_v15-key-unique-constraint ntkF8kT0TiSWMADE1B_Tzw 1 1 1 0 7.2kb 3.6kb
green open .ent-search-actastic-engines_v23-loco_moco_account_id-slug-unique-constraint D1cyiMFqT4yiYbjaWswKKw 1 1 3 0 8.4kb 4.2kb
green open .ent-search-actastic-engines_v19-loco_moco_account_id-slug-unique-constraint kyZXujGjR1uMPtBTudQNKQ 1 1 1 0 7.4kb 3.7kb
green open .ent-search-actastic-engines_v22-loco_moco_account_id-slug-unique-constraint oorX_oQdQKKzB-yM7lEQzA 1 1 3 0 8.6kb 4.3kb
I've checked inside the latest (v25) and it does not contain the engine name that is causing me issues to create
Seems that I have multiple versions of indexes, is that normal? Should I try to delete all but the latest (v25) ?
If you don't have problems with amount of space taken, I'd probably keep them - they might help troubleshoot the issues if any happen in future.
I've checked inside the latest (v25) and it does not contain the engine name that is causing me issues to create
I see. Seems like the problem is deeper, than it seems. Do you think you can set log_level: debug in the configuration of Enterprise Search, do the request to the API and share the corresponding log entry here?
I think the queries that are done to Elasticsearch can give us a hint about what's going on.
To do so you will need to do 2 things:
-
Enable log shipping to use enhanced logging.
You need to log in into your cloud account, choose the deployment and go to "Logs and metrics" section of the deployment. There you need to enable "Ship to deployment option" - it will require you to have another deployment to ship logs to:
-
Change
log_levelsetting for the original deployment.
To do it you will need to go to "Edit" section of your deployment, scroll down to Enterprise Search and click on "Edit user settings":
Here you can enter the log_level setting:
If you haven't enabled the log shipping option, this step will fail, as amount of logs that will be generated will be quite huge.
this seems to be the only debug log generated which contains the name of the engine I'm trying to create, I'm not sure what it tells though
{
"request": {
"url": "http://7b5bbc27bf784cbfa6bcc89e447d7fb6.containerhost:9244/.ent-search-actastic-search_indices_v1/_search?request_cache=true",
"method": "get",
"headers": {
"X-Elastic-App-Auth": "[FILTERED]",
"Authorization": "[FILTERED]",
"Content-Type": "application/json",
"x-elastic-product-origin": "enterprise-search",
"User-Agent": "Faraday v1.8.0",
"X-Cloud-Delegated-Request-Id": "Dy0RvMwQRX6Jko5DveKhMA"
},
"params": null,
"body": "{\"query\":{\"bool\":{\"filter\":[{\"terms\":{\"name\":[\"np-local-search-np-lt-11-brisbane\"]}}]}},\"sort\":[\"_doc\"],\"size\":100,\"from\":0,\"seq_no_primary_term\":true}"
},
"response": {
"status": 200,
"headers": {
"content-type": "application/json",
"x-cloud-request-id": "mxKvXS0eQoieZZGGaddp4Q",
"x-elastic-product": "Elasticsearch",
"x-found-handling-cluster": "7b5bbc27bf784cbfa6bcc89e447d7fb6",
"x-found-handling-instance": "instance-0000000000",
"date": "Mon, 27 Jun 2022 11:58:10 GMT"
},
"body": "{\"took\":2,\"timed_out\":false,\"_shards\":{\"total\":1,\"successful\":1,\"skipped\":0,\"failed\":0},\"hits\":{\"total\":{\"value\":5,\"relation\":\"eq\"},\"max_score\":null,\"hits\":[{\"_index\":\".ent-search-actastic-search_indices_v1\",\"_id\":\"62a5eb039f2108f23200ede7\",\"_seq_no\":119,\"_primary_term\":2,\"_score\":null,\"_source\":{\"id\":\"62a5eb039f2108f23200ede7\",\"created_at\":\"2022-06-12T13:32:51Z\",\"updated_at\":\"2022-06-12T13:32:51Z\",\"type_\":\"SearchIndex::ManagedSearchIndex\",\"name\":\"np-local-search-np-lt-11-brisbane\",\"index_name\":\".ent-search-engine-documents-np-local-search-np-lt-11-brisbane\",\"language\":null,\"field_mapping_hash\":{},\"unconfirmed_fields\":null,\"index_settings_override\":{},\"index_create_settings_override\":{},\"content_source_id\":null},\"sort\":[9]},{\"_index\":\".ent-search-actastic-search_indices_v1\",\"_id\":\"62a5eb034b8da02662378973\",\"_seq_no\":120,\"_primary_term\":2,\"_score\":null,\"_source\":{\"id\":\"62a5eb034b8da02662378973\",\"created_at\":\"2022-06-12T13:32:51Z\",\"updated_at\":\"2022-06-12T13:32:51Z\",\"type_\":\"SearchIndex::ManagedSearchIndex\",\"name\":\"np-local-search-np-lt-11-brisbane\",\"index_name\":\".ent-search-engine-documents-np-local-search-np-lt-11-brisbane\",\"language\":null,\"field_mapping_hash\":{},\"unconfirmed_fields\":null,\"index_settings_override\":{},\"index_create_settings_override\":{},\"content_source_id\":null},\"sort\":[10]},{\"_index\":\".ent-search-actastic-search_indices_v1\",\"_id\":\"62a5eb034b8da0438a378975\",\"_seq_no\":121,\"_primary_term\":2,\"_score\":null,\"_source\":{\"id\":\"62a5eb034b8da0438a378975\",\"created_at\":\"2022-06-12T13:32:51Z\",\"updated_at\":\"2022-06-12T13:32:51Z\",\"type_\":\"SearchIndex::ManagedSearchIndex\",\"name\":\"np-local-search-np-lt-11-brisbane\",\"index_name\":\".ent-search-engine-documents-np-local-search-np-lt-11-brisbane\",\"language\":null,\"field_mapping_hash\":{},\"unconfirmed_fields\":null,\"index_settings_override\":{},\"index_create_settings_override\":{},\"content_source_id\":null},\"sort\":[11]},{\"_index\":\".ent-search-actastic-search_indices_v1\",\"_id\":\"62a5eb039f21088bb900ede9\",\"_seq_no\":122,\"_primary_term\":2,\"_score\":null,\"_source\":{\"id\":\"62a5eb039f21088bb900ede9\",\"created_at\":\"2022-06-12T13:32:51Z\",\"updated_at\":\"2022-06-12T13:32:51Z\",\"type_\":\"SearchIndex::ManagedSearchIndex\",\"name\":\"np-local-search-np-lt-11-brisbane\",\"index_name\":\".ent-search-engine-documents-np-local-search-np-lt-11-brisbane\",\"language\":null,\"field_mapping_hash\":{},\"unconfirmed_fields\":null,\"index_settings_override\":{},\"index_create_settings_override\":{},\"content_source_id\":null},\"sort\":[12]},{\"_index\":\".ent-search-actastic-search_indices_v1\",\"_id\":\"62a5eb039f2108d4d200ede8\",\"_seq_no\":123,\"_primary_term\":2,\"_score\":null,\"_source\":{\"id\":\"62a5eb039f2108d4d200ede8\",\"created_at\":\"2022-06-12T13:32:51Z\",\"updated_at\":\"2022-06-12T13:32:51Z\",\"type_\":\"SearchIndex::ManagedSearchIndex\",\"name\":\"np-local-search-np-lt-11-brisbane\",\"index_name\":\".ent-search-engine-documents-np-local-search-np-lt-11-brisbane\",\"language\":null,\"field_mapping_hash\":{},\"unconfirmed_fields\":null,\"index_settings_override\":{},\"index_create_settings_override\":{},\"content_source_id\":null},\"sort\":[13]}]}}"
},
"duration": 7.5,
"stack": ["lib/actastic/schema.class:55:in `search'", "lib/actastic/relation.class:538:in `block in search'", "lib/apm_helpers.class:52:in `es_action_instrument'", "lib/apm_helpers.class:57:in `actastic_instrument'", "lib/actastic/relation.class:518:in `instrument'", "lib/actastic/relation.class:537:in `search'", "lib/actastic/relation.class:225:in `find_each'", "lib/actastic/relation.class:307:in `to_a'", "lib/actastic/relation.class:307:in `load'", "lib/actastic/relation.class:302:in `to_a'", "lib/actastic/relation.class:323:in `each'", "lib/actastic/relation.class:134:in `any?'", "lib/actastic/relation.class:134:in `exists?'", "lib/actastic/validations.class:28:in `validate_each'", "lib/actastic/validations.class:45:in `valid?'", "lib/actastic/actastic_record.class:93:in `save'", "lib/actastic/actastic_record.class:107:in `save!'", "lib/actastic/actastic_record/class_methods.class:351:in `tap'", "lib/actastic/actastic_record/class_methods.class:351:in `create!'", "loco_moco/lib/forms/loco_moco/create_engine.class:87:in `create_search_index'", "loco_moco/lib/forms/loco_moco/create_engine.class:59:in `block in create_engine!'", "loco_moco/lib/forms/loco_moco/create_engine.class:55:in `tap'", "loco_moco/lib/forms/loco_moco/create_engine.class:55:in `create_engine!'", "loco_moco/lib/forms/loco_moco/create_engine.class:35:in `save'", "loco_moco/app/controllers/loco_moco/shared/concerns/engines_controller_concern.class:11:in `create'", "loco_moco/app/controllers/loco_moco/api/base_controller.class:29:in `rescue_with_logging'", "app/controllers/concerns/request_audit_logger_concern.class:36:in `audit_request'", "app/controllers/concerns/request_audit_logger_concern.class:22:in `audit_request_authentication'", "vendor/fishwife-servlet/lib/fishwife/rack_servlet.rb:74:in `service'"]
}
It actually exposed the place where the uniqueness constraint takes place, it's this index: .ent-search-actastic-search_indices_v1.
I suppose, that you can query it to confirm that the index is there:
GET .ent-search-actastic-search_indices_v1/_search
{
"query": {
"term": {
"name": {
"value": "np-local-search-np-lt-11-brisbane"
}
}
}
}
If the index is there, you can then drop it:
DELETE .ent-search-actastic-search_indices_v1/_doc/<INDEX_ID_FROM_PREVIOUS_STEP>
It's something that can happen when Enterprise Search is overloaded, crashes in the middle of index creation or if Elasticsearch goes away during creation of index. It's not something that should happen during normal work with Enterprise Search, though.
there is 5 results returned from the query
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 5,
"relation" : "eq"
},
"max_score" : 2.1238933,
"hits" : [
{
"_index" : ".ent-search-actastic-search_indices_v1",
"_id" : "62a5eb039f2108f23200ede7",
"_score" : 2.1238933,
"_source" : {
"id" : "62a5eb039f2108f23200ede7",
"created_at" : "2022-06-12T13:32:51Z",
"updated_at" : "2022-06-12T13:32:51Z",
"type_" : "SearchIndex::ManagedSearchIndex",
"name" : "np-local-search-np-lt-11-brisbane",
"index_name" : ".ent-search-engine-documents-np-local-search-np-lt-11-brisbane",
"language" : null,
"field_mapping_hash" : { },
"unconfirmed_fields" : null,
"index_settings_override" : { },
"index_create_settings_override" : { },
"content_source_id" : null
}
},
{
"_index" : ".ent-search-actastic-search_indices_v1",
"_id" : "62a5eb034b8da02662378973",
"_score" : 2.1238933,
"_source" : {
"id" : "62a5eb034b8da02662378973",
"created_at" : "2022-06-12T13:32:51Z",
"updated_at" : "2022-06-12T13:32:51Z",
"type_" : "SearchIndex::ManagedSearchIndex",
"name" : "np-local-search-np-lt-11-brisbane",
"index_name" : ".ent-search-engine-documents-np-local-search-np-lt-11-brisbane",
"language" : null,
"field_mapping_hash" : { },
"unconfirmed_fields" : null,
"index_settings_override" : { },
"index_create_settings_override" : { },
"content_source_id" : null
}
},
{
"_index" : ".ent-search-actastic-search_indices_v1",
"_id" : "62a5eb034b8da0438a378975",
"_score" : 2.1238933,
"_source" : {
"id" : "62a5eb034b8da0438a378975",
"created_at" : "2022-06-12T13:32:51Z",
"updated_at" : "2022-06-12T13:32:51Z",
"type_" : "SearchIndex::ManagedSearchIndex",
"name" : "np-local-search-np-lt-11-brisbane",
"index_name" : ".ent-search-engine-documents-np-local-search-np-lt-11-brisbane",
"language" : null,
"field_mapping_hash" : { },
"unconfirmed_fields" : null,
"index_settings_override" : { },
"index_create_settings_override" : { },
"content_source_id" : null
}
},
{
"_index" : ".ent-search-actastic-search_indices_v1",
"_id" : "62a5eb039f21088bb900ede9",
"_score" : 2.1238933,
"_source" : {
"id" : "62a5eb039f21088bb900ede9",
"created_at" : "2022-06-12T13:32:51Z",
"updated_at" : "2022-06-12T13:32:51Z",
"type_" : "SearchIndex::ManagedSearchIndex",
"name" : "np-local-search-np-lt-11-brisbane",
"index_name" : ".ent-search-engine-documents-np-local-search-np-lt-11-brisbane",
"language" : null,
"field_mapping_hash" : { },
"unconfirmed_fields" : null,
"index_settings_override" : { },
"index_create_settings_override" : { },
"content_source_id" : null
}
},
{
"_index" : ".ent-search-actastic-search_indices_v1",
"_id" : "62a5eb039f2108d4d200ede8",
"_score" : 2.1238933,
"_source" : {
"id" : "62a5eb039f2108d4d200ede8",
"created_at" : "2022-06-12T13:32:51Z",
"updated_at" : "2022-06-12T13:32:51Z",
"type_" : "SearchIndex::ManagedSearchIndex",
"name" : "np-local-search-np-lt-11-brisbane",
"index_name" : ".ent-search-engine-documents-np-local-search-np-lt-11-brisbane",
"language" : null,
"field_mapping_hash" : { },
"unconfirmed_fields" : null,
"index_settings_override" : { },
"index_create_settings_override" : { },
"content_source_id" : null
}
}
]
}
}
I deleted all of them, however the problem persists.
I can confirm that they were deleted because when I try to delete them again it returns result not_found
The query still returns the same 5 results
hi @Artem_Shelkovnikov could you let me know what to try next please
actually, don't worry it seems to be working now, maybe just needed some time to flush through, thanks
Hi Rav, sorry for a bit of delay from me.
Happy to hear it's working now!
I think in general the best way to debug such issues is to enable the logs in DEBUG mode, run the query that produces unexpected results and see what queries were issued to Elasticsearch.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.