# Could not start some language models for NLP tasks **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-machine-learning\ **Created:** [February 22, 2022, 12:11pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888 "2022-02-22T12:11:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Ilia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilia/32/60042_2.png) [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Post date:** [February 22, 2022, 12:11pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/1 "2022-02-22T12:11:25Z")
I've installed Eland and have imported a model from huggingface as follows: ```auto eland_import_hub_model --url http://localhost:9200/ \ --hub-model-id HooshvareLab/bert-fa-zwnj-base-ner \ --task-type ner ``` I can see it in [http://localhost:5601/app/ml/trained\_models](http://localhost:5601/app/ml/trained_models) but when I try to start it either by using start icon or by this api `POST _ml/trained_models/hooshvarelab__bert-fa-zwnj-base-ner/deployment/_start` I get this error: ```auto { "error" : { "root_cause" : [ { "type" : "status_exception", "reason" : "Could not start trained model deployment, the following nodes failed with errors [{f5IiQjhhTs6-ewaKpQ8xFg=Validation Failed: 1: classification label [B_DAT] is not an entity I-O-B tag.;2: classification label [B_EVE] is not an entity I-O-B tag.;3: classification label [B_FAC] is not an entity I-O-B tag.;4: classification label [B_MON] is not an entity I-O-B tag.;5: classification label [B_PCT] is not an entity I-O-B tag.;6: classification label [B_PRO] is not an entity I-O-B tag.;7: classification label [B_TIM] is not an entity I-O-B tag.;8: classification label [I_DAT] is not an entity I-O-B tag.;9: classification label [I_EVE] is not an entity I-O-B tag.;10: classification label [I_FAC] is not an entity I-O-B tag.;11: classification label [I_MON] is not an entity I-O-B tag.;12: classification label [I_PCT] is not an entity I-O-B tag.;13: classification label [I_PRO] is not an entity I-O-B tag.;14: classification label [I_TIM] is not an entity I-O-B tag.;15: Valid entity I-O-B tags are [O, B_MISC, I_MISC, B_PER, I_PER, B_ORG, I_ORG, B_LOC, I_LOC];}]" } ], "type" : "status_exception", "reason" : "Could not start trained model deployment, the following nodes failed with errors [{f5IiQjhhTs6-ewaKpQ8xFg=Validation Failed: 1: classification label [B_DAT] is not an entity I-O-B tag.;2: classification label [B_EVE] is not an entity I-O-B tag.;3: classification label [B_FAC] is not an entity I-O-B tag.;4: classification label [B_MON] is not an entity I-O-B tag.;5: classification label [B_PCT] is not an entity I-O-B tag.;6: classification label [B_PRO] is not an entity I-O-B tag.;7: classification label [B_TIM] is not an entity I-O-B tag.;8: classification label [I_DAT] is not an entity I-O-B tag.;9: classification label [I_EVE] is not an entity I-O-B tag.;10: classification label [I_FAC] is not an entity I-O-B tag.;11: classification label [I_MON] is not an entity I-O-B tag.;12: classification label [I_PCT] is not an entity I-O-B tag.;13: classification label [I_PRO] is not an entity I-O-B tag.;14: classification label [I_TIM] is not an entity I-O-B tag.;15: Valid entity I-O-B tags are [O, B_MISC, I_MISC, B_PER, I_PER, B_ORG, I_ORG, B_LOC, I_LOC];}]" }, "status" : 500 } ``` Why I get this error? p.s. - Among almost 10 models (from huggingface hub) that I've tried to deploy/import and start, I could just get 3 of them working (including two elastic's models and `dslim/bert-base-NER-uncased`)! Others failed in importing time by eland or in start time in elastic/kibana! - I've noticed the models must be BERT or other generations of it - My Elasticsearch and Kibana version is 8.0.0 - and have activated `trial` license ---
**Author:** ![dkyle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkyle/32/59114_2.png) [@dkyle](https://discuss.elastic.co/u/dkyle)\ **Post date:** [February 22, 2022, 2:19pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/2 "2022-02-22T14:19:50Z")
Hi Ilia Yes in the 8.0.0 release only BERT models are supported, other models including MPNet are coming in future releases. For a list of supported architectures refer to [Third party NLP models | Machine Learning in the Elastic Stack [8.0] | Elastic](https://www.elastic.co/guide/en/machine-learning/8.0/ml-nlp-model-ref.html) That page also recommends 3 models you can use for the Named Entity Recognition, I'm guessing those are the ones you had success with. The error you see is because the [I-O-B tagging](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)) schema is not recognised. The schema is expected to consist of the tags `B_MIS`, `I-MIS`, ... as used by this BERT model [dslim/bert-base-NER · Hugging Face](https://huggingface.co/dslim/bert-base-NER) The tags used by [your model](https://huggingface.co/HooshvareLab/bert-fa-zwnj-base-ner) such as `DAT` and `EVE` are not recognised. There is an open issue in the Elasticsearch repo to support different tagging schemas for NER, please comment on or +1 the issue so that is may be prioritised > > > The Named Entity Recognition task works for \[Inside-Outside-Beginning Tags\](http…s://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning\_(tagging)) but different models can be trained with a different tagging schemas or use extra tags. For example \[KB/bert-base-swedish-cased-ner\](https://huggingface.co/KB/bert-base-swedish-cased-ner/blob/main/config.json) has a different set of labels which makes it incompatible with the current NER task where the expectation that IOB tags are used is hard coded. > > It may be possible to add a configuration option for the different labelling schemas. Have you had any success using models for different tasks such as text classification? ---
**Author:** ![Ilia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilia/32/60042_2.png) [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Post date:** [February 22, 2022, 4:23pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/3 "2022-02-22T16:23:28Z")
Hi David, Thank you for the response, I've seen the [Third party NLP models](https://www.elastic.co/guide/en/machine-learning/8.0/ml-nlp-model-ref.html) page but I guessed it is sufficient for a model to be based on BERT or based on the few other architectures mentioned on that page. If I understand you correctly, there are a list of fixed tags that any compatible model must only use these tags, right? > That page also recommends 3 models you can use for the Named Entity Recognition, I'm guessing those are the ones you had success with. Yes, but I reached to these models almost randomly! Moreover, I get `Segmentation fault (core dumped)` error when tried to import `dslim/bert-base-NER` with `Eland` but could to import and start `dslim/bert-base-NER-uncased` successfully. Unfortunately, none of these 3 models help me and what I need is to deploy a NER model for Persian language (called also Farsi, fa), but it seems that there is no compatible model in huggingface ☹ > There is an open issue in the Elasticsearch repo to support different tagging schemas for NER, please comment on or +1 the issue so that is may be prioritised +1 ed. > Have you had any success using models for different tasks such as text classification? No, at the moment the `NER` is more important for us. We might to use `text classification` later. Oh, does these all means that just those 3 ner models are usable in Elasticsearch ner tasks?!! ---
**Author:** ![dkyle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkyle/32/59114_2.png) [@dkyle](https://discuss.elastic.co/u/dkyle)\ **Post date:** [February 23, 2022, 11:09am UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/4 "2022-02-23T11:09:43Z")
Can you tell me more about the core dump please. What OS are you using and CPU are you using? Did it occur when you started the deployment? > there are a list of fixed tags that any compatible model must only use these tags, right? Yes that is correct. > +1 ed. Thank you ---
**Author:** ![Ilia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilia/32/60042_2.png) [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Post date:** [February 23, 2022, 2:56pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/5 "2022-02-23T14:56:02Z")
> [@](#): > > Can you tell me more about the core dump please. What OS are you using and CPU are you using? Did it occur when you started the deployment? I get that error when I've tried to import model by eland on WSL. To reproduce the error to post here, I've executed the below command again but this time the model imported successfully! `eland_import_hub_model --url http://localhost:9200/ --hub-model-id dslim/bert-base-NER --task-type ner` We are trying to change/map tags of the `hooshvarelab/bert-fa-zwnj-base-ner` model to those are acceptable by elastic. Now, I need to know all acceptable tags by elastic for ner tasks, where I can find these tags? ---
**Author:** ![dkyle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkyle/32/59114_2.png) [@dkyle](https://discuss.elastic.co/u/dkyle)\ **Post date:** [February 24, 2022, 9:37am UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/6 "2022-02-24T09:37:59Z")
From [dslim/bert-base-NER · Hugging Face](https://huggingface.co/dslim/bert-base-NER) these are the tags used. | Abbreviation | Description | | --- | --- | | O | Outside of a named entity | | B-MIS | Beginning of a miscellaneous entity right after another miscellaneous entity | | I-MIS | Miscellaneous entity | | B-PER | Beginning of a person’s name right after another person’s name | | I-PER | Person’s name | | B-ORG | Beginning of an organization right after another organization | | I-ORG | organization | | B-LOC | Beginning of a location right after another location | | I-LOC | Location | I'm afraid I can't see a way to map those to the tags hooshvarelab/bert-fa-zwnj-base-ner uses as there is a different number of them ---
**Author:** ![Ilia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilia/32/60042_2.png) [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Post date:** [February 26, 2022, 1:02pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/7 "2022-02-26T13:02:13Z")
I guess we can map PER, ORG and LOC. Other tags will be mapped to MIS! Some thoughts: - we will miss a lot of useful tags with mapping them to MIS but have no other choice - we gave it a try but yet didn't completed the work - if supporting other tagging schemas is possible by just adding them to an existing list in Elasticsearch's source code, it might be possible to do so and build source code. Searching in the source code I found only [this](https://github.com/elastic/elasticsearch/blob/8.0/docs/reference/ml/trained-models/apis/get-trained-models.asciidoc) and [this one](https://github.com/elastic/elasticsearch/blob/8.0/docs/reference/ml/trained-models/apis/put-trained-models.asciidoc), but they are docs! - this work is kinda poc (prove of concept) and we might need to train our model ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 26, 2022, 1:03pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/8 "2022-03-26T13:03:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![dkyle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkyle/32/59114_2.png) [@dkyle](https://discuss.elastic.co/u/dkyle)\ **Post date:** [May 30, 2022, 3:52pm UTC](https://discuss.elastic.co/t/could-not-start-some-language-models-for-nlp-tasks/297888/9 "2022-05-30T15:52:47Z")
@Ilia A PR has just been merged in Elasticsearch that makes the entity tags configurable and will fix the problem you are having using the `hooshvarelab/bert-fa-zwnj-base-ner` model. It will be in the 8.4 release so keep an eye out for that. > > > Named entity recognition (NER) is a special form of token classification. The sp…ecific kind of labelling we support is Inside-Outside-Beginning (IOB) tagging. These labels indicate if they are the inside of a token (with a \`I-\` or \`I\_\`), the beginning (\`B-\` or \`B\_\`) or outside (\`O\`). > > Each valid token classification label starts with the require prefix or \`O\`. > > Before this commit, we restricted the labels to a specific set: > > \`\`\` > O(Entity.NONE), // Outside a named entity > B\_MISC(Entity.MISC), // Beginning of a miscellaneous entity right after another miscellaneous entity > I\_MISC(Entity.MISC), // Miscellaneous entity > B\_PER(Entity.PER), // Beginning of a person's name right after another person's name > I\_PER(Entity.PER), // Person's name > B\_ORG(Entity.ORG), // Beginning of an organization right after another organization > I\_ORG(Entity.ORG), // Organisation > B\_LOC(Entity.LOC), // Beginning of a location right after another location > I\_LOC(Entity.LOC); // Location > \`\`\` > > But now, any entity is allowed, as long as the naming of the labels adhere to IOB tagging rules. > > Here is an inference response containing other token labels: > > \`\`\` > { > "predicted\_value": "\[Birth defects\](ADR&Birth+defects) associated with \[thalidomide\](DRUG&thalidomide).", > "entities": \[ > { > "entity": "birth defects", > "class\_name": "ADR", > "class\_probability": 0.9664951378636988, > "start\_pos": 0, > "end\_pos": 13 > }, > { > "entity": "thalidomide", > "class\_name": "DRUG", > "class\_probability": 0.7323781805751934, > "start\_pos": 30, > "end\_pos": 41 > } > \] > } > \`\`\`