# Default language for language detection

**URL:** <https://discuss.elastic.co/t/default-language-for-language-detection/292470>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [December 20, 2021, 2:20pm UTC](https://discuss.elastic.co/t/default-language-for-language-detection/292470 "2021-12-20T14:20:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![uu99ix](https://avatars.discourse-cdn.com/v4/letter/u/ecccb3/32.png) [@uu99ix](https://discuss.elastic.co/u/uu99ix)\
**Post date:** [December 20, 2021, 2:20pm UTC](https://discuss.elastic.co/t/default-language-for-language-detection/292470/1 "2021-12-20T14:20:51Z")

</div>

I don't know if this is the right category to post under, sorry if it's not.

I followed this guide: [Language identification | Machine Learning in the Elastic Stack [7.16] | Elastic](https://www.elastic.co/guide/en/machine-learning/current/ml-lang-ident.html) and it seems to return Japanese ("ja") for empty strings and numbers. Is there a default setting or something? I'd like it to return "en" in these cases.

---

<div class="post-metadata">

**Author:** ![droberts195](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/droberts195/32/17692_2.png) [@droberts195](https://discuss.elastic.co/u/droberts195)\
**Post date:** [December 20, 2021, 3:01pm UTC](https://discuss.elastic.co/t/default-language-for-language-detection/292470/2 "2021-12-20T15:01:23Z")

</div>

Thanks for pointing out that deficiency.

I agree it's an issue and have opened [[ML] What to do about lang\_ident for empty strings and numbers? · Issue #81933 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/81933) to discuss what to do.

In the short term you could add an extra [`set` processor](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-processor.html) after the lang\_ident `inference` processor that changes the predicted language field to `en` if the source field is an empty string. This `set` processor would use an `if` so that it only overrides the prediction for empty strings.

---

<div class="post-metadata">

**Author:** ![uu99ix](https://avatars.discourse-cdn.com/v4/letter/u/ecccb3/32.png) [@uu99ix](https://discuss.elastic.co/u/uu99ix)\
**Post date:** [December 21, 2021, 1:01am UTC](https://discuss.elastic.co/t/default-language-for-language-detection/292470/3 "2021-12-21T01:01:37Z")

</div>

Nice that you may add it as a feature. I'm not sure if this is the best way or whether it's idiomatic, but I managed to accomplish it with this code as per David's instructions:

```auto
{
  "pipeline": {
    "processors": [
      {
        "inference": {
          "model_id": "lang_ident_model_1",
          "inference_config": {
            "classification": {
              "num_top_classes": 5
            }
          }
        }
      },
      {
        "set": {
          "if": "def t = ctx.text.trim(); t == \"\" || /^(\\d+(\\.\\d+)?)$/.matcher(t).matches()",
          "field": "ml.inference.predicted_value",
          "value": "en"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "text": "123.456"
      }
    }
  ]
}

```

It keeps the predicted\_probability which is a bit unclean but I don't think it's important enough to add just for aesthetics 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 18, 2022, 1:01am UTC](https://discuss.elastic.co/t/default-language-for-language-detection/292470/4 "2022-01-18T01:01:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.