# Is there any spaCy libraries for Elasticsearch available?

**URL:** <https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [February 21, 2020, 2:47am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284 "2020-02-21T02:47:04Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![richylyq](https://avatars.discourse-cdn.com/v4/letter/r/41988e/32.png) [@richylyq](https://discuss.elastic.co/u/richylyq)\
**Post date:** [February 21, 2020, 2:47am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/1 "2020-02-21T02:47:04Z")

</div>

Hi guys, wanted to check if there is any spaCy plugins or libraries for Elasticsearch available out there?

I found the openNLP Ingest processor but my professor wants me to find more on using spaCy...

Much appreciated! (:

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 21, 2020, 7:55am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/2 "2020-02-21T07:55:48Z")

</div>

There is no spacy support. There is however another language detection built into elasticsearch that has just been added in the recent 7.6.0 release.

See this sample

```auto
POST _ingest/pipeline/_simulate?filter_path=**.predicted_value
{
  "pipeline": {
    "processors": [
      {
        "inference": {
          "model_id": "lang_ident_model_1",
          "inference_config": { "classification": {}},
          "field_mappings": {}
        }
      }
    ]
  },
  "docs": [
    { "_source": { "text": "This is an english text" } },
    { "_source": { "text": "Das ist ein deutscher Text" } },
    { "_source": { "text": "你好世界" } },
    { "_source": { "text": "Ceci est un texte en français" } }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 21, 2020, 9:55am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/3 "2020-02-21T09:55:17Z")

</div>

Here's [a Python script](https://gist.github.com/markharwood/3df31ca1fa8246524b6b9326aa1c9af5) I use to download RSS headlines and perform entity extraction on the headlines using Spacy. It uses the annotated\_text format used by the [elasticsearch plugin](https://www.elastic.co/blog/search-for-things-not-strings-with-the-annotated-text-plugin) designed to search and highlight entities embedded in text.

---

<div class="post-metadata">

**Author:** ![richylyq](https://avatars.discourse-cdn.com/v4/letter/r/41988e/32.png) [@richylyq](https://discuss.elastic.co/u/richylyq)\
**Post date:** [February 24, 2020, 1:16am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/4 "2020-02-24T01:16:33Z")

</div>

@Mark_Harwood  
i will try to look through to see if i can use any subsets of the codes to make do with... May I ask questions about your code when in doubt? 👌🏻

What is the difference between type: annotated\_text and type: text as in seen in

> ` mapping = { "properties": { "id": {"type": "keyword"}, "url": {"type": "keyword"}, "headline": {"type": "annotated_text", "analyzer":"analyzer_shingle"}, "published": {"type": "date"}, "feedLink": {"type": "keyword"}, "tags":{"type":"keyword"} }`

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 24, 2020, 8:16am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/5 "2020-02-24T08:16:06Z")

</div>

Did you read the blog post I linked to?

---

<div class="post-metadata">

**Author:** ![richylyq](https://avatars.discourse-cdn.com/v4/letter/r/41988e/32.png) [@richylyq](https://discuss.elastic.co/u/richylyq)\
**Post date:** [February 24, 2020, 9:47am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/7 "2020-02-24T09:47:58Z")

</div>

AH HA... GOT IT!

By any chance... Are you going to integrate it into a plugin to be used in Elasticsearch? like the opennlp-ingest-processor?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 24, 2020, 9:59am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/8 "2020-02-24T09:59:32Z")

</div>

> [@richylyq](#):
>
> Are you going to integrate it into a plugin to be used in Elasticsearch? like the opennlp-ingest-processor?

No, Spacy is Python and elasticsearch core is Java-based so that's not an obvious integration for us to consider.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 23, 2020, 9:59am UTC](https://discuss.elastic.co/t/is-there-any-spacy-libraries-for-elasticsearch-available/220284/9 "2020-03-23T09:59:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.