# How we built Elasticsearch plugins for indexing ontologies

**URL:** <https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217>\
**Category:** Elasticsearch\
**Created:** [January 27, 2016, 10:59am UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217 "2016-01-27T10:59:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Charlie\_Hull](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@Charlie\_Hull](https://discuss.elastic.co/u/Charlie_Hull)\
**Post date:** [January 27, 2016, 10:59am UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/1 "2016-01-27T10:59:36Z")

</div>

Hello! My colleague Matt has just written about a recent project where he's been building ES plugins for ontology indexing. Hope you find it interesting and the plugins useful - and we welcome feedback, especially hints as to where we've gone wrong and/or better ways to do it.  
[http://www.flax.co.uk/blog/2016/01/27/fun-frustration-writing-plugin-elasticsearch-ontology-indexing/](http://www.flax.co.uk/blog/2016/01/27/fun-frustration-writing-plugin-elasticsearch-ontology-indexing/)

This is part of a publically-funded [project](http://www.flax.co.uk/blog/client/european-bioinformatics-institute/) here in the UK to develop better open source search software for bioinformaticians - it's called BioSolr for historical reasons but we're also using ES. We'll be talking about this at a [workshop event](http://www.ebi.ac.uk/pdbe/about/events/open-source-search-bioinformatics) near Cambridge next week.

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [January 27, 2016, 12:51pm UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/2 "2016-01-27T12:51:07Z")

</div>

> [@Charlie\_Hull](#):
>
> The fun and frustration of writing a plugin for Elasticsearch for ontology indexing

Nice job!

Implementing custom field mappers is definitely one of the most advanced way of all possible things a plugin can do (aside from custom aggregation methods).

Why not just set up a Lucene analyzer or a token filter for ontology matching? I did something similar when porting the lucene-skos project to ES

> **[GitHub - jprante/elasticsearch-analysis-skos: SKOS analysis for Elasticsearch](https://github.com/jprante/elasticsearch-analysis-skos/)**
>
> SKOS analysis for Elasticsearch. Contribute to jprante/elasticsearch-analysis-skos development by creating an account on GitHub.

I think the comparison to Solr's UpdateRequestProcessor is quite unfair. It's a different approach.

If it's all about lack of documentation, why not just describe the implementation process in a dedicated blog post ? Or open pull requests to add developer notes to the ES doc site? I bet the ES team is always happy about such contributions. Or submitting a new user story. I wish I could document more, too, but I'm very lousy at this job, and can't find enough time.

Personally I am convinced the (well-thought and elaborative) open source code of Elasticsearch is the best documentation possible 🙂 but that's not what is of great comfort for most beginners.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 27, 2016, 1:20pm UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/3 "2016-01-27T13:20:47Z")

</div>

> [@jprante](#):
>
> I bet the ES team is always happy about such contributions.

++ You won your bet Jörg! We are more than happy when users contribute code, doc, tests, stories, whatever... 🙂

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [January 28, 2016, 12:12am UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/4 "2016-01-28T00:12:28Z")

</div>

One of the reasons why I have never documented the Elasticsearch internals  
(for example the TransportAction families) is that it is a moving target  
for which you have no insight about the direction it is traveling in. Every  
time I dig deep in the code and want to change something via a plugin, I  
learn that such code might go away.

Cheers,

Ivan

---

<div class="post-metadata">

**Author:** ![Matt\_Pearce](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@Matt\_Pearce](https://discuss.elastic.co/u/Matt_Pearce)\
**Post date:** [January 28, 2016, 9:48am UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/5 "2016-01-28T09:48:10Z")

</div>

Thanks for the suggestions.

I'm not sure that writing an analyzer or token filter would help for our purposes. The object of the plugin is to retrieve ontology data and add it to the document, so it's not really a question of tokenising, unless I'm missing your point (entirely possible!).

It is a completely different approach to the UpdateRequestProcessor, which I did make clear in the article. There's no real equivalent in ElasticSearch though, so creating a new field mapping seemed like a good approach.

Regarding documentation, as mentioned below, it seems like I'd be documenting a moving target, which would potentially end up with something more misleading than digging through other people's plugins to see how they approached the problem. I might look at a more detailed blog post in the future, though.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:20pm UTC](https://discuss.elastic.co/t/how-we-built-elasticsearch-plugins-for-indexing-ontologies/40217/6 "2017-07-05T23:20:50Z")

</div>


