Hocon support in Elasticsearch?

Elastic Stack Elasticsearch
#1

Since Elasticsearch already uses jackson for parsing, it would probably be kind of easy to plugin hocon using the jackson plugin for this. I learned about hocon last week when somebody from Solr was pointing out they were supporting that in Solr 6. This would make a lot of sense in Elasticsearch as well.

I recently played with this on my own pet project jsonj and there's not much to it to integrate it. One line of code and It just kind of works. Json is fully supported so there are no backwards compatibility issues.

Has this been done already and/or would it be easy to support in a plugin? I'd be interested in this for several reasons:

  1. more forgiving syntax and a bit more compact syntax
  2. easy to add comments so people can document their complex queries with inline comments
  3. Hocon's value substitution would make a lot of sense for more complex ES queries and templating those.
#2

You can use comments already in Elasticsearch JSON dialect:

{
    /* this query matches all documents */
    "query" : {
        "match_all" : {
            
        }
    }
}

Note that Elasticsearch does not only use JSON for formulating queries or writing configurations, it is also the internal lingua franca used by the Jackson library which translates to Lucene field names and value structure. HOCON is a superset of JSON and does not align with such a procedure e.g. when processing duplicate keys.

Try this

PUT /test/test/1
{
    "a" : {"b":"c"},
    "a" : {"d":"e"}
}

Elasticsearch will create a document {"a":{"d":"e"}} but HOCON would create {"a":[{"b","c"},{"d":"e"}]}

While Elasticsearch JSON is valid Javascript, which fits nicely into HTTP and the Web, I find HOCON rather a burden when other languages must process it. You would have to convert back and forth with lots of boilerplate code like

val config = ConfigFactory.load(); 
val configJSON : String = 
  config.root().render( ConfigRenderOptions.concise() )

Also HOCON comes from Scala, and the Scala multiline string format used in HOCON would break e.g. Elasticsearch bulk JSON-Lines format.

1 Like
#3