Incorrect/incomplete examples in es documentation

I've been struggling to get familiar with the elastic search API for the
past few months due to some issues with the examples in the documentation
that I'm sure others must be experiencing as well.

  1. a lot of the json examples are incomplete fragments and essential things
    are missing.
  2. some of the examples don't parse as valid json due to e.g. missing commas
  3. the documentation is fragmented over many pages, no single page has the
    full story. This makes using some of the more exotic stuff quite hard when
    from an API point of view it would actually be quite straight forward.

A good example is geo-shape-filter.html. The json misses a few commas and
doesn't parse. Once you fix that, it still doesn't work because filtered is
not a valid top level field in a query. Filtered belongs in a query object.
This is not clear from either this page or the documentation for the
filtered element, which makes the same mistake and contains an example that
doesn't work either. I figured it out eventually from this post:
http://elasticsearch-users.115913.n3.nabble.com/No-parser-for-element-filtered-td3277649.html.

I appreciate all the hard work people are doing on elastic search and its
documentation and am wondering how to improve things for everybody. Fixing
the documentation is probably a lot of work at this point and in my view it
would be fighting the symptoms rather than the underlying problem. Good API
reference documentation is a hard problem.

Has anyone thought about using simple unit testable examples and including
fragments from that working code into the documentation? This should be
doable and it would make it fairly obvious when the documentation "breaks"
relative to the code (or vice versa).

If we can get some consensus on an approach, I'd be more than happy to
contribute some of my time here.

--

Responses inline.

On Mon, Dec 31, 2012 at 7:02 AM, Jilles van Gurp jillesvangurp@gmail.comwrote:

...

Once you fix that, it still doesn't work because filtered is not a valid
top level field in a query. Filtered belongs in a query object.

This format holds true for every query in the documention. They all need to
be wrapped in a query object or be part of a larger query (BooleanQuery).
Once you realize this behavior, the documentation is easier to understand,
but it could be more explicit.

I appreciate all the hard work people are doing on Elasticsearch and its
documentation and am wondering how to improve things for everybody. Fixing
the documentation is probably a lot of work at this point and in my view it
would be fighting the symptoms rather than the underlying problem. Good API
reference documentation is a hard problem.

The documentation is open-sourced at
https://github.com/elasticsearch/elasticsearch.github.com, so anyone can
contribute. I added some documentation for some filters that were not
documented and fixed some other small issues. Shay and company are good
about accepting pull requests for documentation. Unparsable JSON is
something that should be fixed.

Has anyone thought about using simple unit testable examples and including
fragments from that working code into the documentation? This should be
doable and it would make it fairly obvious when the documentation "breaks"
relative to the code (or vice versa).

There are lots of unit tests, but they are all against the Java API, which
is what the REST endpoints use behind the scenes. I find the unit tests to
be better documentation for many of the features, but then again, I only
use the Java API.

--
Ivan

--

On Monday, December 31, 2012 5:54:01 PM UTC+1, Ivan Brusic wrote:

Responses inline.

On Mon, Dec 31, 2012 at 7:02 AM, Jilles van Gurp <jilles...@gmail.com<javascript:>

wrote:

...

Once you fix that, it still doesn't work because filtered is not a valid
top level field in a query. Filtered belongs in a query object.

This format holds true for every query in the documention. They all need
to be wrapped in a query object or be part of a larger query
(BooleanQuery). Once you realize this behavior, the documentation is easier
to understand, but it could be more explicit.

That's what appears to be the case indeed. But since it is essentially not
documented/ambiguous, you are left to trial and error and hunting around
different pages for scraps of useful fragments that may or may not work.
Since, I've been spending quite a bit of time on this, I wanted to flag
this as an issue. The difference between a working, complete fragment and a
non working, partial fragment is only a few lines of json. So, why not just
make these examples as explicit as possible? Elastic search seems designed
to make this really easy: you can get most stuff working using a handful of
curl commands. It's easier to adapt actually working examples then to copy
paste together several half working examples.

I appreciate all the hard work people are doing on Elasticsearch and its
documentation and am wondering how to improve things for everybody. Fixing
the documentation is probably a lot of work at this point and in my view it
would be fighting the symptoms rather than the underlying problem. Good API
reference documentation is a hard problem.

The documentation is open-sourced at
https://github.com/elasticsearch/elasticsearch.github.com, so anyone can
contribute. I added some documentation for some filters that were not
documented and fixed some other small issues. Shay and company are good
about accepting pull requests for documentation. Unparsable JSON is
something that should be fixed.

Yes, I forked it and took a look at it. As I said, fixing the documentation
is going to be a lot of work. I listed some symptoms here but I suspect the
actual problems are all over the place. Rather than just patching up the
problem, I'd like to actually do something more meaningful with it and fix
things properly.

Before I barge in and start fixing & restructuring things in a major way
(which I think is needed here), I think it is important to agree on an
overal strategy for how to document stuff. Another issue is that I've only
partially succeeded in making the feature I was complaining about work for
me so I don't feel comfortable fixing the documentation for that feature
yet. I posted about that issue separately. It looks to me that either I
completely missed the point on what it is supposed to do (which would be a
major documentation issue) or it is in fact broken in the current
implementation.

Has anyone thought about using simple unit testable examples and
including fragments from that working code into the documentation? This
should be doable and it would make it fairly obvious when the documentation
"breaks" relative to the code (or vice versa).

There are lots of unit tests, but they are all against the Java API, which
is what the REST endpoints use behind the scenes. I find the unit tests to
be better documentation for many of the features, but then again, I only
use the Java API.

I've over the years of using REST and SOAP services developed the opinion
that using client libraries are kind of an anti pattern. Usually they just
serve to cover up bad API design (cough Solr cough) or cripple them by
not exposing them properly. Well designed REST api's ,like the elastic
search API (and in fact this is one of its key selling points IMHO),
basically make this unnecessary.

--
Ivan

--
Jilles

--