OT: Formal description documentation


(plaflamme) #1

Hi,

I've forked the es.org site to start the formal description of the Query DSL
mentioned here[1].

I'm trying to make blocks similar to the normal code blocks but with the
ability to have links inside. The normal code blocks use a

, so it's a
no-go if I want links. I tried to make a
with appropriate style and
classes, but the Textile renderer is eating it, so I'm not sure what the
best approach would be here.

Can anyone with experience with the site/textile/jekyll etc. help me out?

My test documentation is here[2] (scroll all the way down).

Thanks,
Philippe
[1]
http://groups.google.com/a/elasticsearch.com/group/users/msg/efeecc7b64084672
[2]


(Shay Banon) #2

Mayeb notextile will help? (http://textile.thresholdstate.com/).
On Saturday, May 21, 2011 at 5:37 AM, Philippe Laflamme wrote:

Hi,

I've forked the es.org site to start the formal description of the Query DSL mentioned here[1].

I'm trying to make blocks similar to the normal code blocks but with the ability to have links inside. The normal code blocks use a

, so it's a no-go if I want links. I tried to make a 
with appropriate style and classes, but the Textile renderer is eating it, so I'm not sure what the best approach would be here.

Can anyone with experience with the site/textile/jekyll etc. help me out?

My test documentation is here[2] (scroll all the way down).

Thanks,
Philippe
[1] http://groups.google.com/a/elasticsearch.com/group/users/msg/efeecc7b64084672
[2] https://github.com/plaflamme/elasticsearch.github.com/blob/master/guide/reference/api/search/facets/index.textile


(Nicolas Lalevée) #3

Le 21 mai 2011 à 04:37, Philippe Laflamme a écrit :

Hi,

I've forked the es.org site to start the formal description of the Query DSL mentioned here[1].

I'm trying to make blocks similar to the normal code blocks but with the ability to have links inside. The normal code blocks use a

, so it's a no-go if I want links. I tried to make a 
with appropriate style and classes, but the Textile renderer is eating it, so I'm not sure what the best approach would be here.

Can anyone with experience with the site/textile/jekyll etc. help me out?

My test documentation is here[2] (scroll all the way down).

If I may, if a formal documentation should exists, I think it should not be overloaded with the json syntax. I amy be probably better to write the abstract grammar.

And if I can suggest a presentation, I think the Ivy documentation about the format of an ivy.xml is quite friendly (since I'm an Ivy committer, I might be subjective here :wink: ).
See the tree of the left of the doc [1]. Each node of the tree of the table of content of the doc corresponds to an xml node in the ivy.xml file. Then on each page, some html tables describes each possible subnode, with links to the page of the subnode. This makes easy to know where a particular node should be placed in the final document, and easy to find the documentation of a node.

Nicolas

[1] http://ant.apache.org/ivy/history/latest-milestone/ivyfile.html


(Clinton Gormley) #4

Hiya

I've forked the es.org site to start the formal description of the
Query DSL mentioned here[1].

I used to maintain a semi-formal query dsl spec, but it is way out of
date now:

http://elasticsearch-users.115913.n3.nabble.com/Formal-spec-for-Query-DSL-tp217565p217565.html

clint


(plaflamme) #5

If I may, if a formal documentation should exists, I think it should not be
overloaded with the json syntax. I amy be probably better to write the
abstract grammar.

Yeah, I agree that not using any JSON would be better. But it's also
important that the result is clear and easy to transfer back to an actual
JSON syntax. We'll have to find a balance between both: not using any JSON
and easy transition from doc to JSON.

I thought of documenting the grammar first, but then I decided that working
with an actual use-case would make it easier to come up with the grammar.
Have any idea where you'd like to start?

And if I can suggest a presentation, I think the Ivy documentation about
the format of an ivy.xml is quite friendly (since I'm an Ivy committer, I
might be subjective here :wink: ).
See the tree of the left of the doc [1]. Each node of the tree of the table
of content of the doc corresponds to an xml node in the ivy.xml file. Then
on each page, some html tables describes each possible subnode, with links
to the page of the subnode. This makes easy to know where a particular node
should be placed in the final document, and easy to find the documentation
of a node.

That's pretty much what I had in mind. Making a tree structure for ES is
somewhat tricky since some node types can appear in multiple
places. Otherwise, each node would have its own page (which is pretty much
already the case) and each page would have the same structure. Something
along the lines of:

  1. Description
    Textual description of what the node does and why you'd use it
  2. Formal Description
    Pretty much what your Ivy example shows. A formal description of each node
    that states what is required, optional, etc. A table structure is great.
    Nodes must be linked together.
  3. Examples
    Lots of 'em.
  4. Notes
    I noticed that some of nodes have notes about caching and performance.
    This is where this stuff could be captured

Is the Ivy documentation generated from an XSD? Since it's XML and probably
already stated in the XSD, I would guess it be the case. I especially don't
think we should be writing an XSD though!

Thanks,
Philippe


(plaflamme) #6

That's great! This is exactly what I had in mind. We'll definitely want to
use this for starting a formal description.

Thanks,
Philippe

On Sat, May 21, 2011 at 14:01, Clinton Gormley clinton@iannounce.co.ukwrote:

Hiya

I've forked the es.org site to start the formal description of the
Query DSL mentioned here[1].

I used to maintain a semi-formal query dsl spec, but it is way out of
date now:

http://elasticsearch-users.115913.n3.nabble.com/Formal-spec-for-Query-DSL-tp217565p217565.html

clint


(plaflamme) #7

Nope. Now I tried applying style to the

like so:

p{white-space:pre}. content

And the result is a simple

without any style attribute. Could this be a
github-specific issue? Would jekyll render this differently maybe?

Philippe

On Sat, May 21, 2011 at 10:08, Shay Banon shay.banon@elasticsearch.comwrote:

Mayeb notextile will help? (http://textile.thresholdstate.com/).

On Saturday, May 21, 2011 at 5:37 AM, Philippe Laflamme wrote:

Hi,

I've forked the es.org site to start the formal description of the Query
DSL mentioned here[1].

I'm trying to make blocks similar to the normal code blocks but with the
ability to have links inside. The normal code blocks use a

, so it's a
no-go if I want links. I tried to make a
with appropriate style and
classes, but the Textile renderer is eating it, so I'm not sure what the
best approach would be here.

Can anyone with experience with the site/textile/jekyll etc. help me out?

My test documentation is here[2] (scroll all the way down).

Thanks,
Philippe
[1]
http://groups.google.com/a/elasticsearch.com/group/users/msg/efeecc7b64084672
[2]
https://github.com/plaflamme/elasticsearch.github.com/blob/master/guide/reference/api/search/facets/index.textile


(Clinton Gormley) #8

I thought of documenting the grammar first, but then I decided that
working with an actual use-case would make it easier to come up with
the grammar. Have any idea where you'd like to start?

I'm working on an alternate more compact syntax which I plan to
implement in the Perl API - hopefully it will gain wider acceptance.

While doing this, I've noticed that the various filter and query types
can be subdivided into various logical groups, eg:

  • full text queries (text*, field, query_string)
  • term based queries (term, terms, prefix, wildcard, fuzzy, range,
    span*)
  • meta-queries (fuzzy_like_this, more_like_this, top|has_children)
  • logical combinators (bool, and|or|not, dis_max, boosting)
  • custom: custom_score
  • etc

My intention is to write some "navigation" docs to explain the
difference between term and text clauses, queries and filters, and which
query/filter to use for which purpose.

Of course, if somebody gets there before me... :slight_smile:

clint


(Nicolas Lalevée) #9

Le 21 mai 2011 à 20:30, Philippe Laflamme a écrit :

Yeah, I agree that not using any JSON would be better. But it's also important that the result is clear and easy to transfer back to an actual JSON syntax. We'll have to find a balance between both: not using any JSON and easy transition from doc to JSON.

To my mind, if there are examples, the formal description doesn't need to precisely describe the JSON syntax. My user experience of the Ivy documentation (and also Ant's one) is about reading the formal spec to understand what it is about, and then for the syntax I always go see the examples. This may be just me.

That's pretty much what I had in mind. Making a tree structure for ES is somewhat tricky since some node types can appear in multiple places. Otherwise, each node would have its own page (which is pretty much already the case) and each page would have the same structure. Something along the lines of:

  1. Description
    Textual description of what the node does and why you'd use it
  2. Formal Description
    Pretty much what your Ivy example shows. A formal description of each node that states what is required, optional, etc. A table structure is great. Nodes must be linked together.
  3. Examples
    Lots of 'em.
  4. Notes
    I noticed that some of nodes have notes about caching and performance. This is where this stuff could be captured

Seems nice !

Is the Ivy documentation generated from an XSD? Since it's XML and probably already stated in the XSD, I would guess it be the case. I especially don't think we should be writing an XSD though!

It's not generated, no. I know xsddoc [1] which can manage xsd related documentation. But the result is not that sexy and easy to understand. And I cannot imagine maintaing the entire doc of Ivy in some xsd:comment of Ivy's xsd. Currently the doc is not hard to maintain. I have to admit that sometimes some piece of the doc is forgotten to be updated, there's still probably some holes. I don't think this is that critical, and I have seen people easily provide patch for the doc.

Nicolas

[1] http://xframe.sourceforge.net/xsddoc/index.html


(plaflamme) #10

Hi,

Made some improvements to the formal descriptions after some very helpful
comments.

I've reformatted using a table structure, much cleaner (thanks Nicolas for
that suggestion). The "facets" node comes out kinda weird because of its
dynamic nature[1]. I did the "terms" facet node which comes out pretty
nicely[2], I think.

Would something like this be a welcomed addition to the documentation? I'm
not sure about the structure : whether this should go at the top, stay at
the bottom or go somewhere else completely. Any comments/suggestions
welcomed.

Thanks,
Philippe
[1]


[2]

On Fri, May 20, 2011 at 22:37, Philippe Laflamme <
philippe.laflamme@obiba.org> wrote:

Hi,

I've forked the es.org site to start the formal description of the Query
DSL mentioned here[1].

I'm trying to make blocks similar to the normal code blocks but with the
ability to have links inside. The normal code blocks use a

, so it's a
no-go if I want links. I tried to make a
with appropriate style and
classes, but the Textile renderer is eating it, so I'm not sure what the
best approach would be here.

Can anyone with experience with the site/textile/jekyll etc. help me out?

My test documentation is here[2] (scroll all the way down).

Thanks,
Philippe
[1]
http://groups.google.com/a/elasticsearch.com/group/users/msg/efeecc7b64084672
[2]
https://github.com/plaflamme/elasticsearch.github.com/blob/master/guide/reference/api/search/facets/index.textile


(system) #11