### Author:  [@Yannick\_Smits\_2](https://discuss.elastic.co/u/Yannick_Smits_2)
#### Post date: [May 20, 2011, 9:02pm UTC](https://discuss.elastic.co/t/getting-started/4436/21 "2011-05-20T21:02:24Z")
Yes thats a great idea. I've been digging into Search the last week for the
first time and found that getting all the pieces and options together is the
most challenging part. A query DSL format would be very helpful!
From: Philippe Laflamme [[mailto:philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)]
Sent: vrijdag 20 mei 2011 22:42
To: [users@elasticsearch.com](mailto:users@elasticsearch.com)
Subject: Re: Getting Started
What I find is missing most in the docs is a top-down description of the
Query DSL: what is the possible structure of a search request body. It's all
in the docs, but split into many little pieces which makes the "vertical"
view (as Shay puts it) difficult to grasp. This is even complicated further
by the fact that some pieces can go in multiple places (a query within a
filter for example).
What would be very useful is a formal description of the Query DSL (similar
to what an XSD offers to XML). No, not that complicated, but it would be
complete (covers all possibilities) and would show what is required and what
is optional for each "node" type. And, while I'm dreaming, it would also be
clickable so you could drill-down.
The facet documentation has a formal description (uses and
such) but is incomplete (each example only shows part of what is available):
[http://www.elasticsearch.org/guide/reference/api/search/facets/](http://www.elasticsearch.org/guide/reference/api/search/facets/)
I'm not too familiar with the code that parses the queries, but could such a
document be generated from the code? Maybe using annotations or
introspection or whatever, that would make it easily maintainable...
I'd like to contribute if that makes any sense to anyone.
Philippe
On Fri, May 20, 2011 at 16:21, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
wrote:
What we can do is start a section in the site with samples. I prefer not to
have several places of docs around for elasticsearch. And the site is,
effectively, a wiki (anybody can edit and send pull requests).
On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
I completely agree with that π
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.
I started a wiki here:
[http://elastic.wikispaces.com](http://elastic.wikispaces.com)
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [May 20, 2011, 9:03pm UTC](https://discuss.elastic.co/t/getting-started/4436/22 "2011-05-20T21:03:10Z")
2011/5/20 Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
> Not going to stand in the way of a wiki, but I am certainly not going to
> spend any time there. There is enough things to do already. I hope there
> won't be one since it will create confusion for users as wikis tend to
> quickly get out of date, and its not maintained in the way that the guide
> is.
>
> If someone can't figure out how to run the github site, then its fine by me
> that he won't contribute docs.
Hmmh, that's an ignorant statement IMO. I don't think that you can
1. judge about people who have (version) problems with ruby stuff or their
ruby OS integration like me. Or are my explanations on the mailing list
really that stupid that you don't want me to contribute?
and 2. most documentation in many software products are NOT written by the
developers who assume too many thinks a newbie cannot know.
---
### Author:  [@plaflamme](https://discuss.elastic.co/u/plaflamme)
#### Post date: [May 20, 2011, 9:03pm UTC](https://discuss.elastic.co/t/getting-started/4436/23 "2011-05-20T21:03:44Z")
Here's what I'd really love to have:
>
It's probably invalid, but anyway, it gets the idea across...
[https://gist.github.com/983779](https://gist.github.com/983779)I tried to use the RFC-way for formal
descriptions. This syntax is used to describe HTTP headers, network
protocols and such, so it needs to be complete and being an RFC, it's very
terse. It would need to be adapted to the fact that it's JSON (the colons
and the brackets are conflicting). Now that I think about it, JSON has a
formal description on their site. Maybe it could use that syntax (re-using
the JSON terms even): [www.json.org](http://www.json.org)
Philippe
On Fri, May 20, 2011 at 16:55, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> There is no way to generate it automatically, but contributions are
> warmly welcomed.
>
> On Friday, May 20, 2011 at 11:42 PM, Philippe Laflamme wrote:
>
> What I find is missing most in the docs is a top-down description of the
> Query DSL: what is the possible structure of a search request body. It's all
> in the docs, but split into many little pieces which makes the "vertical"
> view (as Shay puts it) difficult to grasp. This is even complicated further
> by the fact that some pieces can go in multiple places (a query within a
> filter for example).
>
> What would be very useful is a formal description of the Query DSL (similar
> to what an XSD offers to XML). No, not that complicated, but it would be
> complete (covers all possibilities) and would show what is required and what
> is optional for each "node" type. And, while I'm dreaming, it would also be
> clickable so you could drill-down.
>
> The facet documentation has a formal description (uses and
> such) but is incomplete (each example only shows part of what is
> available):
> [Elasticsearch Platform β Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/)
>
> I'm not too familiar with the code that parses the queries, but could such
> a document be generated from the code? Maybe using annotations or
> introspection or whatever, that would make it easily maintainable...
>
> I'd like to contribute if that makes any sense to anyone.
>
> Philippe
>
> On Fri, May 20, 2011 at 16:21, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
>
> What we can do is start a section in the site with samples. I prefer not to
> have several places of docs around for elasticsearch. And the site is,
> effectively, a wiki (anybody can edit and send pull requests).
>
> On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
>
> I completely agree with that π
>
> but for the sake of just working examples (sometimes user needs a
> lot!) a wiki would be very nice - even if sprinkled here and there.
>
> I started a wiki here:
>
> [http://elastic.wikispaces.com](http://elastic.wikispaces.com)
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [May 20, 2011, 9:21pm UTC](https://discuss.elastic.co/t/getting-started/4436/24 "2011-05-20T21:21:43Z")
Lol, after complaining I tried to install it again with success π
But my system previously didn't use rvm which is a great simplifier for at
least my crappy (?) system ...
2011/5/20 Stefan MΓΌller [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com)
> 2011/5/20 Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
>
> > Not going to stand in the way of a wiki, but I am certainly not going to
> > spend any time there. There is enough things to do already. I hope there
> > won't be one since it will create confusion for users as wikis tend to
> > quickly get out of date, and its not maintained in the way that the guide
> > is.
> >
> > If someone can't figure out how to run the github site, then its fine by
> > me that he won't contribute docs.
>
> Hmmh, that's an ignorant statement IMO. I don't think that you can
>
> 1. judge about people who have (version) problems with ruby stuff or their
> ruby OS integration like me. Or are my explanations on the mailing list
> really that stupid that you don't want me to contribute?
>
> and 2. most documentation in many software products are NOT written by the
> developers who assume too many thinks a newbie cannot know.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [May 20, 2011, 9:27pm UTC](https://discuss.elastic.co/t/getting-started/4436/25 "2011-05-20T21:27:37Z")
On Saturday, May 21, 2011 at 12:03 AM, Stefan MΓΒΌller wrote:
> 2011/5/20 Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
>
> > Not going to stand in the way of a wiki, but I am certainly not going to spend any time there. There is enough things to do already. I hope there won't be one since it will create confusion for users as wikis tend to quickly get out of date, and its not maintained in the way that the guide is.
> >
> > If someone can't figure out how to run the github site, then its fine by me that he won't contribute docs.
> > Hmmh, that's an ignorant statement IMO. I don't think that you can
>
> 1. judge about people who have (version) problems with ruby stuff or their ruby OS integration like me. Or are my explanations on the mailing list really that stupid that you don't want me to contribute?
> Small changes don't require you to know how to run the site, since its just textile files. Github even allow you to edit in place and send pull requests all within the browser.
Bigger changes do require you to know how to do that, and yes, if you are going to become a major contributor to the site / guide, then you will have to find out how to run it. And if you can't figure out how to run the site, and thats whats stopping you from contributing docs, then I am fine that you won't contribute. No judging.
Have no idea what you are talking about in the mailing list.
> and 2. most documentation in many software products are NOT written by the developers who assume too many thinks a newbie cannot know.
> I strongly disagree here. I think the best documented open source projects out there are the ones where the people who document are also great developers. It might not be me, so here's to hoping we will have more of those in the community.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [May 20, 2011, 9:35pm UTC](https://discuss.elastic.co/t/getting-started/4436/26 "2011-05-20T21:35:09Z")
My gut feeling thinks that this will confuse people even more.
On Saturday, May 21, 2011 at 12:03 AM, Philippe Laflamme wrote:
> Here's what I'd really love to have:
>
> [An example formal description of facets in ES Β· GitHub](https://gist.github.com/983779)
>
> It's probably invalid, but anyway, it gets the idea across...
>
> I tried to use the RFC-way for formal descriptions. This syntax is used to describe HTTP headers, network protocols and such, so it needs to be complete and being an RFC, it's very terse. It would need to be adapted to the fact that it's JSON (the colons and the brackets are conflicting). Now that I think about it, JSON has a formal description on their site. Maybe it could use that syntax (re-using the JSON terms even): [www.json.org](http://www.json.org)
>
> Philippe
>
> On Fri, May 20, 2011 at 16:55, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
>
> > There is no way to generate it automatically, but contributions are warmly welcomed.
> > On Friday, May 20, 2011 at 11:42 PM, Philippe Laflamme wrote:
> >
> > > What I find is missing most in the docs is a top-down description of the Query DSL: what is the possible structure of a search request body. It's all in the docs, but split into many little pieces which makes the "vertical" view (as Shay puts it) difficult to grasp. This is even complicated further by the fact that some pieces can go in multiple places (a query within a filter for example).
> > >
> > > What would be very useful is a formal description of the Query DSL (similar to what an XSD offers to XML). No, not that complicated, but it would be complete (covers all possibilities) and would show what is required and what is optional for each "node" type. And, while I'm dreaming, it would also be clickable so you could drill-down.
> > >
> > > The facet documentation has a formal description (uses and such) but is incomplete (each example only shows part of what is available): [Elasticsearch Platform β Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/)
> > >
> > > I'm not too familiar with the code that parses the queries, but could such a document be generated from the code? Maybe using annotations or introspection or whatever, that would make it easily maintainable...
> > >
> > > I'd like to contribute if that makes any sense to anyone.
> > >
> > > Philippe
> > >
> > > On Fri, May 20, 2011 at 16:21, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> > >
> > > > What we can do is start a section in the site with samples. I prefer not to have several places of docs around for elasticsearch. And the site is, effectively, a wiki (anybody can edit and send pull requests).
> > > > On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
> > > >
> > > > > I completely agree with that π
> > > > >
> > > > > but for the sake of just working examples (sometimes user needs a
> > > > > lot!) a wiki would be very nice - even if sprinkled here and there.
> > > > >
> > > > > I started a wiki here:
> > > > >
> > > > > [http://elastic.wikispaces.com](http://elastic.wikispaces.com)
---
### Author:  [@plaflamme](https://discuss.elastic.co/u/plaflamme)
#### Post date: [May 21, 2011, 12:32am UTC](https://discuss.elastic.co/t/getting-started/4436/27 "2011-05-21T00:32:13Z")
I'm not sure how a formal description can be confusing since it leaves no
place for any guesswork or interpretation. Would you accept pull requests
for contributions towards making such a formal description?
Cheers,
Philippe
On Fri, May 20, 2011 at 17:35, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> My gut feeling thinks that this will confuse people even more.
>
> On Saturday, May 21, 2011 at 12:03 AM, Philippe Laflamme wrote:
>
> Here's what I'd really love to have:
>
> [An example formal description of facets in ES Β· GitHub](https://gist.github.com/983779)
>
> It's probably invalid, but anyway, it gets the idea across...
>
> [https://gist.github.com/983779](https://gist.github.com/983779)I tried to use the RFC-way for formal
> descriptions. This syntax is used to describe HTTP headers, network
> protocols and such, so it needs to be complete and being an RFC, it's very
> terse. It would need to be adapted to the fact that it's JSON (the colons
> and the brackets are conflicting). Now that I think about it, JSON has a
> formal description on their site. Maybe it could use that syntax (re-using
> the JSON terms even): [www.json.org](http://www.json.org)
>
> Philippe
>
> On Fri, May 20, 2011 at 16:55, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
>
> There is no way to generate it automatically, but contributions are
> warmly welcomed.
>
> On Friday, May 20, 2011 at 11:42 PM, Philippe Laflamme wrote:
>
> What I find is missing most in the docs is a top-down description of the
> Query DSL: what is the possible structure of a search request body. It's all
> in the docs, but split into many little pieces which makes the "vertical"
> view (as Shay puts it) difficult to grasp. This is even complicated further
> by the fact that some pieces can go in multiple places (a query within a
> filter for example).
>
> What would be very useful is a formal description of the Query DSL (similar
> to what an XSD offers to XML). No, not that complicated, but it would be
> complete (covers all possibilities) and would show what is required and what
> is optional for each "node" type. And, while I'm dreaming, it would also be
> clickable so you could drill-down.
>
> The facet documentation has a formal description (uses and
> such) but is incomplete (each example only shows part of what is
> available):
> [Elasticsearch Platform β Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/)
>
> I'm not too familiar with the code that parses the queries, but could such
> a document be generated from the code? Maybe using annotations or
> introspection or whatever, that would make it easily maintainable...
>
> I'd like to contribute if that makes any sense to anyone.
>
> Philippe
>
> On Fri, May 20, 2011 at 16:21, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
>
> What we can do is start a section in the site with samples. I prefer not to
> have several places of docs around for elasticsearch. And the site is,
> effectively, a wiki (anybody can edit and send pull requests).
>
> On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
>
> I completely agree with that π
>
> but for the sake of just working examples (sometimes user needs a
> lot!) a wiki would be very nice - even if sprinkled here and there.
>
> I started a wiki here:
>
> [http://elastic.wikispaces.com](http://elastic.wikispaces.com)
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [May 21, 2011, 12:33am UTC](https://discuss.elastic.co/t/getting-started/4436/28 "2011-05-21T00:33:22Z")
Sure, no problem. I think ti will be confusing since not everybody know how to read it.
On Saturday, May 21, 2011 at 3:32 AM, Philippe Laflamme wrote:
> I'm not sure how a formal description can be confusing since it leaves no place for any guesswork or interpretation. Would you accept pull requests for contributions towards making such a formal description?
>
> Cheers,
> Philippe
>
> On Fri, May 20, 2011 at 17:35, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
>
> > My gut feeling thinks that this will confuse people even more.
> > On Saturday, May 21, 2011 at 12:03 AM, Philippe Laflamme wrote:
> >
> > > Here's what I'd really love to have:
> > >
> > > [An example formal description of facets in ES Β· GitHub](https://gist.github.com/983779)
> > >
> > > It's probably invalid, but anyway, it gets the idea across...
> > >
> > > I tried to use the RFC-way for formal descriptions. This syntax is used to describe HTTP headers, network protocols and such, so it needs to be complete and being an RFC, it's very terse. It would need to be adapted to the fact that it's JSON (the colons and the brackets are conflicting). Now that I think about it, JSON has a formal description on their site. Maybe it could use that syntax (re-using the JSON terms even): [www.json.org](http://www.json.org)
> > >
> > > Philippe
> > >
> > > On Fri, May 20, 2011 at 16:55, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> > >
> > > > There is no way to generate it automatically, but contributions are warmly welcomed.
> > > > On Friday, May 20, 2011 at 11:42 PM, Philippe Laflamme wrote:
> > > >
> > > > > What I find is missing most in the docs is a top-down description of the Query DSL: what is the possible structure of a search request body. It's all in the docs, but split into many little pieces which makes the "vertical" view (as Shay puts it) difficult to grasp. This is even complicated further by the fact that some pieces can go in multiple places (a query within a filter for example).
> > > > >
> > > > > What would be very useful is a formal description of the Query DSL (similar to what an XSD offers to XML). No, not that complicated, but it would be complete (covers all possibilities) and would show what is required and what is optional for each "node" type. And, while I'm dreaming, it would also be clickable so you could drill-down.
> > > > >
> > > > > The facet documentation has a formal description (uses and such) but is incomplete (each example only shows part of what is available): [Elasticsearch Platform β Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/facets/)
> > > > >
> > > > > I'm not too familiar with the code that parses the queries, but could such a document be generated from the code? Maybe using annotations or introspection or whatever, that would make it easily maintainable...
> > > > >
> > > > > I'd like to contribute if that makes any sense to anyone.
> > > > >
> > > > > Philippe
> > > > >
> > > > > On Fri, May 20, 2011 at 16:21, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> > > > >
> > > > > > What we can do is start a section in the site with samples. I prefer not to have several places of docs around for elasticsearch. And the site is, effectively, a wiki (anybody can edit and send pull requests).
> > > > > > On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
> > > > > >
> > > > > > > I completely agree with that π
> > > > > > >
> > > > > > > but for the sake of just working examples (sometimes user needs a
> > > > > > > lot!) a wiki would be very nice - even if sprinkled here and there.
> > > > > > >
> > > > > > > I started a wiki here:
> > > > > > >
> > > > > > > [http://elastic.wikispaces.com](http://elastic.wikispaces.com)
---
### Author:  [@venkatnehatha](https://discuss.elastic.co/u/venkatnehatha)
#### Post date: [June 7, 2011, 6:18pm UTC](https://discuss.elastic.co/t/getting-started/4436/29 "2011-06-07T18:18:15Z")
Hi Clinton,
Thank you for your post. It is very useful. Can I user crawler.pl as a generic crawler for any url?? I'm not a perl guy, but the code seems to crawling based on
and links, which is fine in case of "[http://www.elasticsearch.org/blog/](http://www.elasticsearch.org/blog/)".
If that is true, Can you provide a generic crawler code plz...
Thanks in advace.
-Nehatha
---
### Author:  [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)
#### Post date: [June 9, 2011, 12:21pm UTC](https://discuss.elastic.co/t/getting-started/4436/30 "2011-06-09T12:21:51Z")
Hi Nehatha
> Thank you for your post. It is very useful. Can I user crawler.pl as a
> generic crawler for any url?? I'm not a perl guy, but the code seems to
> crawling based on
>
> and links, which is fine in case of
> "[Elasticsearch Platform β Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/)".
That particular crawler uses xpath to express the content that should be
extracted from each page.
> If that is true, Can you provide a generic crawler code plz...
I'm afraid that is left as an exercise for the reader π
clint
---
### Author:  [@venkatnehatha](https://discuss.elastic.co/u/venkatnehatha)
#### Post date: [June 9, 2011, 5:47pm UTC](https://discuss.elastic.co/t/getting-started/4436/31 "2011-06-09T17:47:18Z")
Thank you for the reply Clinton.
Can you suggest best php/java crawler api's, best suited to feed ES.
Thanks,
Nehatha
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:04am UTC](https://discuss.elastic.co/t/getting-started/4436/32 "2017-07-06T04:04:15Z")
[Previous page](https://discuss.elastic.co/t/getting-started/4436.md?page=1)