I am really trying to use elastic search but I just can't seem to get
things to work consistently.
What I really need are examples. What is on the main website
(elasticsearch.org) is misc snippets that don't actually run on their
own. Does anyone know of a place I can go to look at actual working
examples? I'm thinking, creating an index (not a stupid toy twitter /
get by ID example), adding things to it, and then doing complicated
searches on it.
Thanks for any help.
Yes, docs needs to be improved in exactly this area.
My suggestion: go to freenode chat and ask if you can't get the things
done. Then document these things/queries and make a blog post 
If you are using Java API you can check out:
to see how to create indices.
On May 19, 11:35 am, Vectorjohn vectorj...@gmail.com wrote:
I am really trying to use Elasticsearch but I just can't seem to get
things to work consistently.What I really need are examples. What is on the main website
(elasticsearch.org) is misc snippets that don't actually run on their
own. Does anyone know of a place I can go to look at actual working
examples? I'm thinking, creating an index (not a stupid toy twitter /
get by ID example), adding things to it, and then doing complicated
searches on it.Thanks for any help.
BTW: ES-HEAD is the way to query from browser:
What exactly is missing in the getting started? Does the readme: GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine not enough? Or the intro video: Elasticsearch Platform — Find real-time answers at scale | Elastic.
On Thursday, May 19, 2011 at 12:35 PM, Vectorjohn wrote:
I am really trying to use Elasticsearch but I just can't seem to get
things to work consistently.What I really need are examples. What is on the main website
(elasticsearch.org) is misc snippets that don't actually run on their
own. Does anyone know of a place I can go to look at actual working
examples? I'm thinking, creating an index (not a stupid toy twitter /
get by ID example), adding things to it, and then doing complicated
searches on it.Thanks for any help.
From my perspective also as a new user it drops you off with a running ES
server. All fine and dandy but it leaves out the part about how to integrate
a crawler, or feed content other than via running curl PUTs (which it
specifically states is not the way to do in production), and severely lacks
any configuration guide (just some hints on a few curl commands or that it
can use jquery... I would rather have the ability to know what Im doing than
just see wow, jquery! posted everywhere in the docs). I have gone and
started using OpenSearchServer (its gui is intuitive enough to figure out
most of what it can and cant do, even with a similar lack of docs) instead
until I can either gather enough info to get ES working with a crawler and
front-end (ed-head) or it gets updated and I can follow a more complete
guide. Even better would be a ES-Nutch how-to. I have seen enough discussion
about getting the projects working together, and the upcoming 3.1 nutch
release might have more for that, but any sort of "this is where it looks
for plugins or crawlers or other integration stuff" would be most helpful.
Docs right now seem more centered for devs to use to imbed it in existing
code rather than as a pure stand-alone (with minimal crawler and frontend)
search engine.
-T
On Thu, May 19, 2011 at 6:17 AM, Shay Banon shay.banon@elasticsearch.comwrote:
What exactly is missing in the getting started? Does the readme:
GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine not enough? Or the intro
video: Elasticsearch Platform — Find real-time answers at scale | Elastic
.On Thursday, May 19, 2011 at 12:35 PM, Vectorjohn wrote:
I am really trying to use Elasticsearch but I just can't seem to get
things to work consistently.What I really need are examples. What is on the main website
(elasticsearch.org) is misc snippets that don't actually run on their
own. Does anyone know of a place I can go to look at actual working
examples? I'm thinking, creating an index (not a stupid toy twitter /
get by ID example), adding things to it, and then doing complicated
searches on it.Thanks for any help.
s/jquery/json/
too many j* stuff in there
-T
On Thu, May 19, 2011 at 4:27 PM, Theral Mackey tmackey@zetta.net wrote:
From my perspective also as a new user it drops you off with a running ES
server. All fine and dandy but it leaves out the part about how to integrate
a crawler, or feed content other than via running curl PUTs (which it
specifically states is not the way to do in production), and severely lacks
any configuration guide (just some hints on a few curl commands or that it
can use jquery... I would rather have the ability to know what Im doing than
just see wow, jquery! posted everywhere in the docs). I have gone and
started using OpenSearchServer (its gui is intuitive enough to figure out
most of what it can and cant do, even with a similar lack of docs) instead
until I can either gather enough info to get ES working with a crawler and
front-end (ed-head) or it gets updated and I can follow a more complete
guide. Even better would be a ES-Nutch how-to. I have seen enough discussion
about getting the projects working together, and the upcoming 3.1 nutch
release might have more for that, but any sort of "this is where it looks
for plugins or crawlers or other integration stuff" would be most helpful.
Docs right now seem more centered for devs to use to imbed it in existing
code rather than as a pure stand-alone (with minimal crawler and frontend)
search engine.-T
On Thu, May 19, 2011 at 6:17 AM, Shay Banon shay.banon@elasticsearch.comwrote:
What exactly is missing in the getting started? Does the readme:
GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine not enough? Or the intro
video:
Elasticsearch Platform — Find real-time answers at scale | Elastic.On Thursday, May 19, 2011 at 12:35 PM, Vectorjohn wrote:
I am really trying to use Elasticsearch but I just can't seem to get
things to work consistently.What I really need are examples. What is on the main website
(elasticsearch.org) is misc snippets that don't actually run on their
own. Does anyone know of a place I can go to look at actual working
examples? I'm thinking, creating an index (not a stupid toy twitter /
get by ID example), adding things to it, and then doing complicated
searches on it.Thanks for any help.
Hi Theral
Yes, we need more ES tutorials, and "introduction-to" articles.
On Thu, 2011-05-19 at 16:27 -0700, Theral Mackey wrote:
From my perspective also as a new user it drops you off with a running
ES server. All fine and dandy but it leaves out the part about how to
integrate a crawler, or feed content other than via running curl PUTs
(which it specifically states is not the way to do in production),
This kinda depends what language you want to use. There are several
APIs: most used are Java, Perl, Ruby, PHP, Python, and you may want to
try out elasticsearch-head for a simple Javascript console for making
requests: GitHub - mobz/elasticsearch-head: A web front end for an elastic search cluster
This is why the examples are explained using curl - it's the "universal
language" that all of the APIs (except Java) translate their requests
into.
You may want to read through some of the blog entries on
Elasticsearch Platform — Find real-time answers at scale | Elastic - it will give you some taste for how
to use ES.
Also, here are a few examples in different languages:
and severely lacks any configuration guide (just some hints on a few
curl commands or that it can use jquery...
The point about Elasticsearch is that you shouldn't configure anything.
At least, not until you actually have to. ES has got very good
defaults, and generally just works out of the box. When you have a
better idea of your encounter a problem, that's the time to see what you
can configure.
I would rather have the ability to know what Im doing than just see
wow, jquery! posted everywhere in the docs). I have gone and started
using OpenSearchServer (its gui is intuitive enough to figure out most
of what it can and cant do, even with a similar lack of docs) instead
until I can either gather enough info to get ES working with a crawler
and front-end (ed-head) or it gets updated and I can follow a more
complete guide. Even better would be a ES-Nutch how-to. I have seen
enough discussion about getting the projects working together, and the
upcoming 3.1 nutch release might have more for that, but any sort of
"this is where it looks for plugins or crawlers or other integration
stuff" would be most helpful. Docs right now seem more centered for
devs to use to imbed it in existing code rather than as a pure
stand-alone (with minimal crawler and frontend) search engine.
Yes - it is early days and kimchy has been focused on developing a
powerful, easy-to-integrate search server which developers from any
language can use in their applications, in a way that makes sense for
their app.
It is now up to us, the community, to develop reusable "pluggable"
applications and write about specific use cases.
It sounds like you are looking for a prepackaged search service, rather
than a highly customisable search engine.
Perhaps have a look at Karmi's field notes which explain how he setup ES
search for the elasticsearch.org website:
Your use case is one of many, and so far nobody has written that
particular app, although the easy part would be implementing search with
ES.
PS See my next email for an example in Perl
clint
Hi Theral
PS See my next email for an example in Perl
I've put together a quick example of how you could crawl and search a
website in Perl.
crawl: Crawl ElasticSearch blogposts · GitHub
search: Search ElasticSearch blogposts · GitHub
To get this example running, you need to:
wget http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.16.1.tar.gz
tar -xzf elasticsearch-0.16.1.tar.gz
cd elasticsearch-0.16.1
./bin/elasticsearch
Easiest way to install Perl modules is using 'cpanm':
curl -L http://cpanmin.us | perl - --sudo App::cpanminus
Then:
cpanm Elasticsearch
cpanm Web::Scraper
curl http://gist.github.com/gists/982552/download > search.pl
curl http://gist.github.com/gists/982551/download > crawl.pl
perl crawl.pl
perl search.pl
hth
clint
@Shay: I think Vectorjohn refers to some of the queries which cannot
run as they are printed. Most of the time not the full query (only the
important part) is listed in the docs. The problem for beginners (and
also sometimes for me ;)) is where they belongs to etc.
On May 19, 3:17 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
What exactly is missing in the getting started? Does the readme:https://github.com/elasticsearch/elasticsearchnot enough? Or the intro video:Elasticsearch Platform — Find real-time answers at scale | Elastic.
I kinda agree with that. Not just for queries but for config too. Perhaps we
can have an examples section where user-contributed examples can be placed
and linked to from the relevant sections? I'm happy to collate/contribute to
these.
On Fri, May 20, 2011 at 8:41 PM, Karussell tableyourtime@googlemail.comwrote:
@Shay: I think Vectorjohn refers to some of the queries which cannot
run as they are printed. Most of the time not the full query (only the
important part) is listed in the docs. The problem for beginners (and
also sometimes for me ;)) is where they belongs to etc.On May 19, 3:17 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
What exactly is missing in the getting started? Does the readme:
https://github.com/elasticsearch/elasticsearchnot enough? Or the intro
video:Elasticsearch Platform — Find real-time answers at scale | Elastic.
Paul Loy
paul@keteracel.com
http://uk.linkedin.com/in/paulloy
The thing that I am concerned about is that this will quickly run out of control and sensible order.
The current state of docs has some logic in it. What is missing are "vertical" docs. For example, Analysis, what does it mean, how does it work, whats the meaning of it when mapping, whats the meaning of it when querying. This will include samples and so on.
Same can be applied to things like searching.
That vertical concepts area is something that I certainly plan to doc. Its a lot of work, and would love more help there. But, it has to be done with structure in mind, and not sprinkled here and there with samples. This will quickly get out of hand.
-shay.banon
On Friday, May 20, 2011 at 10:45 PM, Paul Loy wrote:
I kinda agree with that. Not just for queries but for config too. Perhaps we can have an examples section where user-contributed examples can be placed and linked to from the relevant sections? I'm happy to collate/contribute to these.
On Fri, May 20, 2011 at 8:41 PM, Karussell tableyourtime@googlemail.com wrote:
@Shay: I think Vectorjohn refers to some of the queries which cannot
run as they are printed. Most of the time not the full query (only the
important part) is listed in the docs. The problem for beginners (and
also sometimes for me ;)) is where they belongs to etc.On May 19, 3:17 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
What exactly is missing in the getting started? Does the readme:https://github.com/elasticsearch/elasticsearchnot enough? Or the intro video:Elasticsearch Platform — Find real-time answers at scale | Elastic.
--
Paul Loy
paul@keteracel.com
http://uk.linkedin.com/in/paulloy
I completely agree with that 
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.
I started a wiki here:
Aeh ... @shay: please let me know if I shouldn't do that. Don't want
to fork docs or something ...
On May 20, 10:19 pm, Karussell tableyourt...@googlemail.com wrote:
I completely agree with that
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.I started a wiki here:
What we can do is start a section in the site with samples. I prefer not to have several places of docs around for elasticsearch. And the site is, effectively, a wiki (anybody can edit and send pull requests).
On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
I completely agree with that
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.I started a wiki here:
I prefer not to have several places of docs around for elasticsearch.
ok.
And the site is, effectively, a wiki (anybody can edit and send pull requests).
Yes and no, the problem is (like I told you some months ago). That it
is not easy as a wiki: you need to get the doc server up and running +
git etc.
Probably not something were users will fix minor typos or contribute
minor queries etc.
That being to said, the solr wiki works pretty well IMO.
What do you think?
What I find is missing most in the docs is a top-down description of the
Query DSL: what is the possible structure of a search request body. It's all
in the docs, but split into many little pieces which makes the "vertical"
view (as Shay puts it) difficult to grasp. This is even complicated further
by the fact that some pieces can go in multiple places (a query within a
filter for example).
What would be very useful is a formal description of the Query DSL (similar
to what an XSD offers to XML). No, not that complicated, but it would be
complete (covers all possibilities) and would show what is required and what
is optional for each "node" type. And, while I'm dreaming, it would also be
clickable so you could drill-down.
The facet documentation has a formal description (uses and
such) but is incomplete (each example only shows part of what is
available): Elasticsearch Platform — Find real-time answers at scale | Elastic
I'm not too familiar with the code that parses the queries, but could such a
document be generated from the code? Maybe using annotations or
introspection or whatever, that would make it easily maintainable...
I'd like to contribute if that makes any sense to anyone.
Philippe
On Fri, May 20, 2011 at 16:21, Shay Banon shay.banon@elasticsearch.comwrote:
What we can do is start a section in the site with samples. I prefer not to
have several places of docs around for elasticsearch. And the site is,
effectively, a wiki (anybody can edit and send pull requests).On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:
I completely agree with that
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.I started a wiki here:
Would it be an option to point to the wiki of ElasticSearch for the
'sprinkled here and there' examples section?
Not going to stand in the way of a wiki, but I am certainly not going to spend any time there. There is enough things to do already. I hope there won't be one since it will create confusion for users as wikis tend to quickly get out of date, and its not maintained in the way that the guide is.
If someone can't figure out how to run the github site, then its fine by me that he won't contribute docs.
-shay.banon
On Friday, May 20, 2011 at 11:41 PM, Karussell wrote:
I prefer not to have several places of docs around for elasticsearch.
ok.
And the site is, effectively, a wiki (anybody can edit and send pull requests).
Yes and no, the problem is (like I told you some months ago). That it
is not easy as a wiki: you need to get the doc server up and running +
git etc.Probably not something were users will fix minor typos or contribute
minor queries etc.That being to said, the solr wiki works pretty well IMO.
What do you think?
No.
On Friday, May 20, 2011 at 11:43 PM, Karussell wrote:
Would it be an option to point to the wiki of Elasticsearch for the
'sprinkled here and there' examples section?
There is no way to generate it automatically, but contributions are warmly welcomed.
On Friday, May 20, 2011 at 11:42 PM, Philippe Laflamme wrote:
What I find is missing most in the docs is a top-down description of the Query DSL: what is the possible structure of a search request body. It's all in the docs, but split into many little pieces which makes the "vertical" view (as Shay puts it) difficult to grasp. This is even complicated further by the fact that some pieces can go in multiple places (a query within a filter for example).
What would be very useful is a formal description of the Query DSL (similar to what an XSD offers to XML). No, not that complicated, but it would be complete (covers all possibilities) and would show what is required and what is optional for each "node" type. And, while I'm dreaming, it would also be clickable so you could drill-down.
The facet documentation has a formal description (uses and such) but is incomplete (each example only shows part of what is available): Elasticsearch Platform — Find real-time answers at scale | Elastic
I'm not too familiar with the code that parses the queries, but could such a document be generated from the code? Maybe using annotations or introspection or whatever, that would make it easily maintainable...
I'd like to contribute if that makes any sense to anyone.
Philippe
On Fri, May 20, 2011 at 16:21, Shay Banon shay.banon@elasticsearch.com wrote:
What we can do is start a section in the site with samples. I prefer not to have several places of docs around for elasticsearch. And the site is, effectively, a wiki (anybody can edit and send pull requests).
On Friday, May 20, 2011 at 11:19 PM, Karussell wrote:I completely agree with that
but for the sake of just working examples (sometimes user needs a
lot!) a wiki would be very nice - even if sprinkled here and there.I started a wiki here:
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.