Recommendation on reading the heart of the code

Elastic Stack Elasticsearch
1

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

In my experience, when trying to work on a large/new code base, fixing bugs
is a great way to start.

You can see how pieces of the code work together and ask questions of other
developers. They're excited to help you as you're fixing bugs and helping
improve the system.

On Sunday, October 5, 2014 8:44:05 AM UTC-7, ahmed jamal maaz wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a65b10b4-7b5c-43b7-955f-a06aef82c0c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

The code is difficult to debug due to the distributed nature of
Elasticsearch. Requests get serialized and are sent via a binary protocol,
so you cannot focus on specific classes. Dependency injection adds in
additional complexity. You cannot simply determine the construction time
values in the constructor by browsing the code.

My advice if you want to learn the code is by starting with the unit tests.
Focus on the smaller pieces and work your way up.

Cheers,

Ivan
On Oct 5, 2014 8:44 AM, "ahmed jamal maaz" jamalahmedmaaz@gmail.com wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCJJSispLonWUNS1zjLVFmJ5UNKNpN8f1Td3G93EYDs5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

And regardless of these issues, and yes, tracing is hard, you can always
set it in a debugger and trace specific components.

and more and more software is being constructed via components with
dependency injection. So starting wth the components and seeing what they
do and how they are composed would help out.

On Sunday, October 5, 2014 8:01:12 PM UTC-7, Ivan Brusic wrote:

The code is difficult to debug due to the distributed nature of
Elasticsearch. Requests get serialized and are sent via a binary protocol,
so you cannot focus on specific classes. Dependency injection adds in
additional complexity. You cannot simply determine the construction time
values in the constructor by browsing the code.

My advice if you want to learn the code is by starting with the unit
tests. Focus on the smaller pieces and work your way up.

Cheers,

Ivan
On Oct 5, 2014 8:44 AM, "ahmed jamal maaz" <jamalah...@gmail.com
<javascript:>> wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa6dc4b9-4a23-4216-b153-6d06ba5bb4d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

When it comes to understanding any large code base your approach will
always depend on your own preference. Things that I have found useful​ to
get started:

  • Instead of working on the core inner workings of the code start with
    checking out the project, building it and using it. Build small demo
    applications. Go from your personal use case of the project. Often you'll
    find pieces of the project that you would like to see a little different to
    be really useful for your case. As you go ahead and make those changes you
    will understand more and more of the project. There's two advantages to
    this approach: a) you know exactly what the outcome should be, b) the
    result matters to you personally and keeps you motivated.

  • You may find functionality that you would like to have that's just too
    much for a small and focussed patch - start with writing a plugin for that.

  • Find areas that are lacking in terms of documentation - developers tend
    to be really grateful if they don't have to write the documentation
    themselves and will be happy to answer your in depth questions.

  • As mentioned earlier in the thread in addition to fixing bugs reading -
    but also writing - tests is a great way to get started understanding the
    code base better as well. Make sure to choose an area that is interesting
    for you and start exploring from there.

IMHO the answer to how to understand the code base (and remain motivated to
learn more) isn't that much different from how to get started contributing
to any open source project: http://commons.apache.org/volunteering.html

Isabel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAFSgB-BXodXZMo8H8ndaQdLShbJF%3DA4J6xqgOMV62RzZgCSPnw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

A great start is by studying and writing plugins.

Elasticsearch is one of the rare masterpieces of software that allow to
plug in code that you have authored to extend functionality, without
forking the main code base.

There are a lot of plugins out there with small code base and easy to
study, e.g. on github.

A very nice side effect is that you can open source your plugins so you can
give something back to the community, and also have more eyes on your code
to find and fix bugs.

Like Ivan already said, if you want to dive deeper into the ES API, look at
the ES tests. The Java test code base is extensive and covers almost all
use cases. You will find usage examples of the API all over the place.

Jörg

On Sun, Oct 5, 2014 at 5:44 PM, ahmed jamal maaz jamalahmedmaaz@gmail.com
wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGiv7AWWk7nkOcyrPazxVz7ZkPSUcTC0khN0JJzW04dzA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Hi all,

These are very good advises. I really appreciate it.

I am planning to start all of these in parallel and find which one will
suite me the best.

Thank you: Joerg, Isabel, Kevin and Ivan.

I have a question: How do we vote an answer (like Quora) here (is this
available with Google groups).

Obliged
Jamal

On Mon, Oct 6, 2014 at 2:45 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

A great start is by studying and writing plugins.

Elasticsearch is one of the rare masterpieces of software that allow to
plug in code that you have authored to extend functionality, without
forking the main code base.

There are a lot of plugins out there with small code base and easy to
study, e.g. on github.

A very nice side effect is that you can open source your plugins so you
can give something back to the community, and also have more eyes on your
code to find and fix bugs.

Like Ivan already said, if you want to dive deeper into the ES API, look
at the ES tests. The Java test code base is extensive and covers almost all
use cases. You will find usage examples of the API all over the place.

Jörg

On Sun, Oct 5, 2014 at 5:44 PM, ahmed jamal maaz <jamalahmedmaaz@gmail.com

wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vVk99L_i1rs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGiv7AWWk7nkOcyrPazxVz7ZkPSUcTC0khN0JJzW04dzA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGiv7AWWk7nkOcyrPazxVz7ZkPSUcTC0khN0JJzW04dzA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2Ba4NmVvYWEamanOz-3%2B9ZwRNf8ADwBXN%2BHuf2tgHA-Ru%3D3OsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

8

There is no voting or other gamification, just a plain ol' mailing list.
Many of us respond as just another way to contribute to open-source.

--
Ivan

On Mon, Oct 6, 2014 at 3:37 AM, ahmed jamal maaz jamalahmedmaaz@gmail.com
wrote:

Hi all,

These are very good advises. I really appreciate it.

I am planning to start all of these in parallel and find which one will
suite me the best.

Thank you: Joerg, Isabel, Kevin and Ivan.

I have a question: How do we vote an answer (like Quora) here (is this
available with Google groups).

Obliged
Jamal

On Mon, Oct 6, 2014 at 2:45 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

A great start is by studying and writing plugins.

Elasticsearch is one of the rare masterpieces of software that allow to
plug in code that you have authored to extend functionality, without
forking the main code base.

There are a lot of plugins out there with small code base and easy to
study, e.g. on github.

A very nice side effect is that you can open source your plugins so you
can give something back to the community, and also have more eyes on your
code to find and fix bugs.

Like Ivan already said, if you want to dive deeper into the ES API, look
at the ES tests. The Java test code base is extensive and covers almost all
use cases. You will find usage examples of the API all over the place.

Jörg

On Sun, Oct 5, 2014 at 5:44 PM, ahmed jamal maaz <
jamalahmedmaaz@gmail.com> wrote:

Hi,

I am a newbie to elasticsearch and lucene. I am going through its source
code. I like the general idea of search on Lucene.

My Problem:
There is tons of code elasticsearch + lucene in the project. I am
requesting (curl url) and going through step by step (in debug mode). Are
there any recommendations and tips which can help me in increasing the pace
of this. This is taking too long. I don't have any issues, i will spend
time. But are there any parts which i should look first and then the
others. Such that i can learn complete and in a structured manner.

Current State:
I have touch the bottom of insertion (creating new index) and general
querying.

Thanks in advance.

Obliged
Jamal

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834b9e4d-5564-4471-9564-33b81d9f2935%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vVk99L_i1rs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGiv7AWWk7nkOcyrPazxVz7ZkPSUcTC0khN0JJzW04dzA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGiv7AWWk7nkOcyrPazxVz7ZkPSUcTC0khN0JJzW04dzA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2Ba4NmVvYWEamanOz-3%2B9ZwRNf8ADwBXN%2BHuf2tgHA-Ru%3D3OsQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2Ba4NmVvYWEamanOz-3%2B9ZwRNf8ADwBXN%2BHuf2tgHA-Ru%3D3OsQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBkNchnPU-TrMMX%3D4b5T7m%3Dck_3LMwR8GP48RiWhzhNRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.