Are there any comprehensive documents which describe the detailed process of document indexing and searching?


(shihpeng) #1

Although I have read a lot of guides and articles, there seems still
something I missed in understanding the process of document indexing and
searching. I would like to know if there are any comprehensive documents
which describe the process very detailly?

For example, the document indexing process involves document preprocessing,
text analyzing, load into memory buffer (in the mean time also write this
operation into translog), write out as segment files, ..., etc. The
document indexing operation in Elasticsearch is actually quite complicated,
but the articles I can found on the Internet always only cover some of
these steps.

Are there any such detailed documents available?

Thanks in advance.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jason Wee) #2

If you have that basic knowledge, perhaps the next steps would be read into
the code, for instance, you mentioned about translog, so find for the
translog class file and start from there?

jason

On Fri, May 22, 2015 at 10:47 PM, Shih-Peng Lin shihpeng.lin@gmail.com
wrote:

Although I have read a lot of guides and articles, there seems still
something I missed in understanding the process of document indexing and
searching. I would like to know if there are any comprehensive documents
which describe the process very detailly?

For example, the document indexing process involves document
preprocessing, text analyzing, load into memory buffer (in the mean time
also write this operation into translog), write out as segment files, ...,
etc. The document indexing operation in Elasticsearch is actually quite
complicated, but the articles I can found on the Internet always only cover
some of these steps.

Are there any such detailed documents available?

Thanks in advance.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHO4itx7q%2BiKojEAasxZDwGDBR6o7WKJKCw29TmcZCqU36RKkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Nik Everett) #3

I get the sense that this is a good start though I haven't watched it
myself:

https://www.elastic.co/elasticon/2015/sf/elasticsearch-data-journey-life-of-a-document-in-elasticsearch
On May 25, 2015 5:45 AM, "Jason Wee" peichieh@gmail.com wrote:

If you have that basic knowledge, perhaps the next steps would be read
into the code, for instance, you mentioned about translog, so find for the
translog class file and start from there?

jason

On Fri, May 22, 2015 at 10:47 PM, Shih-Peng Lin shihpeng.lin@gmail.com
wrote:

Although I have read a lot of guides and articles, there seems still
something I missed in understanding the process of document indexing and
searching. I would like to know if there are any comprehensive documents
which describe the process very detailly?

For example, the document indexing process involves document
preprocessing, text analyzing, load into memory buffer (in the mean time
also write this operation into translog), write out as segment files, ...,
etc. The document indexing operation in Elasticsearch is actually quite
complicated, but the articles I can found on the Internet always only cover
some of these steps.

Are there any such detailed documents available?

Thanks in advance.

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHO4itx7q%2BiKojEAasxZDwGDBR6o7WKJKCw29TmcZCqU36RKkA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHO4itx7q%2BiKojEAasxZDwGDBR6o7WKJKCw29TmcZCqU36RKkA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3t0AzG6wzS_wJsRxRjG_D8TdhrRmUz7AApGPD_YAcAPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jason Wee) #4

that's good to know, will read the link :slight_smile:

On Mon, May 25, 2015 at 8:11 PM, Nikolas Everett nik9000@gmail.com wrote:

I get the sense that this is a good start though I haven't watched it
myself:

https://www.elastic.co/elasticon/2015/sf/elasticsearch-data-journey-life-of-a-document-in-elasticsearch
On May 25, 2015 5:45 AM, "Jason Wee" peichieh@gmail.com wrote:

If you have that basic knowledge, perhaps the next steps would be read
into the code, for instance, you mentioned about translog, so find for the
translog class file and start from there?

jason

On Fri, May 22, 2015 at 10:47 PM, Shih-Peng Lin shihpeng.lin@gmail.com
wrote:

Although I have read a lot of guides and articles, there seems still
something I missed in understanding the process of document indexing and
searching. I would like to know if there are any comprehensive documents
which describe the process very detailly?

For example, the document indexing process involves document
preprocessing, text analyzing, load into memory buffer (in the mean time
also write this operation into translog), write out as segment files, ...,
etc. The document indexing operation in Elasticsearch is actually quite
complicated, but the articles I can found on the Internet always only cover
some of these steps.

Are there any such detailed documents available?

Thanks in advance.

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4cf44fe9-e14b-4957-954e-8782c5b3c2cc%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to
https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHO4itx7q%2BiKojEAasxZDwGDBR6o7WKJKCw29TmcZCqU36RKkA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHO4itx7q%2BiKojEAasxZDwGDBR6o7WKJKCw29TmcZCqU36RKkA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3t0AzG6wzS_wJsRxRjG_D8TdhrRmUz7AApGPD_YAcAPQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3t0AzG6wzS_wJsRxRjG_D8TdhrRmUz7AApGPD_YAcAPQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We have moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHO4itxwb7oHsBR8s0bLVJJ1dXvCzUaLfvG815M9aomjAA%2BcvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5