I was going through the elasticsearch code and docs (couchdb river)
and had some doubts. (I am still in the process of setting up ES
0.11).
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
CouchDB filter args - CouchDB supports arguments to filters [3],
which can be useful in filtering out specific data. From the code, it
looks like we can add this as part of 'filter' configuration
"filter" = "abc&option1=value1"
In the code for CouchDB river, it is just
if (couchFilter != null) {
file = file + "&filter=" + couchFilter;
}
So, the HTTP url will get formed as '....&filter=abc&option1=value1',
but it looks like a hack.
Would it be better to provide a different configuration for filter
parameters or is the above hack a safe way to do things ?
I guess I found an answer to one of my own questions.
On Wed, Sep 29, 2010 at 11:52 AM, Mahendra M wrote:
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
From trying it out, I think the interpretation for the index settings
is as follows. (Please correct me if I am wrong)
While creating a couchdb river, if "index" is not specified, then the
default values are
index = <db_name>
type = <db_name>
So, searching for data has to be done against : http://localhost:9200/<db_name>/<db_name>/_search
I was (wrongly) assuming that the index will be created as
index = _river
type = <db_name>
Yea, the _river index acts as the registry for rivers and their metadata
and state (providing persistency and state management for the rivers, not
the data they index). For example, the coucdb one stores the last indexed
seq in it. You can control the index name and type you index to.
Regarding the filter, I misses the part where you can provide parameters
to filters, I can add it as a configuration option. Want to open an issue
for it?
I guess I found an answer to one of my own questions.
On Wed, Sep 29, 2010 at 11:52 AM, Mahendra M wrote:
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
From trying it out, I think the interpretation for the index settings
is as follows. (Please correct me if I am wrong)
While creating a couchdb river, if "index" is not specified, then the
default values are
index = <db_name>
type = <db_name>
So, searching for data has to be done against : http://localhost:9200/<db_name>/<db_name>/_search
I was (wrongly) assuming that the index will be created as
index = _river
type = <db_name>
Just out of curiosity, how it is possible to control the index and type name
for RabbitMQ river?
Also I found that it is possible to customize the river registry name by
settings river.index_name property (defaults to "_river"). Probably should
be added to the documentation.
Yea, the _river index acts as the registry for rivers and their metadata
and state (providing persistency and state management for the rivers, not
the data they index). For example, the coucdb one stores the last indexed
seq in it. You can control the index name and type you index to.
Regarding the filter, I misses the part where you can provide parameters
to filters, I can add it as a configuration option. Want to open an issue
for it?
I guess I found an answer to one of my own questions.
On Wed, Sep 29, 2010 at 11:52 AM, Mahendra M wrote:
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
From trying it out, I think the interpretation for the index settings
is as follows. (Please correct me if I am wrong)
While creating a couchdb river, if "index" is not specified, then the
default values are
index = <db_name>
type = <db_name>
So, searching for data has to be done against : http://localhost:9200/<db_name>/<db_name>/_search
I was (wrongly) assuming that the index will be created as
index = _river
type = <db_name>
Just out of curiosity, how it is possible to control the index and type
name for RabbitMQ river?
Also I found that it is possible to customize the river registry name by
settings river.index_name property (defaults to "_river"). Probably should
be added to the documentation.
Yea, the _river index acts as the registry for rivers and their
metadata and state (providing persistency and state management for the
rivers, not the data they index). For example, the coucdb one stores the
last indexed seq in it. You can control the index name and type you index
to.
Regarding the filter, I misses the part where you can provide
parameters to filters, I can add it as a configuration option. Want to open
an issue for it?
I guess I found an answer to one of my own questions.
On Wed, Sep 29, 2010 at 11:52 AM, Mahendra M wrote:
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
From trying it out, I think the interpretation for the index settings
is as follows. (Please correct me if I am wrong)
While creating a couchdb river, if "index" is not specified, then the
default values are
index = <db_name>
type = <db_name>
So, searching for data has to be done against : http://localhost:9200/<db_name>/<db_name>/_search
I was (wrongly) assuming that the index will be created as
index = _river
type = <db_name>
Just out of curiosity, how it is possible to control the index and type
name for RabbitMQ river?
Also I found that it is possible to customize the river registry name by
settings river.index_name property (defaults to "_river"). Probably should
be added to the documentation.
Yea, the _river index acts as the registry for rivers and their
metadata and state (providing persistency and state management for the
rivers, not the data they index). For example, the coucdb one stores the
last indexed seq in it. You can control the index name and type you index
to.
Regarding the filter, I misses the part where you can provide
parameters to filters, I can add it as a configuration option. Want to open
an issue for it?
I guess I found an answer to one of my own questions.
On Wed, Sep 29, 2010 at 11:52 AM, Mahendra M wrote:
The couchdb river plugin provides extra settings for "index" [1].
Does this provide support for creating a index other than "_river" ?
From the rivers documentation [2], it looks like the index name will
be _river and the user can control only the type. Or is it actually
possible to override _river with a different index name ?
From trying it out, I think the interpretation for the index settings
is as follows. (Please correct me if I am wrong)
While creating a couchdb river, if "index" is not specified, then the
default values are
index = <db_name>
type = <db_name>
So, searching for data has to be done against : http://localhost:9200/<db_name>/<db_name>/_search
I was (wrongly) assuming that the index will be created as
index = _river
type = <db_name>
On Wed, Sep 29, 2010 at 3:03 PM, Shay Banon wrote:
Yea, the _river index acts as the registry for rivers and their metadata
and state (providing persistency and state management for the rivers, not
the data they index). For example, the coucdb one stores the last indexed
seq in it. You can control the index name and type you index to.
OK cool. So, I guess if one has to upload a mapping for an index
(river), it must be on the actual index and not the _river index.
Regarding the filter, I misses the part where you can provide parameters
to filters, I can add it as a configuration option. Want to open an issue
for it?
OK. I will open an issue for this.
Also, couchdb supports user authentication. Some setups may require
authentication for registering a stream.
On Wed, Sep 29, 2010 at 3:03 PM, Shay Banon wrote:
Yea, the _river index acts as the registry for rivers and their
metadata
and state (providing persistency and state management for the rivers, not
the data they index). For example, the coucdb one stores the last indexed
seq in it. You can control the index name and type you index to.
OK cool. So, I guess if one has to upload a mapping for an index
(river), it must be on the actual index and not the _river index.
Yea, you can precreate the index and put mappings and then create the river
that indexes data from couchdb into it.
Regarding the filter, I misses the part where you can provide
parameters
to filters, I can add it as a configuration option. Want to open an issue
for it?
OK. I will open an issue for this.
Also, couchdb supports user authentication. Some setups may require
authentication for registering a stream.
On Wed, Sep 29, 2010 at 3:03 PM, Shay Banon wrote:
Yea, the _river index acts as the registry for rivers and their
metadata
and state (providing persistency and state management for the rivers,
not
the data they index). For example, the coucdb one stores the last
indexed
seq in it. You can control the index name and type you index to.
OK cool. So, I guess if one has to upload a mapping for an index
(river), it must be on the actual index and not the _river index.
Yea, you can precreate the index and put mappings and then create the river
that indexes data from couchdb into it.
Regarding the filter, I misses the part where you can provide
parameters
to filters, I can add it as a configuration option. Want to open an
issue
for it?
OK. I will open an issue for this.
Also, couchdb supports user authentication. Some setups may require
authentication for registering a stream.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.