# Indexing different types from a CouchDB database

**URL:** <https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729>\
**Category:** Elasticsearch\
**Created:** [August 13, 2012, 9:38am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729 "2012-08-13T09:38:24Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 13, 2012, 9:38am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/1 "2012-08-13T09:38:24Z")

</div>

Hi,

Reading [http://www.elasticsearch.org/blog/2010/02/12/yourdatayoursearch.html](http://www.elasticsearch.org/blog/2010/02/12/yourdatayoursearch.html),  
I realized my problem is exactly the same as the article except for one  
huge difference: Instead of indexing one book or cd at a time, I already  
have a bunch of data in my couchdb database. I'm adding elastic search  
capabilities to my project now and I'm having a ton of problems finding out  
how to index my couchDB database by type.

The documentation says that one way to index ALL documents is like the  
following:

curl -XPUT 'localhost:9200/\_river/amazon/\_meta' -d '{  
"type" : "couchdb",  
"couchdb" : {  
"host" : "localhost",  
"port" : 5984,  
"db" : "amazon",  
"filter" : null  
},  
"index" : {  
"index" : "amazon",  
"type" : "book",  
"bulk\_size" : "100",  
"bulk\_timeout" : "10ms"  
}  
}'

Here I specified the index being amazon and book being the type. But this still indexes ALL the documents in CouchDB! _How do I make it ONLY index the book type documents?_ Is this what the filter is for? Or can this be accomplished with script field: ctx.type = "book" \<-- Tried this but this didn't work either.

Thanks in advance!

--

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 13, 2012, 10:01am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/2 "2012-08-13T10:01:28Z")

</div>

I think that you can use ctx.ignore=true  
To ignore some documents.

Read here [GitHub - elastic/elasticsearch-river-couchdb: CouchDB River Plugin for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-river-couchdb)

BTW, I wrote a support for CouchDb views but it's not merged.

> <https://github.com/elastic/elasticsearch-river-couchdb/pull/2>
>
> Moving from https://github.com/elasticsearch/elasticsearch/issues/1512
> 
> Real imp…lementation of pull request https://github.com/elasticsearch/elasticsearch/issues/1258, as there was a problem with GitHub.
> 
> In CouchDB, you can retrieve docs by \`GET\`, \`\_changes\` API and views.
> CouchDB river uses \`\_changes\` API to get documents.
> 
> I would like to be able to get documents that changed (getting ID with the \_changes API) using a view with parameter \`key="DOCID"\`.
> 
> As views return a collection of results (aka rows), we will index in ES each row with an id like DOCID\_seq where seq is the sequence number of each row.
> If you get back 3 rows for one single change for document with ID=1234, the river will index 3 documents :
> \- 1234\_1
> \- 1234\_2
> \- 1234\_3
> 
> To use it, you have to define a view in couchDB. For instance, \`\_design/vues/\_view/test\_dpi\` with 
> 
> \`\`\` javascript
> function(doc) {
> listArt=doc.document.articles;
> 
> for(var i=0; i\<listArt.length;i++) {  
> var artJson = {};
> artJson = { 'docid' : doc.\_id, 'num' : listArt\[i\].numeroArticle };
> artJson = JSON.stringify( artJson );
> emit(doc.\_id, eval('('+artJson+')') );
> };
> }
> \`\`\`
> 
> You can use it in your couchDb river as follow :
> 
> \`\`\` javascript
> {
> "type":"couchdb",
> "couchdb": {
> "host":"localhost",
> "port":"5984",
> "db":"dau\_test",
> "view":"vues/\_view/test\_dpi",
> "viewIgnoreRemove":false
> }
> }
> \`\`\`
> 
> New options :
> \- \`view\` : if not null, couchDB river will not fetch content from \`\_changes\` API but only IDs and then will use the view to retrieve rows using the ID as a key. By default : null
> \- \`viewIgnoreRemove\` : ask the river to ignore removal of rows if there is less rows after a document update. By default : false so non existing rows will be removed from elastic search.
> 
> For example, with the 3 rows described earlier, if you push a new version of the document 1234 in couchDB with only 2 docs, 
> 
> If \`viewIgnoreRemove\` is false (default), then
> \- 1234\_1 will be updated
> \- 1234\_2 will be updated
> \- 1234\_3 will be removed
> 
> If \`viewIgnoreRemove\` is true, then
> \- 1234\_1 will be updated
> \- 1234\_2 will be updated
> \- 1234\_3 will not be updated
> 
> BTW, I will push an update to the pull request when the \`ids\_prefix\` filter will be available to make code more efficient. (See issue https://github.com/elasticsearch/elasticsearch/issues/1259).
> 
> Thanks
> David

David

--

Le 13 août 2012 à 11:38, Mark Huang [zhenghao12@gmail.com](mailto:zhenghao12@gmail.com) a écrit :

> Hi,
> 
> Reading [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/2010/02/12/yourdatayoursearch.html), I realized my problem is exactly the same as the article except for one huge difference: Instead of indexing one book or cd at a time, I already have a bunch of data in my couchdb database. I'm adding Elasticsearch capabilities to my project now and I'm having a ton of problems finding out how to index my couchDB database by type.
> 
> The documentation says that one way to index ALL documents is like the following:
> 
> ## curl -XPUT 'localhost:9200/\_river/amazon/\_meta' -d '{ "type" : "couchdb", "couchdb" : { "host" : "localhost", "port" : 5984, "db" : "amazon", "filter" : null }, "index" : { "index" : "amazon", "type" : "book", "bulk\_size" : "100", "bulk\_timeout" : "10ms" } }' Here I specified the index being amazon and book being the type. But this still indexes ALL the documents in CouchDB! How do I make it ONLY index the book type documents? Is this what the filter is for? Or can this be accomplished with script field: ctx.type = "book" \<-- Tried this but this didn't work either. Thanks in advance!

--

---

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 13, 2012, 10:16am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/3 "2012-08-13T10:16:13Z")

</div>

Are you saying it should be like this:

curl -XPUT 'localhost:9200/\_river/amazon/\_meta' -d '{  
"type" : "couchdb",  
"couchdb" : {  
"host" : "localhost",  
"port" : 5984,  
"db" : "amazon",  
"filter" : null,  
"script": "_ctx.ignore = true;_ ctx.doc.name = ctx.doc.content.name  
;"  
},  
"index" : {  
"index" : "amazon",  
"type" : "book",  
"bulk\_size" : "100",  
"bulk\_timeout" : "10ms"  
}  
}'

The documentation says : Also, if _ctx.ignore_ is set to true, the change  
seq will be ignore and not applied.  
Wouldn't this mean that the book document will not be applied? Can I add  
if else statements? if (ctx.type !== 'book') ctx.ignore = True; ?

On Mon, Aug 13, 2012 at 5:01 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote:

> I think that you can use ctx\*.ignore=true \*  
> To ignore some documents.
> 
> Read here [GitHub - elastic/elasticsearch-river-couchdb: CouchDB River Plugin for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-river-couchdb)
> 
> BTW, I wrote a support for CouchDb views but it's not merged.  
> [Add support for views by dadoonet · Pull Request #2 · elastic/elasticsearch-river-couchdb · GitHub](https://github.com/elasticsearch/elasticsearch-river-couchdb/pull/2)
> 
> David
> 
> --
> 
> Le 13 août 2012 à 11:38, Mark Huang [zhenghao12@gmail.com](mailto:zhenghao12@gmail.com) a écrit :
> 
> Hi,
> 
> Reading  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/2010/02/12/yourdatayoursearch.html), I  
> realized my problem is exactly the same as the article except for one huge  
> difference: Instead of indexing one book or cd at a time, I already have a  
> bunch of data in my couchdb database. I'm adding Elasticsearch  
> capabilities to my project now and I'm having a ton of problems finding out  
> how to index my couchDB database by type.
> 
> The documentation says that one way to index ALL documents is like the  
> following:
> 
> curl -XPUT 'localhost:9200/\_river/amazon/\_meta' -d '{  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "filter" : null  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book",  
> "bulk\_size" : "100",  
> "bulk\_timeout" : "10ms"  
> }  
> }'
> 
> Here I specified the index being amazon and book being the type. But this still indexes ALL the documents in CouchDB! _How do I make it ONLY index the book type documents?_ Is this what the filter is for? Or can this be accomplished with script field: ctx.type = "book" \<-- Tried this but this didn't work either.
> 
> Thanks in advance!
> 
> --
> 
> --

--

---

<div class="post-metadata">

**Author:** ![Dan\_Everton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_everton/32/1865_2.png) [@Dan\_Everton](https://discuss.elastic.co/u/Dan_Everton)\
**Post date:** [August 13, 2012, 11:44pm UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/4 "2012-08-13T23:44:51Z")

</div>

I'm assuming you have a CouchDB database that has multiple types of  
documents in it and you only want to index one of those types? You can do  
that in two ways:

1. A filter function on the CouchDB
2. A filter function on the Elasticsearch side.

The first way requires you to add a filter function to a design document in  
the CouchDB database. So in your design document you'll have something like:

{  
"\_id" : "\_design/books",  
"filters" : {  
"only" : "function(doc, req) { return doc.type == 'book' }"  
}  
}

And then your river definition looks like this:

{  
"type" : "couchdb",  
"couchdb" : {  
"host" : "localhost",  
"port" : 5984,  
"db" : "amazon",  
"filter" : "books/only"  
},  
"index" : {  
"index" : "amazon",  
"type" : "book"  
}  
}

The second way to do it is to use a script on the Elasticsearch side. This  
is essentially the same as the previous way, but puts the script execution  
burden on ES not CouchDB. So your river definition in this case looks like:

{  
"type" : "couchdb",  
"couchdb" : {  
"host" : "localhost",  
"port" : 5984,  
"db" : "amazon",  
"script" : "ctx.ignore = ctx.doc.type != 'book'"  
},  
"index" : {  
"index" : "amazon",  
"type" : "book"  
}  
}

Note that this river definition requires the Javascript plugin to be added  
to your Elasticsearch install.

Cheers,  
Dan

--

---

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 14, 2012, 3:36am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/5 "2012-08-14T03:36:06Z")

</div>

Thanks a million Dan! You have saved me from complete despair to full of  
hope!

Elasticsearch CouchDB river is great, but (no offense), I feel it lacks  
some proper usage examples. Maybe it's just me, but I doubt any n00b can  
understand what to do with ctx.ignore just by reading the documentation.

One _FINAL_ thing:

Is it possible to save the javascript "script" field value into a file and  
then reference it from the river definition?

On Monday, 13 August 2012 18:44:51 UTC-5, Dan Everton wrote:

> I'm assuming you have a CouchDB database that has multiple types of  
> documents in it and you only want to index one of those types? You can do  
> that in two ways:
> 
> 1. A filter function on the CouchDB
> 2. A filter function on the Elasticsearch side.
> 
> The first way requires you to add a filter function to a design document  
> in the CouchDB database. So in your design document you'll have something  
> like:
> 
> {  
> "\_id" : "\_design/books",  
> "filters" : {  
> "only" : "function(doc, req) { return doc.type == 'book' }"  
> }  
> }
> 
> And then your river definition looks like this:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "filter" : "books/only"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> The second way to do it is to use a script on the Elasticsearch side. This  
> is essentially the same as the previous way, but puts the script execution  
> burden on ES not CouchDB. So your river definition in this case looks like:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "script" : "ctx.ignore = ctx.doc.type != 'book'"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> Note that this river definition requires the Javascript plugin to be added  
> to your Elasticsearch install.
> 
> Cheers,  
> Dan

--

---

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 14, 2012, 4:37am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/6 "2012-08-14T04:37:58Z")

</div>

What if after indexing the book type, I now want to index another type  
called "cd". Do I just create a second \_river definition? I tried  
creating a second \_river definition to index cd but nothing seemed to  
happen. Help?

On Monday, 13 August 2012 18:44:51 UTC-5, Dan Everton wrote:

> I'm assuming you have a CouchDB database that has multiple types of  
> documents in it and you only want to index one of those types? You can do  
> that in two ways:
> 
> 1. A filter function on the CouchDB
> 2. A filter function on the Elasticsearch side.
> 
> The first way requires you to add a filter function to a design document  
> in the CouchDB database. So in your design document you'll have something  
> like:
> 
> {  
> "\_id" : "\_design/books",  
> "filters" : {  
> "only" : "function(doc, req) { return doc.type == 'book' }"  
> }  
> }
> 
> And then your river definition looks like this:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "filter" : "books/only"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> The second way to do it is to use a script on the Elasticsearch side. This  
> is essentially the same as the previous way, but puts the script execution  
> burden on ES not CouchDB. So your river definition in this case looks like:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "script" : "ctx.ignore = ctx.doc.type != 'book'"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> Note that this river definition requires the Javascript plugin to be added  
> to your Elasticsearch install.
> 
> Cheers,  
> Dan

--

---

<div class="post-metadata">

**Author:** ![Dan\_Everton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_everton/32/1865_2.png) [@Dan\_Everton](https://discuss.elastic.co/u/Dan_Everton)\
**Post date:** [August 14, 2012, 5:27am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/7 "2012-08-14T05:27:13Z")

</div>

On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:

> What if after indexing the book type, I now want to index another type  
> called "cd". Do I just create a second \_river definition? I tried  
> creating a second \_river definition to index cd but nothing seemed to  
> happen. Help?

If everything's in the same CouchDB database you can just use a single  
river definition and use a script on the Elasticsearch side to set the  
type. So say you have a "type" field in your documents in CouchDB that is  
also the same as the type you want to use in Elasticsearch. Your river  
definition would look like this:

{  
"type" : "couchdb",  
"couchdb" : {  
"host" : "localhost",  
"port" : 5984,  
"db" : "amazon",  
"script" : "ctx.\_type = ctx.doc.type"  
},  
"index" : {  
"index" : "amazon"  
}  
}

Setting ctx.\_type tells Elasticsearch what type in the index to use. So if  
doc.type for a CouchDB changeset is "book" then Elasticsearch will index it  
as the "book" type. Obviously you can make the script a bit more  
sophisticated if your types don't map one to one. I'm not aware of any way  
to externalise that script unfortunately.

Cheers,  
Dan

--

---

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 14, 2012, 5:39am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/8 "2012-08-14T05:39:51Z")

</div>

Damn....I feel so stupid for not realizing that this could be done. I read  
about the Elasticsearch \_type.....but wasn't sure what to make of it.

Thank you once again

On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:

> On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> 
> > What if after indexing the book type, I now want to index another type  
> > called "cd". Do I just create a second \_river definition? I tried  
> > creating a second \_river definition to index cd but nothing seemed to  
> > happen. Help?
> 
> If everything's in the same CouchDB database you can just use a single  
> river definition and use a script on the Elasticsearch side to set the  
> type. So say you have a "type" field in your documents in CouchDB that is  
> also the same as the type you want to use in Elasticsearch. Your river  
> definition would look like this:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "script" : "ctx.\_type = ctx.doc.type"  
> },  
> "index" : {  
> "index" : "amazon"  
> }  
> }
> 
> Setting ctx.\_type tells Elasticsearch what type in the index to use. So if  
> doc.type for a CouchDB changeset is "book" then Elasticsearch will index it  
> as the "book" type. Obviously you can make the script a bit more  
> sophisticated if your types don't map one to one. I'm not aware of any way  
> to externalise that script unfortunately.
> 
> Cheers,  
> Dan

--

---

<div class="post-metadata">

**Author:** ![Mark\_Huang\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_huang_2/32/2641_2.png) [@Mark\_Huang\_2](https://discuss.elastic.co/u/Mark_Huang_2)\
**Post date:** [August 14, 2012, 7:43am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/9 "2012-08-14T07:43:21Z")

</div>

So I had this requirement which David Pilato solved for me:  
[https://groups.google.com/d/msg/elasticsearch/KRPldny3wUk/nrfJtkOsv-oJ](https://groups.google.com/d/msg/elasticsearch/KRPldny3wUk/nrfJtkOsv-oJ)

As you can see, a book may have a different output fields when compared  
with a cd. And hence I wanted a way to index different types (matching my  
couchdb type with Elasticsearch type) separately. If I were to use:

script: ctx.\_type = ctx.doc.type, I wouldn't be able to set the different  
output fields per document type. Someone else told me that I could use  
partial fields to accomplish this, is this right?

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

On Tuesday, 14 August 2012 00:39:51 UTC-5, Mark Huang wrote:

> Damn....I feel so stupid for not realizing that this could be done. I  
> read about the Elasticsearch \_type.....but wasn't sure what to make of it.
> 
> Thank you once again
> 
> On Tuesday, 14 August 2012 00:27:13 UTC-5, Dan Everton wrote:
> 
> > On Tuesday, August 14, 2012 2:37:58 PM UTC+10, Mark Huang wrote:
> > 
> > > What if after indexing the book type, I now want to index another type  
> > > called "cd". Do I just create a second \_river definition? I tried  
> > > creating a second \_river definition to index cd but nothing seemed to  
> > > happen. Help?
> > 
> > If everything's in the same CouchDB database you can just use a single  
> > river definition and use a script on the Elasticsearch side to set the  
> > type. So say you have a "type" field in your documents in CouchDB that is  
> > also the same as the type you want to use in Elasticsearch. Your river  
> > definition would look like this:
> > 
> > {  
> > "type" : "couchdb",  
> > "couchdb" : {  
> > "host" : "localhost",  
> > "port" : 5984,  
> > "db" : "amazon",  
> > "script" : "ctx.\_type = ctx.doc.type"  
> > },  
> > "index" : {  
> > "index" : "amazon"  
> > }  
> > }
> > 
> > Setting ctx.\_type tells Elasticsearch what type in the index to use. So  
> > if doc.type for a CouchDB changeset is "book" then Elasticsearch will index  
> > it as the "book" type. Obviously you can make the script a bit more  
> > sophisticated if your types don't map one to one. I'm not aware of any way  
> > to externalise that script unfortunately.
> > 
> > Cheers,  
> > Dan

--

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 14, 2012, 8:36am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/10 "2012-08-14T08:36:33Z")

</div>

Hi Dan,

Nice and complete answer.

May I suggest that you send a pull request with that full example on the couchDb river (README file)?  
It could help many other users.

David

--

Le 14 août 2012 à 01:44, Dan Everton [dan@iocaine.org](mailto:dan@iocaine.org) a écrit :

> I'm assuming you have a CouchDB database that has multiple types of documents in it and you only want to index one of those types? You can do that in two ways:
> 
> 1. A filter function on the CouchDB
> 2. A filter function on the Elasticsearch side.
> 
> The first way requires you to add a filter function to a design document in the CouchDB database. So in your design document you'll have something like:
> 
> {  
> "\_id" : "\_design/books",  
> "filters" : {  
> "only" : "function(doc, req) { return doc.type == 'book' }"  
> }  
> }
> 
> And then your river definition looks like this:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "filter" : "books/only"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> The second way to do it is to use a script on the Elasticsearch side. This is essentially the same as the previous way, but puts the script execution burden on ES not CouchDB. So your river definition in this case looks like:
> 
> {  
> "type" : "couchdb",  
> "couchdb" : {  
> "host" : "localhost",  
> "port" : 5984,  
> "db" : "amazon",  
> "script" : "ctx.ignore = ctx.doc.type != 'book'"  
> },  
> "index" : {  
> "index" : "amazon",  
> "type" : "book"  
> }  
> }
> 
> Note that this river definition requires the Javascript plugin to be added to your Elasticsearch install.
> 
> Cheers,  
> Dan
> 
> --

--

---

<div class="post-metadata">

**Author:** ![Dan\_Everton](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_everton/32/1865_2.png) [@Dan\_Everton](https://discuss.elastic.co/u/Dan_Everton)\
**Post date:** [August 20, 2012, 1:30am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/11 "2012-08-20T01:30:55Z")

</div>

On Tuesday, August 14, 2012 6:36:33 PM UTC+10, David Pilato wrote:

> May I suggest that you send a pull request with that full example on the  
> couchDb river (README file)?  
> It could help many other users.

Hrm, good idea.  
See [Add indexing multiple types example. by deverton · Pull Request #15 · elastic/elasticsearch-river-couchdb · GitHub](https://github.com/elasticsearch/elasticsearch-river-couchdb/pull/15)

Cheers,  
Dan

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:15am UTC](https://discuss.elastic.co/t/indexing-different-types-from-a-couchdb-database/8729/12 "2017-07-06T03:15:51Z")

</div>