CouchDB index problem

Elastic Stack Elasticsearch
1

Hello! I'm working in a project which uses CouchDB as data base and ES as
search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but it
does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The
    fourth doc is never indexed.

Can someone help me? Thank you.

2

Can you check if the couchDb _changes API provided the 5 five documents you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using Ektorp library), the index updates correctly. However, sometimes it doesn't. I have been doing some tests, and I don't find an answer. The _changes stream always updates, so I don't understand why the index doesn't do that.

In addition, if the index doesn't changes with the new document, but it does with the next one, the first will never be counted. I mean, for example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The fourth doc is never indexed.

Can someone help me? Thank you.

3

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but it
does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The
    fourth doc is never indexed.

Can someone help me? Thank you.

4

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query: http://192.168.2.11:5984/debug_db/_changes and I get the new changes every time I create a new document, for example:
{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (http://192.168.2.11:9200/my_index/_status?pretty=true) still shows the previous status:
"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:
Can you check if the couchDb _changes API provided the 5 five documents you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using Ektorp library), the index updates correctly. However, sometimes it doesn't. I have been doing some tests, and I don't find an answer. The _changes stream always updates, so I don't understand why the index doesn't do that.

In addition, if the index doesn't changes with the new document, but it does with the next one, the first will never be counted. I mean, for example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The fourth doc is never indexed.

Can someone help me? Thank you.

5

No, I don't... Database has exactly 5 docs, _changes has notified 5
changes, but the index _status only has 4 docs (the fourth is missing)

PD I think I pressed the wrong button and sent you an email. I'm sorry if I
did that!

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but it
does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but it
does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but it
does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

6

And did you see something in ES logs ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:37, jose2b bangojose@gmail.com a écrit :

No, I don't... Database has exactly 5 docs, _changes has notified 5 changes, but the index _status only has 4 docs (the fourth is missing)

PD I think I pressed the wrong button and sent you an email. I'm sorry if I did that!

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:
Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query: http://192.168.2.11:5984/debug_db/_changes and I get the new changes every time I create a new document, for example:
{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (http://192.168.2.11:9200/my_index/_status?pretty=true) still shows the previous status:
"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:
Can you check if the couchDb _changes API provided the 5 five documents you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using Ektorp library), the index updates correctly. However, sometimes it doesn't. I have been doing some tests, and I don't find an answer. The _changes stream always updates, so I don't understand why the index doesn't do that.

In addition, if the index doesn't changes with the new document, but it does with the next one, the first will never be counted. I mean, for example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:
Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query: http://192.168.2.11:5984/debug_db/_changes and I get the new changes every time I create a new document, for example:
{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (http://192.168.2.11:9200/my_index/_status?pretty=true) still shows the previous status:
"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:
Can you check if the couchDb _changes API provided the 5 five documents you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using Ektorp library), the index updates correctly. However, sometimes it doesn't. I have been doing some tests, and I don't find an answer. The _changes stream always updates, so I don't understand why the index doesn't do that.

In addition, if the index doesn't changes with the new document, but it does with the next one, the first will never be counted. I mean, for example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:
Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query: http://192.168.2.11:5984/debug_db/_changes and I get the new changes every time I create a new document, for example:
{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (http://192.168.2.11:9200/my_index/_status?pretty=true) still shows the previous status:
"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:
Can you check if the couchDb _changes API provided the 5 five documents you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using Ektorp library), the index updates correctly. However, sometimes it doesn't. I have been doing some tests, and I don't find an answer. The _changes stream always updates, so I don't understand why the index doesn't do that.

In addition, if the index doesn't changes with the new document, but it does with the next one, the first will never be counted. I mean, for example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one. The fourth doc is never indexed.

Can someone help me? Thank you.

7

Oh, thank you, I din't see the logs before. There are different problems
(because I did different test with different index, diferent docs,
different db, etc.)

For example, in this case I get:

[2012-05-17 13:05:56,772][DEBUG][action.bulk ] [Sofen, Karla]
[hhh][3] failed to bulk item (index) index
{[hhh][hhh][39d895974c8b7616c3052b0bb220e78e],
source[{"_rev":"1-a2bc78e58c59ac6130916b6e61ae0d61","adminDate":"adn","_id":"39d895974c8b7616c3052b0bb220e78e","value":"1234567887654321","dateString":"15/05/2012
18:20","date":1337098800000,"type":"contraseña_offline"}]}
org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[hhh] tried to parse as object, but got EOF, has a concrete value been
provided to it?
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:442)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:473)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:417)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:311)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:157)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:529)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:427)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2012-05-17 13:05:56,773][WARN ][river.couchdb ] [Sofen, Karla]
[couchdb][hhh] failed to executefailure in bulk execution:
[0]: index [hhh], type [hhh], id [39d895974c8b7616c3052b0bb220e78e],
message [MapperParsingException[object mapping for [hhh] tried to parse as
object, but got EOF, has a concrete value been provided to it?]]

Note that 'hhh' is the name for db, index and type (ugly name, just for
tests). The doc that caused this conflict was:

{
"_id": "39d895974c8b7616c3052b0bb220e78e",
"_rev": "1-a2bc78e58c59ac6130916b6e61ae0d61",
"value": "1234567887654321",
"type": "contraseña_offline",
"date": 1337098800000,
"dateString": "15/05/2012 18:20",
"adminDate": "adn"
}

On Thursday, May 17, 2012 12:40:57 PM UTC+2, David Pilato wrote:

And did you see something in ES logs ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:37, jose2b bangojose@gmail.com a écrit :

No, I don't... Database has exactly 5 docs, _changes has notified 5
changes, but the index _status only has 4 docs (the fourth is missing)

PD I think I pressed the wrong button and sent you an email. I'm sorry if
I did that!

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but
it does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but
it does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

On Thursday, May 17, 2012 12:23:16 PM UTC+2, David Pilato wrote:

Did you update one of your doc in couchDb ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 12:14, jose2b bangojose@gmail.com a écrit :

Hi David, thank you.

I check the _changes stream with this query:
http://192.168.2.11:5984/debug_db/_changeshttp://192.168.2.11:5984/debug_key_control/_changes and
I get the new changes every time I create a new document, for example:

{"seq":5,"id":"39d895974c8b7616c3052b0bb220c0eb","changes":[{"rev":"1-a2bc78e58c59ac6130916b6e61ae0d61"}]}

But the index _status (
http://192.168.2.11:9200/my_index/_status?pretty=truehttp://192.168.2.11:9200/hhh/_status?pretty=true)
still shows the previous status:

"docs" : {
"num_docs" : 4,
"max_doc" : 4,
"deleted_docs" : 0
},

Any ideas? Do you want more info? How could I set log for CouchDB River?

On Thursday, May 17, 2012 11:53:45 AM UTC+2, David Pilato wrote:

Can you check if the couchDb _changes API provided the 5 five documents
you expect ?

Perhaps, you could set log for couchDb river to debug ?

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 11:50, jose2b bangojose@gmail.com a écrit :

Hello! I'm working in a project which uses CouchDB as data base and ES
as search tool, using the River plugin. I have created this index:

curl -XPUT '192.168.2.11:9200/_river/es_index/_meta' -d '{
"type" : "couchdb",
"couchdb" : {
"host" : "192.168.2.11",
"port" : 5984,
"db" : "debug_db","user" : "my_user","password" : "my_pwd",
"filter" : null
},
"index" : {
"index" : "my_index",
"type" : "my_type",
"bulk_size" : "100",
"bulk_timeout" : "10ms"
}
}'

Sometimes, when I create a new document (from Futon or from code using
Ektorp library), the index updates correctly. However, sometimes it
doesn't. I have been doing some tests, and I don't find an answer. The
_changes stream always updates, so I don't understand why the index doesn't
do that.

In addition, if the index doesn't changes with the new document, but
it does with the next one, the first will never be counted. I mean, for
example:

  1. I have 3 docs. I create another one. The index still has 3 docs.
  2. I create my fifth doc. The index only updates with the last one.
    The fourth doc is never indexed.

Can someone help me? Thank you.

8

I think it's relative to ñ
Try to modify it with n

Also, make sure that documents are similars (a number field can not become a string field)

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 13:33, jose2b bangojose@gmail.com a écrit :

contraseña

9

Hello again!

Yes I changed the ñ with n and it works! But, if I create the same doc
(with ñ) in another db it works... Why does it work? Do you have an
explanation?

I don't understand what you mean with "make sure that documents are
similars". Is it realitve to the field "value" in my doc? Why I can not
have a string only with numbers?

El jueves, 17 de mayo de 2012 13:43:24 UTC+2, David Pilato escribió:

I think it's relative to ñ
Try to modify it with n

Also, make sure that documents are similars (a number field can not become
a string field)

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 13:33, jose2b bangojose@gmail.com a écrit :

contraseña

10

As far as I remember, someone had the same problem some months ago.
But I don't remember if there was a fix for it.
Perhaps, your couchDb instance is not in UTF8 ?

Google the mailing list.

But if someone else here knows how to solve it ???

HTH
David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 18 mai 2012 à 10:11, jose2b bangojose@gmail.com a écrit :

Hello again!

Yes I changed the ñ with n and it works! But, if I create the same doc (with ñ) in another db it works... Why does it work? Do you have an explanation?

I don't understand what you mean with "make sure that documents are similars". Is it realitve to the field "value" in my doc? Why I can not have a string only with numbers?

El jueves, 17 de mayo de 2012 13:43:24 UTC+2, David Pilato escribió:
I think it's relative to ñ
Try to modify it with n

Also, make sure that documents are similars (a number field can not become a string field)

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 13:33, jose2b bangojose@gmail.com a écrit :

contraseña

11

Well, thank you again, I keep trying...

But it is not the only problem. In fact, this is more important. Can I have
a complex field wtih a field called "_id"? I get a "MapperParsingException:
Provided id [id_subfield] does not match the content one [id_doc]" and the
index does not update. But, if I randomly delete some other fields and
create the new doc (keeping both "_id"s), it is ok. I do not understand why
the behaviour is not the same...

Can you help me with this issue?

El viernes, 18 de mayo de 2012 10:33:50 UTC+2, David Pilato escribió:

As far as I remember, someone had the same problem some months ago.
But I don't remember if there was a fix for it.
Perhaps, your couchDb instance is not in UTF8 ?

Google the mailing list.

But if someone else here knows how to solve it ???

HTH
David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 18 mai 2012 à 10:11, jose2b bangojose@gmail.com a écrit :

Hello again!

Yes I changed the ñ with n and it works! But, if I create the same doc
(with ñ) in another db it works... Why does it work? Do you have an
explanation?

I don't understand what you mean with "make sure that documents are
similars". Is it realitve to the field "value" in my doc? Why I can not
have a string only with numbers?

El jueves, 17 de mayo de 2012 13:43:24 UTC+2, David Pilato escribió:

I think it's relative to ñ
Try to modify it with n

Also, make sure that documents are similars (a number field can not
become a string field)

David :wink:
Twitter : @dadoonet / @elasticsearchfr

Le 17 mai 2012 à 13:33, jose2b bangojose@gmail.com a écrit :

contraseña