Problem with indexed documents, after calling drop on collection

Hi everyone,
I'm using elasticsearch mongodb river. After calling drop() on my
collection all the documents that was dropped are still in index, even
after removing of index and creating new one they still appears in results.
I suspect that somehow drop() command not affect oplog file(which is used
by mongodb river) that is why these documents are still in index. Are there
any ideas how can I make index not "see" these dropped documents?
Thank you,
Taras

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You probably deleted the _river index not the index itself.
Removing an index removes all your documents (rm -rf behind the scene).

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:17, Taras Petrytsyn pegasus.ua@gmail.com a écrit :

Hi everyone,
I'm using elasticsearch mongodb river. After calling drop() on my collection all the documents that was dropped are still in index, even after removing of index and creating new one they still appears in results. I suspect that somehow drop() command not affect oplog file(which is used by mongodb river) that is why these documents are still in index. Are there any ideas how can I make index not "see" these dropped documents?
Thank you,
Taras

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi David,
no actually I also removed index. And when I created newone, these
documents were already in index :frowning: as I mentioned before I think problem is
in oplog, because as I understood, mongodb river doesn't work directly with
data, but use oplog. And in case of calling drop() command documents that
were dropped aren't identified by river as deleted. Interesting, that in
case of calling remove() command (without parameters it removes all
documents) threre is no problem.
Thank you,
Taras

Середа, 26 червня 2013 р. 11:30:18 UTC+3 користувач David Pilato написав:

You probably deleted the _river index not the index itself.
Removing an index removes all your documents (rm -rf behind the scene).

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:17, Taras Petrytsyn <pegas...@gmail.com <javascript:>>
a écrit :

Hi everyone,
I'm using elasticsearch mongodb river. After calling drop() on my
collection all the documents that was dropped are still in index, even
after removing of index and creating new one they still appears in results.
I suspect that somehow drop() command not affect oplog file(which is used
by mongodb river) that is why these documents are still in index. Are there
any ideas how can I make index not "see" these dropped documents?
Thank you,
Taras

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

If you remove the index, docs in Elasticsearch should be removed.
If not, there's something weird here.

Could you share curl commands you are running to delete the index?

You're right about oplog thing. Removing a database in mongodb won't remove the index in ES AFAIK.

May be after you removed the index, all documents are again fetched from the oplog by the river?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:42, Taras Petrytsyn pegasus.ua@gmail.com a écrit :

Hi David,
no actually I also removed index. And when I created newone, these documents were already in index :frowning: as I mentioned before I think problem is in oplog, because as I understood, mongodb river doesn't work directly with data, but use oplog. And in case of calling drop() command documents that were dropped aren't identified by river as deleted. Interesting, that in case of calling remove() command (without parameters it removes all documents) threre is no problem.
Thank you,
Taras

Середа, 26 червня 2013 р. 11:30:18 UTC+3 користувач David Pilato написав:

You probably deleted the _river index not the index itself.
Removing an index removes all your documents (rm -rf behind the scene).

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:17, Taras Petrytsyn pegas...@gmail.com a écrit :

Hi everyone,
I'm using elasticsearch mongodb river. After calling drop() on my collection all the documents that was dropped are still in index, even after removing of index and creating new one they still appears in results. I suspect that somehow drop() command not affect oplog file(which is used by mongodb river) that is why these documents are still in index. Are there any ideas how can I make index not "see" these dropped documents?
Thank you,
Taras

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I removed index using UI controls of head plugin.

  • May be after you removed the index, all documents are again fetched from
    the oplog by the river?
    I think so too. But don't know how to handle fetching of documents that
    were dropped.

Середа, 26 червня 2013 р. 11:50:50 UTC+3 користувач David Pilato написав:

If you remove the index, docs in Elasticsearch should be removed.
If not, there's something weird here.

Could you share curl commands you are running to delete the index?

You're right about oplog thing. Removing a database in mongodb won't
remove the index in ES AFAIK.

May be after you removed the index, all documents are again fetched from
the oplog by the river?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:42, Taras Petrytsyn <pegas...@gmail.com <javascript:>>
a écrit :

Hi David,
no actually I also removed index. And when I created newone, these
documents were already in index :frowning: as I mentioned before I think problem is
in oplog, because as I understood, mongodb river doesn't work directly with
data, but use oplog. And in case of calling drop() command documents that
were dropped aren't identified by river as deleted. Interesting, that in
case of calling remove() command (without parameters it removes all
documents) threre is no problem.
Thank you,
Taras

Середа, 26 червня 2013 р. 11:30:18 UTC+3 користувач David Pilato написав:

You probably deleted the _river index not the index itself.
Removing an index removes all your documents (rm -rf behind the scene).

My 2 cents

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 26 juin 2013 à 10:17, Taras Petrytsyn pegas...@gmail.com a écrit :

Hi everyone,
I'm using elasticsearch mongodb river. After calling drop() on my
collection all the documents that was dropped are still in index, even
after removing of index and creating new one they still appears in results.
I suspect that somehow drop() command not affect oplog file(which is used
by mongodb river) that is why these documents are still in index. Are there
any ideas how can I make index not "see" these dropped documents?
Thank you,
Taras

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.