Inconsistent Results From Percolator

[Kenneth_Loafman]

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist
heregit://gist.github.com/1287348.git.
The gist contains four files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[kimchy (Shay Banon)]

Heya, can you post the full link to the gist? The "here" is not linked for
some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman
kenneth.loafman@gmail.comwrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist here. The gist contains four
files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[Kenneth_Loafman_2]

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked for
some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist here. The gist contains four
files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[kimchy (Shay Banon)]

Can you post the parts that also registered the percolator queries? A very
simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked for
some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist here. The gist contains four
files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[Kenneth_Loafman_2]

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to run
test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was able to
duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon kimchy@gmail.com wrote:

Can you post the parts that also registered the percolator queries? A very
simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked
for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist here. The gist contains four
files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[Kenneth_Loafman_2]

There's a new make-filters.sh up that provides a readable output and
corrects some quoting issues.

...Ken

On Fri, Oct 14, 2011 at 1:43 PM, Kenneth Loafman kenneth@loafman.comwrote:

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to run
test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was able to
duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon kimchy@gmail.com wrote:

Can you post the parts that also registered the percolator queries? A very
simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked
for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're getting
inconsistent results from percolator. We have a three node cluster and it
appears that node 1 is getting correct results while nodes 2 and 3 are not.
This means that round-robbin access of percolator will cause us to miss
matches. I've tied it all up in a gist here. The gist contains four
files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[kimchy (Shay Banon)]

Hey, I ran the test, started two nodes, ran the curl that registers the
queries, also added one to create the filters index, and then ran the
percolate request, and got consistent results. I also noticed that in your
first output of the percolate request, I see results that are not part of
the registered percolator queries...

On Fri, Oct 14, 2011 at 10:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

There's a new make-filters.sh up that provides a readable output and
corrects some quoting issues.

...Ken

On Fri, Oct 14, 2011 at 1:43 PM, Kenneth Loafman kenneth@loafman.comwrote:

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to run
test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was able to
duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon kimchy@gmail.com wrote:

Can you post the parts that also registered the percolator queries? A
very simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked
for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're
getting inconsistent results from percolator. We have a three node cluster
and it appears that node 1 is getting correct results while nodes 2 and 3
are not. This means that round-robbin access of percolator will cause us to
miss matches. I've tied it all up in a gist here. The gist contains
four files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[Kenneth_Loafman_2]

Shay,

We updated the cluster to pick up last Saturday's fix for another problem
and this problem has gone away now.

Just part of being on the bleeding edge.

...Thanks,
...Ken

On Sun, Oct 16, 2011 at 6:40 PM, Shay Banon kimchy@gmail.com wrote:

Hey, I ran the test, started two nodes, ran the curl that registers the
queries, also added one to create the filters index, and then ran the
percolate request, and got consistent results. I also noticed that in your
first output of the percolate request, I see results that are not part of
the registered percolator queries...

On Fri, Oct 14, 2011 at 10:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

There's a new make-filters.sh up that provides a readable output and
corrects some quoting issues.

...Ken

On Fri, Oct 14, 2011 at 1:43 PM, Kenneth Loafman kenneth@loafman.comwrote:

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to
run test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was able
to duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon kimchy@gmail.com wrote:

Can you post the parts that also registered the percolator queries? A
very simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.com wrote:

Heya, can you post the full link to the gist? The "here" is not linked
for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're
getting inconsistent results from percolator. We have a three node cluster
and it appears that node 1 is getting correct results while nodes 2 and 3
are not. This means that round-robbin access of percolator will cause us to
miss matches. I've tied it all up in a gist here. The gist
contains four files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[kimchy (Shay Banon)]

Strange..., if you hit it again, don't hesitate to ping and I would love to
fix it if its a problem.

On Mon, Oct 17, 2011 at 1:26 PM, Kenneth Loafman kenneth@loafman.comwrote:

Shay,

We updated the cluster to pick up last Saturday's fix for another problem
and this problem has gone away now.

Just part of being on the bleeding edge.

...Thanks,
...Ken

On Sun, Oct 16, 2011 at 6:40 PM, Shay Banon kimchy@gmail.com wrote:

Hey, I ran the test, started two nodes, ran the curl that registers the
queries, also added one to create the filters index, and then ran the
percolate request, and got consistent results. I also noticed that in your
first output of the percolate request, I see results that are not part of
the registered percolator queries...

On Fri, Oct 14, 2011 at 10:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

There's a new make-filters.sh up that provides a readable output and
corrects some quoting issues.

...Ken

On Fri, Oct 14, 2011 at 1:43 PM, Kenneth Loafman kenneth@loafman.comwrote:

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to
run test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was able
to duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon kimchy@gmail.com wrote:

Can you post the parts that also registered the percolator queries? A
very simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman kenneth@loafman.comwrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon kimchy@gmail.comwrote:

Heya, can you post the full link to the gist? The "here" is not
linked for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're
getting inconsistent results from percolator. We have a three node cluster
and it appears that node 1 is getting correct results while nodes 2 and 3
are not. This means that round-robbin access of percolator will cause us to
miss matches. I've tied it all up in a gist here. The gist
contains four files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

[Ollie]

Hi,

We're seeing this exact issue on our cluster right now - running 0.19.0.
Was the cause of this ever found / fixed?

Ollie

On Tuesday, October 18, 2011 12:46:52 AM UTC+1, kimchy wrote:

Strange..., if you hit it again, don't hesitate to ping and I would love
to fix it if its a problem.

On Mon, Oct 17, 2011 at 1:26 PM, Kenneth Loafman <ken...@loafman.com<javascript:>

wrote:

Shay,

We updated the cluster to pick up last Saturday's fix for another problem
and this problem has gone away now.

Just part of being on the bleeding edge.

...Thanks,
...Ken

On Sun, Oct 16, 2011 at 6:40 PM, Shay Banon <kim...@gmail.com<javascript:>

wrote:

Hey, I ran the test, started two nodes, ran the curl that registers the
queries, also added one to create the filters index, and then ran the
percolate request, and got consistent results. I also noticed that in your
first output of the percolate request, I see results that are not part of
the registered percolator queries...

On Fri, Oct 14, 2011 at 10:19 PM, Kenneth Loafman <ken...@loafman.com<javascript:>

wrote:

There's a new make-filters.sh up that provides a readable output and
corrects some quoting issues.

...Ken

On Fri, Oct 14, 2011 at 1:43 PM, Kenneth Loafman <ken...@loafman.com<javascript:>

wrote:

inconsistent results from percolator · GitHub file make-filters.sh. You'll need to
run test-percolate.sh (updated) afterwards.

Note: we normally populate these dynamically using Pyes, but I was
able to duplicate this on a 2-node test machine and it failed like below:

mmadm@polaris:~/bugreport$ ./test-percolate.sh

9200

{"ok":true,"matches":["filter_0131_0222_0002403","filter_0131_0323_0007790","filter_0131_0222_0007958"]}

9201
{"ok":true,"matches":["filter_0131_0222_0007958"]}

...Thanks,
...Ken

On Fri, Oct 14, 2011 at 11:47 AM, Shay Banon <kim...@gmail.com<javascript:>

wrote:

Can you post the parts that also registered the percolator queries? A
very simple gist, having curl requests that set it up, and then percolate.

On Fri, Oct 14, 2011 at 6:39 PM, Kenneth Loafman <ken...@loafman.com<javascript:>

wrote:

Sorry, here it is: inconsistent results from percolator · GitHub

On Fri, Oct 14, 2011 at 10:58 AM, Shay Banon <kim...@gmail.com<javascript:>

wrote:

Heya, can you post the full link to the gist? The "here" is not
linked for some reason.

On Fri, Oct 14, 2011 at 5:08 PM, Kenneth Loafman <
kenneth...@gmail.com <javascript:>> wrote:

Hi,

We are running 18-SNAPSHOT from a couple of weeks ago and we're
getting inconsistent results from percolator. We have a three node cluster
and it appears that node 1 is getting correct results while nodes 2 and 3
are not. This means that round-robbin access of percolator will cause us
to miss matches. I've tied it all up in a gist here. The gist
contains four files:

1. simple testcase
2. testcase results
3. percolator status
4. percolator mapping

If you need anything else, I'll be happy to provide it.

...Thanks,
...Ken

--