Query side synonyms and boosts

Hello,

While doing some testing with query side synonyms and boosts, I am noticing
that the boosts do not seem to be honored for synonym derived keywords. For
example, if I have defined a boost when the keyword is "ipad" and I have
also defined a synonym where "i pad" => "ipad", if then submit a search
with "i pad" as my keyword, I do see that the synonym fired accordingly,
but the boost I have for ipad is not honored. Is there anyway to leverage
both the synonym and boost, without having to duplicate all my boosts
across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

After further review, it was dumbass user error that was the root cause for
the boosts not firing :slight_smile: However, that being said, I am still looking
for a way to honor both synonyms and boosts on the query side if possible.
If I understood better how synonyms are implemented under the hood, that
might help me come up with solution. Does it simply create a bool "OR"
query for all the terms in the synonym?

On Wed, May 9, 2012 at 6:45 PM, Carey Boldenow carey.boldenow@gmail.comwrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Please disregard the above as I was able to get the boosts to fire on my
synonyms. It's amazing what you can do when one actually reviews their own
code :wink: If not a problem, I would still be interested to know at a high
level how the synonyms are implemented on the query side.

On Wed, May 9, 2012 at 8:38 PM, Carey Boldenow carey.boldenow@gmail.comwrote:

After further review, it was dumbass user error that was the root cause
for the boosts not firing :slight_smile: However, that being said, I am still
looking for a way to honor both synonyms and boosts on the query side if
possible. If I understood better how synonyms are implemented under the
hood, that might help me come up with solution. Does it simply create a
bool "OR" query for all the terms in the synonym?

On Wed, May 9, 2012 at 6:45 PM, Carey Boldenow carey.boldenow@gmail.comwrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Effectively, if you do a text/query_string query on a field with an
analyzer (search_analyzer more specifically) that expands to several
tokens, they will be OR'ed, yes.

On Thu, May 10, 2012 at 4:59 AM, Carey Boldenow carey.boldenow@gmail.comwrote:

Please disregard the above as I was able to get the boosts to fire on my
synonyms. It's amazing what you can do when one actually reviews their own
code :wink: If not a problem, I would still be interested to know at a high
level how the synonyms are implemented on the query side.

On Wed, May 9, 2012 at 8:38 PM, Carey Boldenow carey.boldenow@gmail.comwrote:

After further review, it was dumbass user error that was the root cause
for the boosts not firing :slight_smile: However, that being said, I am still
looking for a way to honor both synonyms and boosts on the query side if
possible. If I understood better how synonyms are implemented under the
hood, that might help me come up with solution. Does it simply create a
bool "OR" query for all the terms in the synonym?

On Wed, May 9, 2012 at 6:45 PM, Carey Boldenow carey.boldenow@gmail.comwrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.com wrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.com wrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

That would be a great feature. Many of our search relevancy issues are
solved with synonyms. :wink:

--
Ivan

On Tue, May 15, 2012 at 1:22 PM, Shay Banon kimchy@gmail.com wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.com wrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution for
a rather large ecommerce site, and two guiding principles we know going in
are; change will be constant, and outages are never acceptable. Therefore,
for synonyms, it appears as though even if only executed on the query side,
any changes to the synonym list require a close/open on the index at a
minimum. Is there any other way to make those synonym changes real-time
(i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Absolutely, will do...

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I know
what the answer is. We are using ES to hopefully build a search solution
for a rather large ecommerce site, and two guiding principles we know going
in are; change will be constant, and outages are never acceptable.
Therefore, for synonyms, it appears as though even if only executed on the
query side, any changes to the synonym list require a close/open on the
index at a minimum. Is there any other way to make those synonym changes
real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

done.... https://github.com/elasticsearch/elasticsearch/issues/1956

On Tue, May 15, 2012 at 9:10 PM, my3sons carey.boldenow@gmail.com wrote:

Absolutely, will do...

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I
know what the answer is. We are using ES to hopefully build a search
solution for a rather large ecommerce site, and two guiding principles we
know going in are; change will be constant, and outages are never
acceptable. Therefore, for synonyms, it appears as though even if only
executed on the query side, any changes to the synonym list require a
close/open on the index at a minimum. Is there any other way to make those
synonym changes real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I
know what the answer is. We are using ES to hopefully build a search
solution for a rather large ecommerce site, and two guiding principles we
know going in are; change will be constant, and outages are never
acceptable. Therefore, for synonyms, it appears as though even if only
executed on the query side, any changes to the synonym list require a
close/open on the index at a minimum. Is there any other way to make those
synonym changes real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I
know what the answer is. We are using ES to hopefully build a search
solution for a rather large ecommerce site, and two guiding principles we
know going in are; change will be constant, and outages are never
acceptable. Therefore, for synonyms, it appears as though even if only
executed on the query side, any changes to the synonym list require a
close/open on the index at a minimum. Is there any other way to make those
synonym changes real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

On Tuesday, May 15, 2012 3:22:08 PM UTC-5, kimchy wrote:

Btw, can you open an issue for it?

On Tue, May 15, 2012 at 11:21 PM, Shay Banon kimchy@gmail.com wrote:

Yes, its is possible to automatic load changes to the synonym file, just
require some work.

On Sun, May 13, 2012 at 4:05 PM, my3sons carey.boldenow@gmail.comwrote:

Thanks Shay! One last question regarding this, although I think I
know what the answer is. We are using ES to hopefully build a search
solution for a rather large ecommerce site, and two guiding principles we
know going in are; change will be constant, and outages are never
acceptable. Therefore, for synonyms, it appears as though even if only
executed on the query side, any changes to the synonym list require a
close/open on the index at a minimum. Is there any other way to make those
synonym changes real-time (i.e. cache flush of some sort)?

On Wednesday, May 9, 2012 6:45:18 PM UTC-5, my3sons wrote:

Hello,

While doing some testing with query side synonyms and boosts, I am
noticing that the boosts do not seem to be honored for synonym derived
keywords. For example, if I have defined a boost when the keyword is "ipad"
and I have also defined a synonym where "i pad" => "ipad", if then submit a
search with "i pad" as my keyword, I do see that the synonym fired
accordingly, but the boost I have for ipad is not honored. Is there anyway
to leverage both the synonym and boost, without having to duplicate all my
boosts across the synonym terms, as we will have thousands of synonyms.

I would be curious to know how ES implements synonyms under the hood.

Thanks!

Hi,

Synonym search is working sometimes and not working sometimes. I'm not able to find the root cause of that.
I'm using synonym at search analyzer. Earlier what all the synonym related records got score is different from now.

Can someone help me in understanding the issue.

Thanks,
@vaniaravinda