Mult-language searchable in one field


(Martin-3) #1

Hi,

I've got descriptions in different languages with different analyzers like
this:

description_lang: {

  • properties: {
    • de: {
      • analyzer: my_analyzer_de
      • type: string
        }
    • en: {
      • analyzer: my_analyzer_en
      • type: string
        }
        }

}

They are searchabel with "description_lang.de" and "description_lang.en"but I'd like to search them all together with only
"
description_lang" is that possible somehow ?

Thank you very much,
Martin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b1edecec-4722-4e3f-b82c-eb742d7cb3ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Martin-3) #2

and I cant' use the "_all" field because of own scoring.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5d87fc34-f433-4b2f-83b9-7a0118b1ab8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Itamar Syn-Hershko) #3

And then what analyzer you will use for that?

It is doable, but I'd strongly suggest against it unless you know what you
are doing:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_multi_field_2

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Fri, May 9, 2014 at 1:40 PM, Martin martin.konradi@echtzeiten.comwrote:

Hi,

I've got descriptions in different languages with different analyzers like
this:

description_lang: {

  • properties: {
    • de: {
      • analyzer: my_analyzer_de
      • type: string
        }
    • en: {
      • analyzer: my_analyzer_en
      • type: string
        }
        }

}

They are searchabel with "description_lang.de" and "description_lang.en"but I'd like to search them all together with only
"
description_lang" is that possible somehow ?

Thank you very much,
Martin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b1edecec-4722-4e3f-b82c-eb742d7cb3ad%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b1edecec-4722-4e3f-b82c-eb742d7cb3ad%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtryLtndSjLdZu8dnmARMpsZhzxHc1y18XFKxUtv_DzYQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Martin-3) #4

Thank you - I thought may be there is another type like object-type which
would provide it.
Actually it should be a type like the array-type but with fields you could
analyse separately.
There would be different analysers for each language (especially because of
stemming).

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a8aa260b-a91b-4300-877a-c69928bdcfc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Itamar Syn-Hershko) #5

You can do that, but by generating the query on the client side. I'd then
use a Boolean query wrapping a Match query, each on a different language
field and specifying the analyzer explicitly.

I think that's a healthier approach as it's easily changable and easier to
debug, as often times multi-lingual search demands a lot of attention

Not long ago I gave a talk about this topic, you might find it helpful:
https://skillsmatter.com/skillscasts/4968-approaches-to-multi-lingual-text-search-with-elasticsearch-and-lucene

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Fri, May 9, 2014 at 3:46 PM, Martin martin.konradi@echtzeiten.comwrote:

Thank you - I thought may be there is another type like object-type which
would provide it.
Actually it should be a type like the array-type but with fields you could
analyse separately.
There would be different analysers for each language (especially because
of stemming).

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a8aa260b-a91b-4300-877a-c69928bdcfc8%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/a8aa260b-a91b-4300-877a-c69928bdcfc8%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtaEHYq8ELY-A0mTnLRkvCk76GXT7KeDqQfYva8by4K8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Martin-3) #6

OK - I thought I could leave all my queries and add an additional filter if
the item is available in that language.
I'll watch your talk!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fcd4b3b2-a254-4f0d-9f44-f4174e303a24%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #7

You can index two fields into a third common field with "index_name" in
combination with "multi_field". This works like a custom "_all" field.

Jörg

On Fri, May 9, 2014 at 3:08 PM, Martin martin.konradi@echtzeiten.comwrote:

OK - I thought I could leave all my queries and add an additional filter
if the item is available in that language.
I'll watch your talk!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fcd4b3b2-a254-4f0d-9f44-f4174e303a24%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/fcd4b3b2-a254-4f0d-9f44-f4174e303a24%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEC018Hmf%3DjXhTOoYtX5WE8255hwWcLjb1QtmbnWr-twQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Martin-3) #8

ah wow - that's cool. I just came across your
http://jprante.github.io/lessons/2012/05/11/ISBN-search-with-Elasticsearch.html
which explains it in detail.
That's pretty close to what I want but I just found a new problem: There
should be a highlighting of the found results - so I guess I'll have to go
with multiple boolean should queries.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7d18189b-9950-4020-bf3d-dd26680cfa46%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #9

Elasticsearch has improved ithe "index_name" with the "copy_to" mechanism.

I have set up a gist to demonstrate multilingual highlighting, maybe it is
useful.

Jörg

On Fri, May 9, 2014 at 4:19 PM, Martin martin.konradi@echtzeiten.comwrote:

ah wow - that's cool. I just came across your
http://jprante.github.io/lessons/2012/05/11/ISBN-search-with-Elasticsearch.htmlwhich explains it in detail.
That's pretty close to what I want but I just found a new problem: There
should be a highlighting of the found results - so I guess I'll have to go
with multiple boolean should queries.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7d18189b-9950-4020-bf3d-dd26680cfa46%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7d18189b-9950-4020-bf3d-dd26680cfa46%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHEzVWVJBkjOV2sLoWczdo2jSvaftmN7yFL4Xvjk7wN4w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #10