probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently use
Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a different
field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution to my
problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
But now I have another problem: our hierarchies are not limited to one
level. I want to make "broadcast" a subfield of some other field "foo" and
its data should be searchable in "foo". So "broadcast" should be a
combination of a "nested" and a "multi_field" type. Is that possible
somehow?
for this and other (such as indexing aggregated fields several times say
normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently use
Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a different
field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution to
my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times say
normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently use
Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a different
field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution to
my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
It is what you have been doing with multifields say you have abcd field
which consists of ab and cd fields and those consist of a,b and c,d
properties of your json. when you map a you would have it mapped to both ab
and abcd same for b,c and d. I think it should work but with deep and wide
hierarchy it will quickly become enormous and error prone. I wish there
were a way to tell that abcd consists of ab and cd then you can roll up
your fields nicely and reuse it across mappings
Alex
On Wednesday, March 20, 2013 4:37:09 AM UTC-4, Pat wrote:
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times say
normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently
use Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a different
field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution to
my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
Ok, i get what you mean. Actually I am already trying to extend my mapping
this way and exactly like you said, this is enormous and error prone...
Am Mittwoch, 20. März 2013 16:03:34 UTC+1 schrieb AlexR:
Pat,
It is what you have been doing with multifields say you have abcd field
which consists of ab and cd fields and those consist of a,b and c,d
properties of your json. when you map a you would have it mapped to both ab
and abcd same for b,c and d. I think it should work but with deep and wide
hierarchy it will quickly become enormous and error prone. I wish there
were a way to tell that abcd consists of ab and cd then you can roll up
your fields nicely and reuse it across mappings
Alex
On Wednesday, March 20, 2013 4:37:09 AM UTC-4, Pat wrote:
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times say
normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently
use Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a different
field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution
to my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
yeah and besides rolling up your composite fields there is another aspect.
You often want to index the same field several times using different
analysis - say normal, stemmed and shingled and even with one big composite
field like built in _all you have to do your multifield mapping 3 times
for each flavor of your indexing. now add to it your roll up and you will
end up with 3 times number of multifield mappings your roll up alone will
need (and it may a dozens if you have fairly big and deeply nested json)
Ok, i get what you mean. Actually I am already trying to extend my mapping
this way and exactly like you said, this is enormous and error prone...
Am Mittwoch, 20. März 2013 16:03:34 UTC+1 schrieb AlexR:
Pat,
It is what you have been doing with multifields say you have abcd field
which consists of ab and cd fields and those consist of a,b and c,d
properties of your json. when you map a you would have it mapped to both ab
and abcd same for b,c and d. I think it should work but with deep and wide
hierarchy it will quickly become enormous and error prone. I wish there
were a way to tell that abcd consists of ab and cd then you can roll up
your fields nicely and reuse it across mappings
Alex
On Wednesday, March 20, 2013 4:37:09 AM UTC-4, Pat wrote:
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times
say normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently
use Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a
different field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution
to my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
I am wondering about the way the _all field is implemented in ES. Since you
can decide if a field should be included in _all or not, it seems to me
there is a kind of "copy field" working behind the scenes. Would it be that
difficult to extend ES so that you can define multiple "_all" fields, each
including different fields?
Am Mittwoch, 20. März 2013 16:24:48 UTC+1 schrieb AlexR:
yeah and besides rolling up your composite fields there is another aspect.
You often want to index the same field several times using different
analysis - say normal, stemmed and shingled and even with one big composite
field like built in _all you have to do your multifield mapping 3 times
for each flavor of your indexing. now add to it your roll up and you will
end up with 3 times number of multifield mappings your roll up alone will
need (and it may a dozens if you have fairly big and deeply nested json)
Ok, i get what you mean. Actually I am already trying to extend my
mapping this way and exactly like you said, this is enormous and error
prone...
Am Mittwoch, 20. März 2013 16:03:34 UTC+1 schrieb AlexR:
Pat,
It is what you have been doing with multifields say you have abcd field
which consists of ab and cd fields and those consist of a,b and c,d
properties of your json. when you map a you would have it mapped to both ab
and abcd same for b,c and d. I think it should work but with deep and wide
hierarchy it will quickly become enormous and error prone. I wish there
were a way to tell that abcd consists of ab and cd then you can roll up
your fields nicely and reuse it across mappings
Alex
On Wednesday, March 20, 2013 4:37:09 AM UTC-4, Pat wrote:
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times
say normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We currently
use Oracle Text, where I can simply query the "broadcast" field and all
subfields are automatically included. I need to migrate to ES for several
reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of the
subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a
different field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a solution
to my problem. I cannot use SOLR since I need nested objects to avoid cross
object matches. I often read that one can achieve the same functionality in
ES, but I couldn't find a solution to my problem.
I am wondering about the way the _all field is implemented in ES. Since
you can decide if a field should be included in _all or not, it seems to me
there is a kind of "copy field" working behind the scenes. Would it be that
difficult to extend ES so that you can define multiple "_all" fields, each
including different fields?
Am Mittwoch, 20. März 2013 16:24:48 UTC+1 schrieb AlexR:
yeah and besides rolling up your composite fields there is another
aspect. You often want to index the same field several times using
different analysis - say normal, stemmed and shingled and even with one big
composite field like built in _all you have to do your multifield mapping 3
times for each flavor of your indexing. now add to it your roll up and you
will end up with 3 times number of multifield mappings your roll up alone
will need (and it may a dozens if you have fairly big and deeply nested
json)
Ok, i get what you mean. Actually I am already trying to extend my
mapping this way and exactly like you said, this is enormous and error
prone...
Am Mittwoch, 20. März 2013 16:03:34 UTC+1 schrieb AlexR:
Pat,
It is what you have been doing with multifields say you have abcd field
which consists of ab and cd fields and those consist of a,b and c,d
properties of your json. when you map a you would have it mapped to both ab
and abcd same for b,c and d. I think it should work but with deep and wide
hierarchy it will quickly become enormous and error prone. I wish there
were a way to tell that abcd consists of ab and cd then you can roll up
your fields nicely and reuse it across mappings
Alex
On Wednesday, March 20, 2013 4:37:09 AM UTC-4, Pat wrote:
Can you give me an example how this can be achieved with multifield?
Am Dienstag, 19. März 2013 18:51:45 UTC+1 schrieb AlexR:
for this and other (such as indexing aggregated fields several times
say normal, stemmed, shingled) reasons it would be nice to be able to
establish an aggregation chain where aggergated (via multi_field) fields
would be able to feed into other aggregated fields
it can be achieved with current multifield but at a cost of a highly
redundant mapping. while having such redundant mapping may not be a big
deal for schemas with few fields it is a very big deal for schemas with
hundreds of fields
On Tuesday, March 19, 2013 9:04:53 AM UTC-4, Pat wrote:
Hi,
probably this question has been asked before, but I couldn't find an
appropriate answer. I have a document structure like this
{
"broadcast": {
"type": "nested",
"properties": {
"date": { "type": "date" },
"duration": { "type": "long" },
"title": { "type": "string" }
}
}
}
and I want to query "broadcast" including all subfields. We
currently use Oracle Text, where I can simply query the "broadcast" field
and all subfields are automatically included. I need to migrate to ES for
several reasons and I need to make subfield searching available again.
I already tried different approaches:
Using a "multi_match" query with wildcards ( "fields" :
["broadcast.*"] ). This doesn't seem to work (the query returns an error)
since the subfields have different types. I know that searching for a
string in a date field doesn't seem to make much sense, but in Oracle Text
you can get a match for a date if you search for a string like "2000-01-01".
Using "include_in_parent". I was hoping that the values of
the subfields are included in the parent field.
Using "multi_field" type. I could copy all values to a
different field, but not to the parent field.
I heard that SOLR has a copyfield function, which would be a
solution to my problem. I cannot use SOLR since I need nested objects to
avoid cross object matches. I often read that one can achieve the same
functionality in ES, but I couldn't find a solution to my problem.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.