Wildcards in field paths within a query


(Josh P) #1

I was wondering whether it was possible in elasticsearch to write a
query that says something like, "return all documents in which the
contents of field 'foo.*.bar' matches a certain keyword." I know
elasticsearch can perform wildcard matches on the content, but for the
data I'm working with I'll need to specify the field names themselves
using wildcards.

My mappings are as follows:

curl -XPOST 'http://localhost:9200/userprofiles' -d '{
"mappings": {
"user": {
"properties: {
"events": {
"type": "object", "dynamic": "true"
}
},

  "dynamic_templates": [
    {
      "type": {
        "path_match": "*.*.type", "mapping": { "type": "string" }
      }
      "dat": {
        "path_match": "*.*.dat", "mapping": { "type": "string" }
      }
    }
  ]
}

}
}'

Let's say I insert these two documents:

curl -XPUT 'http://localhost:9200/userprofiles/user/1' -d '{
"events": {
"10": [
{ type: "view", dat: "http://www.google.com" }
],
"30": [
{ type: "view", dat: "http://www.yahoo.com" },
{ type: "share", dat: "http://www.facebook.com" }
]
}
}'

curl -XPUT 'http://localhost:9200/userprofiles/user/2' -d '{
"events": {
"15": [
{ type: "view", dat: "http://www.cnet.com" }
]
}
}

The numbers "10", "30", and "15" can be any arbitrary numbers.

I want to write a query that says, "fetch all documents that have at
least one 'events.*.type' that equals 'share'" (and would hence fetch
only document 1). Essentially, something like this:

curl -XPOST 'http://localhost:9200/userprofiles/_search?pretty=true' -
d '{
"filter": {
"term": { "user-events.*.type": "share" }
}
}'

...but elasticsearch doesn't seem to like the * here. Is there any
alternate way to achieve this? Unfortunately I do not have any
control over the structure of the documents.


(Josh P) #2

Sorry, meant to write:

curl -XPOST 'http://localhost:9200/userprofiles/_search?pretty=true' -
d '{
"filter": {
"term": { "events.*.type": "share" }
}
}'

On Mar 12, 1:31 am, Josh P josh.por...@gmail.com wrote:

I was wondering whether it was possible in elasticsearch to write a
query that says something like, "return all documents in which the
contents of field 'foo.*.bar' matches a certain keyword." I know
elasticsearch can perform wildcard matches on the content, but for the
data I'm working with I'll need to specify the field names themselves
using wildcards.

My mappings are as follows:

curl -XPOST 'http://localhost:9200/userprofiles'-d '{
"mappings": {
"user": {
"properties: {
"events": {
"type": "object", "dynamic": "true"
}
},

  "dynamic_templates": [
    {
      "type": {
        "path_match": "*.*.type", "mapping": { "type": "string" }
      }
      "dat": {
        "path_match": "*.*.dat", "mapping": { "type": "string" }
      }
    }
  ]
}

}

}'

Let's say I insert these two documents:

curl -XPUT 'http://localhost:9200/userprofiles/user/1'-d '{
"events": {
"10": [
{ type: "view", dat: "http://www.google.com" }
],
"30": [
{ type: "view", dat: "http://www.yahoo.com" },
{ type: "share", dat: "http://www.facebook.com" }
]
}

}'

curl -XPUT 'http://localhost:9200/userprofiles/user/2'-d '{
"events": {
"15": [
{ type: "view", dat: "http://www.cnet.com" }
]
}

}

The numbers "10", "30", and "15" can be any arbitrary numbers.

I want to write a query that says, "fetch all documents that have at
least one 'events.*.type' that equals 'share'" (and would hence fetch
only document 1). Essentially, something like this:

curl -XPOST 'http://localhost:9200/userprofiles/_search?pretty=true'-
d '{
"filter": {
"term": { "user-events.*.type": "share" }
}

}'

...but elasticsearch doesn't seem to like the * here. Is there any
alternate way to achieve this? Unfortunately I do not have any
control over the structure of the documents.


(Igor Motov) #3

The only solution that comes to mind is to use script field that would
iterate through all fields in the document, select fields matching
events.*.type and compare them to the desired event type. Needless to say,
this is going to be extremely slow and really ugly. So, it might be
worth considering adding a separate field "types" on the root level that
would include a list of all different event types for the profile and just
search this field.

Please, notice that every time a new number appears under events, two new
fields are created and mapping is getting dynamically updated on all
shards. It's somewhat expensive operation comparing to plain indexing. So,
if you anticipate to have really large number of different numbers under
events, you might need to reconsider your schema.

On Monday, March 12, 2012 1:34:20 AM UTC-4, Josh P wrote:

Sorry, meant to write:

curl -XPOST 'http://localhost:9200/userprofiles/_search?pretty=true' -
d '{
"filter": {
"term": { "events.*.type": "share" }
}
}'

On Mar 12, 1:31 am, Josh P josh.por...@gmail.com wrote:

I was wondering whether it was possible in elasticsearch to write a
query that says something like, "return all documents in which the
contents of field 'foo.*.bar' matches a certain keyword." I know
elasticsearch can perform wildcard matches on the content, but for the
data I'm working with I'll need to specify the field names themselves
using wildcards.

My mappings are as follows:

curl -XPOST 'http://localhost:9200/userprofiles'-d '{
"mappings": {
"user": {
"properties: {
"events": {
"type": "object", "dynamic": "true"
}
},

  "dynamic_templates": [ 
    { 
      "type": { 
        "path_match": "*.*.type", "mapping": { "type": "string" } 
      } 
      "dat": { 
        "path_match": "*.*.dat", "mapping": { "type": "string" } 
      } 
    } 
  ] 
} 

}

}'

Let's say I insert these two documents:

curl -XPUT 'http://localhost:9200/userprofiles/user/1'-d '{
"events": {
"10": [
{ type: "view", dat: "http://www.google.com" }
],
"30": [
{ type: "view", dat: "http://www.yahoo.com" },
{ type: "share", dat: "http://www.facebook.com" }
]
}

}'

curl -XPUT 'http://localhost:9200/userprofiles/user/2'-d '{
"events": {
"15": [
{ type: "view", dat: "http://www.cnet.com" }
]
}

}

The numbers "10", "30", and "15" can be any arbitrary numbers.

I want to write a query that says, "fetch all documents that have at
least one 'events.*.type' that equals 'share'" (and would hence fetch
only document 1). Essentially, something like this:

curl -XPOST 'http://localhost:9200/userprofiles/_search?pretty=true'-
d '{
"filter": {
"term": { "user-events.*.type": "share" }
}

}'

...but elasticsearch doesn't seem to like the * here. Is there any
alternate way to achieve this? Unfortunately I do not have any
control over the structure of the documents.


(system) #4