Search questions

Elastic Stack Elasticsearch
1

Hello!
I have index with documents which have dynamic set of fields:
{
....
'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx',
'fields': [
{
'name': 'first_name', 'value': ['First']
},
{
'name': 'last_name', 'value': ['Last']},
{
'name': 'phone', 'value': ['+000000000']
}
],
...
}

I need to handle 2 kinds of search requests.

  1. Search only for documents which have specific field ('email' for
    example) and ignore other.
  2. Search for documents specific fields of which should match search
    criteria (for example display only documents which fields with names
    'first_name', 'last_name' or 'email' starts from 'a')

For first type of search request I'm create the following search query:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx'}
},
{
'term': { 'fields.name': 'email'}
},
]
}
},
'query': {
'prefix': {
'fields.value' :'f'
}
}
}
}

I'm expected this query should filter out documents without 'email' in
'fields' object but its is present in response.

For second type of search request I have no idea. Search query may look like:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx'}
},
{
'term': { 'fields.name': 'email'}
},
]
}
},
'query':
{
'prefix': {
'fields.value' :'f'
}
}
}
}

--
Andrew Degtiariov
DA-RIPE

2

Ooops, sorry I have send uncompleted message.

2010/8/25 Andrew Degtiariov andrew.degtiariov@gmail.com:

Hello!
I have index with documents which have dynamic set of fields:
{
....
'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx',
'fields': [
{
'name': 'first_name', 'value': ['First']
},
{
'name': 'last_name', 'value': ['Last']},
{
'name': 'phone', 'value': ['+000000000']
}
],
...
}

I need to handle 2 kinds of search requests.

  1. Search only for documents which have specific field ('email' for
    example) and ignore other.
  2. Search for documents specific fields of which should match search
    criteria (for example display only documents which fields with names
    'first_name', 'last_name' or 'email' starts from 'a')

For first type of search request I'm create the following search query:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx'}
},
{
'term': { 'fields.name': 'email'}
},
]
}
},
'query': {
'prefix': {
'fields.value' :'f'
}
}
}
}

I'm expected this query should filter out documents without 'email' in
'fields' object but its is present in response.

For second type of search request I have no idea. Search query may look like:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx'}
}
]
}
},
'query': {
'or': [
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'email' }
]
},
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'first_name' }
]
},
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'last_name }
]
}
]
}
}
}
}

--
Andrew Degtiariov
DA-RIPE

3

Hi,

The way queries on multi fields works is not by actually returning only
the documents where both fields exists, but think of it as a each one
reduces the documents to search on, and each one works against the whole
document. So, in your case, the filter does filter only for documents that
have email in the "fields.name", but then the prefix query will match on
any fields.value that starts with f.

I think that a better solution for this is to move the "fields.name" to
be fields level document fields, for example:

{
"company_id" : "xxxx",
"first_name" : "first",
"last_name" : "last",
"phone" : "..."
}

This will allow you to do what you want. And thanks to the fact that
fields gets added dynamically, you can add other fields as you go.

-shay.banon

On Wed, Aug 25, 2010 at 10:26 AM, Andrew Degtiariov <
andrew.degtiariov@gmail.com> wrote:

Ooops, sorry I have send uncompleted message.

2010/8/25 Andrew Degtiariov andrew.degtiariov@gmail.com:

Hello!
I have index with documents which have dynamic set of fields:
{
....
'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx',
'fields': [
{
'name': 'first_name', 'value': ['First']
},
{
'name': 'last_name', 'value': ['Last']},
{
'name': 'phone', 'value': ['+000000000']
}
],
...
}

I need to handle 2 kinds of search requests.

  1. Search only for documents which have specific field ('email' for
    example) and ignore other.
  2. Search for documents specific fields of which should match search
    criteria (for example display only documents which fields with names
    'first_name', 'last_name' or 'email' starts from 'a')

For first type of search request I'm create the following search query:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id':
'xxxxxxxxxxxxxxxxxxxxxxx'}
},
{
'term': { 'fields.name': 'email'}
},
]
}
},
'query': {
'prefix': {
'fields.value' :'f'
}
}
}
}

I'm expected this query should filter out documents without 'email' in
'fields' object but its is present in response.

For second type of search request I have no idea. Search query may look
like:

{
'query': {
'filtered': {
'filter': {
'and': {
'filters': [
{
'term': { 'company_id': 'xxxxxxxxxxxxxxxxxxxxxxx'}
}
]
}
},
'query': {
'or': [
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'email' }
]
},
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'first_name' }
]
},
{
'and': [
{ 'prefix': { ''fields.value' :'f' },
{ 'term': { 'field.name': 'last_name }
]
}
]
}
}
}
}

--
Andrew Degtiariov
DA-RIPE

4

2010/8/27 Shay Banon shay.banon@elasticsearch.com:

Hi,
The way queries on multi fields works is not by actually returning only
the documents where both fields exists, but think of it as a each one
reduces the documents to search on, and each one works against the whole
document. So, in your case, the filter does filter only for documents that
have email in the "fields.name", but then the prefix query will match on
any fields.value that starts with f.
I think that a better solution for this is to move the "fields.name" to
be fields level document fields, for example:
{
"company_id" : "xxxx",
"first_name" : "first",
"last_name" : "last",
"phone" : "..."
}

Thank you, but for some purposes I have leave fields in structure of
my document but convert it to be an object:
{
"company_id": "xxxx",
"fields" {
"phone": [ "ddddd", "bbbbbb" ],
"first_name": [ "first" ],
"last_name": [ "last" ]
}
}

Now regular queries worked as expected but not more_like_this queries :frowning:

q1 = {
"query": {
"more_like_this": {
"fields" : [ "fields.email" ],
"like_text" : "pseudo.uzer@gmail.com"
}
}
}

q2 = {
"query": {
"query_string": {
"query" : "fields.email: pseudo.uzer@gmail.com"
}
}
}

q1 match nothing but q2 match 2 documents.

Requests like 'http://localhost:9200/contacts/contact/XXXXXXXXXXXXXXXXXXXX/_mlt?fields=fields.email'
don't show any matches too.

PS. I'm trying to match for field "company_id" and get the same
results (no matches)

--
Andrew Degtiariov
DA-RIPE