Elastic Search - Free Text Search - Nested fields - (Elastic 6.2)


(Sarathmskr) #1

GET /companies_v1/_search
{
"query": {
"simple_query_string": {
"query": "so-well"
}
}
}
This works fine.

The same I need to search in Nested objects and in non nested at the same time. My full document contain more than 500 fields, most of them are in nested level. when I search non nested It's working fine, but not for nested fields.

I tried to create a sample .. like below, but no luck.. getting empty result.

{
"query": {
"bool": {
"must": [{
"nested": {
"query": {
"bool": {
"must": [{
"simple_query_string": {
"query": "so-well"
}
}]
}
},
"path": "companyName"
}
}, {
"nested": {
"query": {
"bool": {
"must": [{
"simple_query_string": {
"query": "so-well"
}
}]
}
},
"path": "companyState"
}
}, {
"simple_query_string": {
"query": "so-well"
}
}]
}
}
}
Let me know if anything need to be add.

Mapping

PUT /companies_v1
{
"mappings": {
"companies": {
"properties": {
"companyName": {
"type": "nested"
},
"companyState": {
"type": "nested"
},
"companySpecification": {
"type": "nested"
},
"companyOperator": {
"type": "nested"
}
}
}
},
"settings" :{
"index.mapping.total_fields.limit": 2000,
"number_of_shards" : 1,
"number_of_replicas" : 0
}
}

I was searching for free Text in all my fields, not specifying any particular, and I have more than 500 fields in each document


(Jimferenczi) #2

Which version of Elasticsearch are you using ? Using a must in your boolean query makes the match in each nested level mandatory. Is it what you're expecting ? A document should match only if it matches a non-nested field and at least one field in each nested path clause ? I tried you recreation in 6.2 and it works as expected, what happens if you change the must to a should ?


(Sarathmskr) #3

FYI,
the "so-well" value is matched one of the property in companyName nested object
in companyName object I have Id, name and city... it matched with name.

GET companies/_search
{
"query": {
"bool": {
"must": [{
"nested": {
"query": {
"bool": {
"must": [{
"simple_query_string": {
"query": "so-well",
"fields": ["companyName.name"]
}
}]
}
},
"path": "companyName"
}
}]
}
}
}
.. this is working for nested when I give exact match and field name. if I am not specifying field name then it is returning by default 10 records which matched 1 record and rest other 9 records

But my requirement is to search the input("so-well") in all fields(including nested objects) in my document wherever it matches. for example if it matches in 10 records it should return matched records.


(Jimferenczi) #5

Can you provide the version of Elasticsearch that you use in your tests ?
Can you also post the output of the following command:

POST companies/_validate/query?explain
{  
   "query":{  
      "bool":{  
         "must":[  
            {  
               "nested":{  
                  "query":{  
                     "query_string":{  
                        "query":"so-well"
                     }
                  },
                  "path":"companyName"
               }
            }
         ]
      }
   }
}

(Sarathmskr) #6

Version:

{
"name" : "",
"cluster_name" : "",
"cluster_uuid" : "",
"version" : {
"number" : "6.2.2",
"build_hash" : "10b1edd",
"build_date" : "2018-02-28T15:42:08.616107Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

Output of the above Query

Document is too big Just I am pasting only few which ever is required

{
"valid": true,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"explanations": [
{
"index": "companies",
"valid": true,
"explanation": """+ToParentBlockJoinQuery (+
companyName.companyName:so-well
}]}

for few fields by default it is appending .keyword

for date fields below
MatchNoDocsQuery("failed [county.effectiveDate] query, caused by number_format_exception:[For input string: "so-well"]")


(David Pilato) #7

we are using aws managed elastic search

Not related to your question but did you look at cloud.elastic.co and https://aws.amazon.com/marketplace/pp/B01N6YCISK ?


(Sarathmskr) #8

Same I am testing with local elastic search, getting same error


(David Pilato) #9

Yeah. As i said it's unrelated to your question. I was just saying that if you are looking for an hosted elasticsearch as a service solution you should checkout this page: https://www.elastic.co/cloud


(Sarathmskr) #10

Sorry, I didn't read your post properly,
Yeah, We are using elastic cloud(14 days trial version) and aws elastic cloud to evaluate.
Later we will take a call on which to go.


(David Pilato) #11

Great that you are aware of it!

(To me the choice is pretty obvious as cloud by elastic is the only way to have access to X-Pack. Think about what is there like Security, Monitoring, Reporting and what is coming like Canvas, SQL... But I might be biased though :smile: )


(Sarathmskr) #12

Hi,
Below is working for me....
is this fine to get nested and non-nested values with same query, if this is correct, what about performance (500 fields), is there any issue while searching more than 2 million records.

Or any other way to get, if yes, could you please provide sample mapping and query for nested or non nested

PUT sarath_test?pretty
{
"mappings": {
"sarath": {
"properties": {
"all_fields": {
"type": "keyword"
},
"xyz": {
"type": "keyword"
},
"test": {
"type": "nested",
"properties": {
"abc": {
"type": "keyword",
"copy_to": "all_fields"
}
}
}
}
}
}
}

--

PUT sarath_test/sarath/1
{
"xyz": "John",
"test":[
{
"abc": "Smith"
}
]
}


GET sarath_test/_search
{
"query": {
"simple_query_string": {
"query": "Smith"
}
}
}

GET sarath_test/_search
{
"query": {
"simple_query_string": {
"query": "John"
}
}
}


(Sarathmskr) #13

Hi,
Could you please update


(system) #14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.