Creating index with field type keyword in app search

Elastic Search
,
1

We are using docker compose to self manage elastic enterprise search , kibana and Elasticsearch 8.4.1 . license Gold

Everything works fine but as we are trying to create geocoding search engine giving us some issues.

we have columns like

Name Text
District_name keyword
Category keyword
address keyword
phone-no text
location lat, long

now i have two problems

First is when we ingest data with app search pipeline it converts data in text search which results category , district _name as full text search and we need them to be keyword search only.

we tried creating engine from index , create index and ingest data ( this engine is managed by Elasticsearch ) . But seems doesn't get it through properly as we were missing something . no errors in particular.

how to allow app search to use keyword as text field .

--second issue is we beleive is because of full text or keywords.

when we search pizza we get matchings at top with name pizza pizza ( two consecutive words) how we can stop this behaviour.

Also experiencing an performance issue when searching using proximity boost first five results are in correct proximity but for example

pizza

a pizza 20 mtr
b pizza 50 mtr
c pizza 4 km
d pizza 200 km
e pizza 56 km
f pizza 30 mtr .

Only searching with two column name and proximity. Name is same pizza no other text . What could be the issue? Why location proximity works for first few and not for the rest.

is there a simpler documentation or link to a video which will tell you how to create index , ingest data with app search but with your own mapping. All in app search not plain Elasticsearch.

2

There is no actual keyword in App Search you just query the field different in 8.4 that is a value filter, take a look at this

So in short an App Search text field in App Search is both full text and keyword

3

You'll need to check both precision tuning and relevance tuning for fine-tuning your results. When looking for text, it's normal that results with more frequency of the terms that you're searching for come first. Are there other fields that you could search for as well? Or other criteria to make other results come on top of those?

when searching using proximity boost first five results are in correct proximity but for example

This could be a problem with the function you're using for the boost value (see proximity boosts function). Most probably it is decaying too quickly for results that are farther away than a specific limit, and thus boosting is not enough to make them being ordered in terms of distance.

If geolocation is really important, you could try sorting by location directly. Otherwise, I think you can experiment with the function and factor parameters for the proximity boost. Also, removing results that are too far away using geo filters could get rid of unwanted results.

is there a simpler documentation or link to a video which will tell you how to create index , ingest data with app search but with your own mapping

App Search abstracts you away from having to create your mapping. You can change your schema dynamically, thus not needing to create a mapping up front. In case you want to use App Search with a custom mapping, you can check Elasticsearch index engines for creating an App Search engine with an existing Elasticsearch index.

Keep in mind that this is a more advanced use case than directly creating an engine for App Search, as it involves you creating the index and adhering to specific field conventions so App Search features work correctly.

I hope that helps!

4

cool. so when we call search api we tell it to be consider a field as a 'keyword' right? that what you mean. Please share the link if it is correct. Search API filters | Elastic App Search Documentation [8.4] | Elastic . trying to understand the link shared havent got to an undeerandding how to do it. already exlained about schema and use case.

5

Thanks carlos for you reply. For the proximity boost issue we are trying to do this as mention here as a woraround to get better results Geo location distance sorting by proximity boost.

The thing about the mapping is we are just creating a very simple search engine with name and district_name search. As i mention i ingestion works fine but when we try to update schema we have only four option( now five nested ) . It doesnt have keyword in it. So all together we need to see as stephenb suggested to query as keyword which is new approach for me . I appreaciate if you can understand my use case and suggest how to get desire results .

we have columns like

Name Text
District_name keyword
Category keyword
address keyword
phone-no text
location lat, long

if elastic_search ingest District_name ,Category ,address as text how i can still restrict them to do only keyword search on these columns instead of full text search. Thanks in advance

6

If you're using App Search, there is no keyword field - it is implicitly added for you. You define the field as text in the schema view.

If you don't need to perform full text search, you can use filters to limit the results you'll get. For example, if you want to only get results from a specific district, you would do:

POST /api/as/v1/engines/<engine>/search
{
  "query": "your-query-here",
  "filters" : {
    "District_name": [ "district-name-you-want-to-look-for" ]
  }
}

If I got correctly your use case, you're looking for the closest results to a specific point (for example, 37.386483, -122.083842) that are in a specific district at a radius of for example 3 km at the most.

You could issue the following query for that:


POST /api/as/v1/engines/<engine>/search
{
  "query": "",
  "sort": [
    {
      "location": {
        "center": "37.386483, -122.083842",
        "order": "asc"
      }
    }
  ],
  "all": [
    {
      "District_name": [
        "district-name-you-want-to-look-for"
      ]
    },
    {
      "location": {
        "center": "37.386483, -122.083842",
        "distance": 3,
        "unit": "km"
      }
    }
  ]
}

See that I set the query to empty to specify that we don't need to search for anything and all results that match the filters will be returned, in the specified sort order.

Is this what you're looking for?

7

Sorry I made this confusing and hard to understand. The real use case is people will search in name not district_name. So for example

Pizza
dominos
MacDonald

it will fetch nearest location with name have pizza in it . but many cases people search for something in different district or city like

hotel in new york
restaurant in delhi
temple in london

like that.

so if new york or delhi and london is fully matched then it would add some score to results.

if user searches

long door office ---- it should not match with district_name london.

Hope you got my point . Currently it is matching some of the letters from district _name .

8

Understood, thanks for explaining!

App Search does not provide a per-field precision tuning. That means that precision for allowing fuzzy and prefix matching is effective on all fields that you're searching.

You could achieve this creating a new index, with a similar mapping from the current engine index (which would be .ent-search-engine-documents-<your-engine-name>). In this new mapping, you can use keyword fields for your district data so it is not affected by analysis.

Once your index is created, create an Elasticsearch based engine using the new index.

This should achieve the results you're expecting.

Hope that helps!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.