How to exclude fields from being indexed in ElasticSearch?

Elastic Stack Elasticsearch
1

I am trying to utilize ElasticSearch to store large sets of data. Most of the data will be searchable, however, there are some field that will be there just so the data is stored and returned upon request.

Here is my mapping

{
  "mappings": {
    "properties": {
      "amenities": {
        "type": "completion"
      },
      "summary": {
        "type": "text"
      },
      "street_number": {
        "type": "text"
      },
      "street_name": {
        "type": "text"
      },
      "street_suffix": {
        "type": "text"
      },
      "city": {
        "type": "text",
        "fields": {
          "raw": { 
            "type":  "keyword"
          }
      },
      "state_or_province": {
        "type": "text"
      },
      "postal_code": {
        "type": "text"
      },
      "mlsid": {
        "type": "text"
      },
      "source_id": {
        "type": "text"
      },
      "status": {
        "type": "keyword"
      },
      "type": {
        "type": "keyword"
      },
      "subtype": {
        "type": "keyword"
      },
      "year_built": {
        "type": "short"
      },
      "community": {
        "type": "keyword"
      },
      "elementary_school": {
        "type": "keyword"
      },
      "middle_school": {
        "type": "keyword"
      },
      "jr_high_school": {
        "type": "keyword"
      },
      "high_school": {
        "type": "keyword"
      },
      "area_size": {
        "type": "double"
      },
      "lot_size": {
        "type": "double"
      },
      "bathrooms": {
        "type": "double"
      },
      "bedrooms": {
        "type": "double"
      },
      "listed_at": {
        "type": "date"
      },
      "price": {
        "type": "double"
      },
      "sold_at": {
        "type": "date"
      },
      "sold_for": {
        "type": "double"
      },
      "total_photos": {
        "type": "short"
      },
      "formatted_addressLine": {
        "type": "text"
      },
      "formatted_address": {
        "type": "text"
      },
      "location": {
        "type": "geo_point"
      },
      "price_changes": {
        "type": "object"
      },
      "fields": {
        "type": "object"
      },
      "deleted_at": {
        "type": "date"
      },
      "is_available": {
        "type": "boolean"
      },
      "is_unable_to_find_coordinates": {
        "type": "boolean"
      },
      "source": {
        "type": "keyword"
      }
    }
  }
}

The fields and price_changes properties are there in case the user want to read that info. But that info should not be searchable or indexed. The fields holds a large list of key-value pairs whereas price_changes fields hold multiple objects of the same type.

Currently, when I attempt to bulk create records, I get Limit of total fields [1000] has been exceeded error. I am guessing this error is happening because every key-value pair in the fields collection is considered a field in elasticsearch.

How can I store the fields and the price_changes object as non-searchable data and not index it or count it toward the fields count?

2

I think enabled is your solution.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.