Switching off the indexing on specific fields

Elastic Stack Elasticsearch
1

Hi I went through ES documentation bit confusing for me

I have a document with 50fields... in that i want to index on 5 fields. 45 fields i dont want to index or dont want to search or sort or filter or aggregate...

I made below changes

 "_all": {
    "enabled": false
  },

for each fields

     "doc_values": false,
      "norms": {
        "enabled": false
      },
      "index": "no"

are these settings sufficient? Can i now assume that these fields are neither indexed or not undergoing any special treatment. I just want it ti be returned as part of whole document. Dont want to query field by field also

Performance is very critical for me. So just want to cut down all computation and indexing overhead.

2

Yeah

3

After this i populated index with docs, but when i did execute some query, do aggregation and do groupby(term order) it returns the results? Why so.. is it expected? if expected as no index or no norms or no doc_values enabled then why its returning results?

Can you tell if am doing anything wrong

4

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem.

5

Here are the details:

Template:

{
"template": "*-log",
"order": 10,
"mappings": {
  "accesslog": {
    "_all": {
    "enabled": false
  },
  "properties": {
    "accountid": {
      "type": "string",
      "index_options": "docs",
      "norms": {
        "enabled": false
      },
      "index": "not_analyzed"
    },
    "applicationid": {
      "type": "string",
      "index_options": "docs",
      "norms": {
        "enabled": false
      },
      "index": "not_analyzed"
    },
    "connstarttime": {
      "type": "long",
      "doc_values": false,
      "norms": {
        "enabled": false
      },
      "index": "no"
    },
    "esttime": {
      "type": "long",
      "doc_values": false,
      "norms": {
        "enabled": false
      },
      "index": "no"
    }
      }
    }
  }
}

liske this i have around 70 fields, 60 fields with below setting

      : {
      "type": "long",
      "doc_values": false,
      "norms": {
        "enabled": false
      },
      "index": "no"
    }

And i keep pumping the data, document from ES looks like below

    {
 "took": 6,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 267582,
"max_score": 1.0,
"hits": [
  {
    "_index": "706963e4-f062-42da-b370-845045eb4160-1815955521-2018.03.16.03-accesslog",
    "_type": "accesslog",
    "_id": "AWIvZ7xquw9w1jqlI1tC",
    "_score": 1.0,
    "_source": {
      "applicationid": 0,
      "accountid": 0,
      "ssl": false,
      "connstarttime": 0,
      "esttime": 0,

When i post query for searching connstarttime and aggregate sum by ssl

like below

 "aggregations" : {
"orders" : {
  "terms" : {
    "field" : "ssl"
  },
  "aggregations" : {
    "histogram" : {
      "histogram" : {
        "field" : "timestamp",
        "interval" : 60000,
        "order" : {
          "_key" : "asc"
        }
      },
     "aggregations" : {
        "responseSize_sum" : {
          "sum" : {
            "field" : "connstarttime"
          }
        }

it returns me the proper results...

How it is possible?

6

I can't take your script as is and run it and reproduce.
If you can provide that, that'd help...

7

Can you show the actual mappings for your index? Given that your template has order set to 10, do you have any other templates that may take effect due to higher priority?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.