[Solved]Elasticsearch mapping {"dynamic" : "false"} not working?


(Feng Yu (Abcfy2)) #1

I don't want a field date to be indexed. So I disabled the dynamic property. Here is my mapping:

curl localhost:9200/logstash-2016.09.16/rtds/_mapping?pretty
{
  "logstash-2016.09.16" : {
    "mappings" : {
      "rtds" : {
        "dynamic" : "false",
        "_all" : {
          "enabled" : false
        },
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "@version" : {
            "type" : "string",
            "index" : "no",
            "doc_values" : true
          },
          "content" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "host" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "message" : {
            "type" : "string",
            "index" : "no",
            "doc_values" : true
          },
          "path" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "tags" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "type" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      }
    }
  }
}

You can see there's no date field in this mapping.

But When I search, the date field appeared:

 curl localhost:9200/logstash-2016.09.16/rtds/_search?pretty
{
  "took" : 1093,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2624596,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "logstash-2016.09.16",
      "_type" : "rtds",
      "_id" : "AVcwSvrWxFnTKGlIg7xt",
      "_score" : 1.0,
      "_source" : {
        "@timestamp" : "2016-09-16T00:00:01.276Z",
        "@version" : "1",
        "path" : "/var/log/rtds/rtds.log",
        "host" : "kaishan",
        "type" : "rtds",
        "tags" : [ "rtds", "vertx", "throttled" ],
        "date" : "2016-09-16 08:00:01,276",
        "loglevel" : "ERROR",
        "content" : "找不到对应的iid信息:12 ,sid = 00003310 ,eid = 00003310",
        "subject" : "找不到对应的iid信息:12 ,sid = 00003310 ,eid = 00003310"
      }
    }, {
      "_index" : "logstash-2016.09.16",
      "_type" : "rtds",
      "_id" : "AVcwSvrWxFnTKGlIg7x5",
      "_score" : 1.0,
      "_source" : {
        "@timestamp" : "2016-09-16T00:00:01.374Z",
        "@version" : "1",
        "path" : "/var/log/rtds/rtds.log",
        "host" : "kaishan",
        "type" : "rtds",
        "tags" : [ "rtds", "vertx", "throttled" ],
        "date" : "2016-09-16 08:00:01,374",
        "loglevel" : "ERROR",
        "content" : "找不到对应的iid信息:5 ,sid = 00002134 ,eid = 00002134",
        "subject" : "找不到对应的iid信息:5 ,sid = 00002134 ,eid = 00002134"
      }
    }
...

You can see the date appeared in the _source. What's my fault?


(Feng Yu (Abcfy2)) #2

And here is my dynamic template setting:

curl localhost:9200/_template/logstash?pretty
{
  "logstash" : {
    "order" : 0,
    "template" : "logstash-*",
    "settings" : {
      "index" : {
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "_default_" : {
        "dynamic" : "false",
        "_all" : {
          "enabled" : false
        },
        "properties" : {
          "path" : {
            "index" : "not_analyzed",
            "type" : "string",
            "doc_values" : true
          },
          "@timestamp" : {
            "index" : "not_analyzed",
            "type" : "date",
            "doc_values" : true
          },
          "@version" : {
            "index" : "no",
            "type" : "string",
            "doc_values" : true
          },
          "host" : {
            "index" : "not_analyzed",
            "type" : "string",
            "doc_values" : true
          },
          "type" : {
            "index" : "not_analyzed",
            "type" : "string",
            "doc_values" : true
          },
          "message" : {
            "index" : "no",
            "type" : "string",
            "doc_values" : true
          },
          "content" : {
            "index" : "not_analyzed",
            "type" : "string",
            "doc_values" : true
          },
          "tags" : {
            "index" : "not_analyzed",
            "type" : "string",
            "doc_values" : true
          }
        }
      }
  }
}

(Zachary Tong) #3

The date field isn't indexed, but it remains in the original _source. The source is the original JSON document that you sent to Elasticsearch, which is never modified. So that's why you see it.

But if you were to execute a search against the date field directly, you won't see any results, as none of the data was indexed.


(Feng Yu (Abcfy2)) #4

Aha, I've got it. Thank you for your answer.


(system) #5