Store and index "no" not working

Elastic Stack Elasticsearch
1

Here comes another offspring to this question: Exclude complete path from indexing and _source

What I want to achieve here is to
-exclude a field called "ignoreme" from the _source AND
-exclude the very same field from the index itself

So in the following example, I start with the following index creation:

curl -XPOST 'http://192.168.56.102:9200/test' -d '{"mappings":
{"my_type":{"_all":{"enabled":false},
"_source":{"excludes":["ignoreme"]},
"dynamic_templates":[{"skip":{"match":"ignoreme","mapping":{"type":"string","store":"no","index":"no"}}}]}}}'

The index then looks like this:

curl -XGET http://192.168.56.102:9200/test?pretty
{
  "test" : {
    "aliases" : { },
    "mappings" : {
      "my_type" : {
        "_all" : {
          "enabled" : false
        },
        "_source" : {
          "excludes" : [ "ignoreme" ]
        },
        "dynamic_templates" : [ {
          "skip" : {
            "mapping" : {
              "index" : "no",
              "store" : "no",
              "type" : "string"
            },
            "match" : "ignoreme"
          }
        } ]
      }
    },....

Next, I post this document:

curl -XPOST 'http://192.168.56.102:9200/test/my_type/1' -d '{"title":"Something","ignoreme":"I should not be here"}'

I now expect that "ignoreme" should not be searchable at all, but when I run a query for the word "here" in "ignoreme", I still get a result:

curl -XGET http://192.168.56.102:9200/test/my_type/_search?pretty&q=ignoreme:here
[1] 3463
ben@ben-ubuntu14:~$ {
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "test",
      "_type" : "my_type",
      "_id" : "1",
      "_score" : 1.0,
      "_source":{"title":"Something"}
    } ]
  }
}

So what do I have to do to TRULY and COMPLETELY exclude "ignoreme"?

In addition, I am wondering whether this type of mapping is also possible for arrays and/or object types.
E.g. if I have an array of strings like this:
"messages":["Help, I need someboy","It's been a hard days night","Norwegian Wood"]
is there a way to exclude the complete array from being indexed and added to the _source?

2

Can you print your mapping?

GET test/my_type/_mapping
3

Sure, here it is:

curl -XGET http://192.168.56.102:9200/test/_mapping/my_type?pretty

{
  "test" : {
    "mappings" : {
      "my_type" : {
        "_all" : {
          "enabled" : false
        },
        "_source" : {
          "excludes" : [ "ignoreme" ]
        },
        "dynamic_templates" : [ {
          "skip" : {
            "mapping" : {
              "index" : "no",
              "store" : "no",
              "type" : "string"
            },
            "match" : "ignoreme"
          }
        } ],
        "properties" : {
          "ignoreme" : {
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

I.e. this is what the mappings look like after I have POSTed my test document.

4

As you can see, your template has not been applied.

May be you add this mapping on an existing index?

This script works well:

DELETE index
PUT index
{
  "mappings": {
    "type": {
      "dynamic_templates": [
        {
          "skip": {
            "match": "ignoreme",
            "mapping": {
              "type": "string",
              "index": "no",
              "store": "no"
            }
          }
        }
      ]
    }
  }
}
PUT index/type/1
{
  "ignoreme": "foo"
}
GET index/type/_mapping
5

Hi David,

If you look at my example, this is basically what I have done.

I have now tested some more and left out this part from the initial POST

 "_all" : {
      "enabled" : false
    },
    "_source" : {
      "excludes" : [ "ignoreme" ]
    },

After that, the mapping looks better:

{
  "test" : {
    "mappings" : {
      "my_type" : {
        "dynamic_templates" : [ {
          "skip" : {
            "mapping" : {
              "index" : "no",
              "store" : "no",
              "type" : "string"
            },
            "match" : "ignoreme"
          }
        } ],
        "properties" : {
          "ignoreme" : {
            "type" : "string",
            "index" : "no"
          },
          "title" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

However, the query:

  curl -XGET http://192.168.56.102:9200/test/my_type/_search?pretty&q=ignoreme:here

still returns the document. This puzzles me.

6

Remove the pretty in case you are doing something wrong with the command line. I suspect your score is 1 here so no criteria has been applied.

This works well for me:

DELETE index
PUT index
{
  "mappings": {
    "type": {
      "dynamic_templates": [
        {
          "skip": {
            "match": "ignoreme",
            "mapping": {
              "type": "string",
              "index": "no",
              "store": "no"
            }
          }
        }
      ]
    }
  }
}
PUT index/type/1
{
  "ignoreme": "foo"
}
GET index/type/_search
{
  "query": {
    "match": {
      "ignoreme": "foo"
    }
  }
}
GET index/type/_search?q=ignoreme:foo
1 Like
7

Yes, that did it.
So

curl -XGET http://192.168.56.102:9200/test/my_type/_search?q=ignoreme:here

returns the correct (empty) result, while

curl -XGET http://192.168.56.102:9200/test/my_type/_search?pretty&q=ignoreme:here

does not.
I wonder if there is still a way to add the "pretty" param to this search URL. All the examples in the documentation are pretty-printed, so it should be possible I guess?

8

Wrap the URL in ""

1 Like
9

:slight_smile:
Cheers!