# Dynamic data and none index fields

**URL:** <https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728>\
**Category:** Elasticsearch\
**Created:** [April 19, 2020, 12:26pm UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728 "2020-04-19T12:26:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rhh](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Rhh](https://discuss.elastic.co/u/Rhh)\
**Post date:** [April 19, 2020, 12:26pm UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728/1 "2020-04-19T12:26:27Z")

</div>

Hi

I am trying to use ES and I am having trouble to get some fields not indexed. I put data into ES using serilog.

In my setup I have a custom template like this :

{  
"index\_patterns": ["tds-_"],  
"settings": {  
"index.refresh\_interval": "5s"  
},  
"mappings": {  
"default": {  
"dynamic\_templates": [  
{  
"numerics\_in\_fields": {  
"path\_match": "fields\.[\d+]$",  
"match\_pattern": "regex",  
"mapping": {  
"type": "text",  
"index": true,  
"norms": false  
}  
}  
},  
{  
"string\_fields": {  
"match": "_",  
"match\_mapping\_type": "string",  
"mapping": {  
"type": "text",  
"index": true,  
"norms": false,  
"fields": {  
"raw": {  
"type": "keyword",  
"index": true,  
"ignore\_above": 256  
}  
}  
}  
}  
}  
],  
"properties": {  
"message": {  
"type": "text",  
"index": true  
},  
"subject": {  
"type": "text",  
"index": true  
},  
"logger": {  
"type": "text",  
"index": true  
},  
"data": {  
"type": "object"  
}  
}  
}  
}  
}  
}

In my data I get results like this :

```
 {
    "_index" : "tds-2020.04.19",
    "_type" : "logevent",
    "_id" : "WtKDkXEBSyGvzhhwIUVi",
    "_score" : 1.0,
    "_source" : {
      "@timestamp" : "2020-04-19T10:16:59.8418722+02:00",
      "level" : "Error",
      "message" : "{\"Message\":\"4 forsøk\",\"p1\":\"heisan\",\"xSystem\":{\"Process\":{\"Id\":20860,\"Name\":\"WindowsFormsApp11\"},\"Thread\":{\"Id\":1,\"Name\":\"trilili\",\"Culture\":\"nb-NO\"}}",
      "fields" : {
        "logger" : "name of application",
        "subject" : "Første forsøk",
        "data" : {
          "p1" : "heisan",
          "system/process/id" : "20860",
          "system/process/name" : "WindowsFormsApp11",
          "system/thread/id" : "1",
          "system/thread/name" : "trilili",
          "system/thread/culture" : "nb-NO"
        }
      }
    }
  },

```

I would like the fields p1, system/… to be not indexed. How can I achieve that ? I think I need to change my template but everything I tried has so far not worked.

The fields are added by an serilog enritcher with logEvent.AddOrUpdateProperty

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 19, 2020, 1:35pm UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728/2 "2020-04-19T13:35:52Z")

</div>

You need to set `index: false` on those fields. See [https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html)

---

<div class="post-metadata">

**Author:** ![Rhh](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Rhh](https://discuss.elastic.co/u/Rhh)\
**Post date:** [April 19, 2020, 1:49pm UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728/3 "2020-04-19T13:49:48Z")

</div>

Hi

Thanks for your feedback. I know about this, but please look at my template 🙂

I need fields added dynamically to the data group to be not indexed.

{  
"index\_patterns": ["tds-_"],  
"settings": {  
"index.refresh\_interval": "5s"  
},  
"mappings": {  
"default": {  
"dynamic\_templates": [  
{  
"string\_fields": {  
"match": "_",  
"match\_mapping\_type": "string",  
"mapping": {  
"type": "text",  
"index": false,  
"norms": false,  
"fields": {  
"raw": {  
"type": "keyword",  
"index": false,  
"ignore\_above": 256  
}  
}  
}  
}  
}  
],  
"properties": {  
"message": {  
"type": "text",  
"index": true  
},  
"subject": {  
"type": "text",  
"index": true  
},  
"logger": {  
"type": "text",  
"index": true  
},  
"central": {  
"type": "nested",  
"properties": {  
"code": {  
"type": "text",  
"index": true  
},  
"name": {  
"type": "text"  
}  
}  
},  
"vehicle": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text",  
"index": true  
},  
"permit": {  
"type": "text"  
}  
}  
},  
"driver": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text",  
"index": true  
},  
"name": {  
"type": "text"  
}  
}  
},  
"booking": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text",  
"index": true  
},  
"text": {  
"type": "text"  
}  
}  
},  
"data": {  
"dynamic": true,  
"properties": {}  
},  
"exceptions": {  
"type": "nested",  
"properties": {  
"depth": {  
"type": "text"  
},  
"HResult": {  
"type": "text"  
},  
"StackTraceString": {  
"type": "text"  
}  
}  
}  
}  
}  
}  
}

I did think my current mapping would solve this, but no it does not. So how can I cange my mapping to acchive this ?

Regards

---

<div class="post-metadata">

**Author:** ![Rhh](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Rhh](https://discuss.elastic.co/u/Rhh)\
**Post date:** [April 20, 2020, 8:18am UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728/4 "2020-04-20T08:18:25Z")

</div>

I have also tried to change my template, but no success. Any help on this ?

{  
"index\_patterns": ["tds-_"],  
"settings": {  
"index.refresh\_interval": "5s"  
},  
"mappings": {  
"default": {  
"dynamic\_templates": [  
{  
{  
"dynamic\_rule": {  
"match": "data_",  
"match\_mapping\_type": "string",  
"mapping": {  
"type": "text",  
"index": false,  
"norms": false  
}  
}  
},  
"standard\_rule": {  
"match": "\*",  
"match\_mapping\_type": "string",  
"mapping": {  
"type": "text",  
"index": true,  
"norms": false,  
"fields": {  
"raw": {  
"type": "keyword",  
"index": true,  
"ignore\_above": 256  
}  
}  
}  
}  
}  
],  
"properties": {  
"message": {  
"type": "text"  
},  
"subject": {  
"type": "text"  
},  
"logger": {  
"type": "text"  
},  
"central": {  
"type": "nested",  
"properties": {  
"code": {  
"type": "text"  
},  
"name": {  
"type": "text",  
"index": false  
}  
}  
},  
"vehicle": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text"  
},  
"permit": {  
"type": "text",  
"index": false  
}  
}  
},  
"driver": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text"  
},  
"name": {  
"type": "text",  
"index": false  
}  
}  
},  
"booking": {  
"type": "nested",  
"properties": {  
"number": {  
"type": "text"  
},  
"text": {  
"type": "text",  
"index": false  
}  
}  
},  
"data": {  
"type": "nested",  
"properties": {  
},  
"exceptions": {  
"type": "nested",  
"properties": {  
"depth": {  
"type": "text",  
"index": false  
},  
"HResult": {  
"type": "text",  
"index": false  
},  
"StackTraceString": {  
"type": "text",  
"index": false  
}  
}  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2020, 8:18am UTC](https://discuss.elastic.co/t/dynamic-data-and-none-index-fields/228728/5 "2020-05-18T08:18:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.