# Not able to index records which contain missing fields ElasticSearch 7.4

**URL:** <https://discuss.elastic.co/t/not-able-to-index-records-which-contain-missing-fields-elasticsearch-7-4/230782>\
**Category:** Elasticsearch\
**Created:** [May 2, 2020, 1:19am UTC](https://discuss.elastic.co/t/not-able-to-index-records-which-contain-missing-fields-elasticsearch-7-4/230782 "2020-05-02T01:19:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![yoyomonkey](https://avatars.discourse-cdn.com/v4/letter/y/8baadc/32.png) [@yoyomonkey](https://discuss.elastic.co/u/yoyomonkey)\
**Post date:** [May 2, 2020, 1:19am UTC](https://discuss.elastic.co/t/not-able-to-index-records-which-contain-missing-fields-elasticsearch-7-4/230782/1 "2020-05-02T01:19:09Z")

</div>

Hello all,

Can someone point me in the right direction of any documents or links that may be able to help me.

I have am using kinesis to ingest just 2 simple events

```
2020-01-09 15:35:51,334 tid:LE7WZpV000NCG4NeSUVwBvW6HuQ DEBUG [org.sourceid.webders.servlet.IntegrationControllerServlet] POST: https://idg.appy.com/idp/mTgCR/resumeSAML20/idp/SSO.ping
2020-01-09 15:35:51,335 DEBUG [com.appy.jgroups.MuxInvocationHandler] invocation of retrieveAndRemoveState on InterReqStateMgmtMapImpl, state map size: 0, attributes map size: 50

```

The first event does get indexed but the second one does not ..

Now my index\_template is fairly simple:

```
PUT /_template/appy-10.0-serverlog
{
  "index_patterns": [
    "appy-srvlog-test*"
  ], 
  "settings": {
    "number_of_shards": 1, 
    "number_of_replicas": 1
  }, 
  "mappings": {

      "_source": {"enabled": true}, 
      "properties": {
      "timestamp": {
        "type": "date",
        "format":"yyyy-MM-dd HH:mm:ss,SSS"
      }, 
      "tid": {"type":"keyword"}, 
      "level": {"type": "keyword"},
      "message": {"type": "text"}

    }
  }
}

```

Is there something i am missing in my template that would allot a field if it didnt exist in the log stream not ot be indexed ? I have been reading should i be using the

`"enabled" : "false"`

or should i be looking at the kinesis side to work out why this is happening. (apologies the debugging in AWS console is a bit limited with kinesis etc)

Thank you again for any help or guidance on this.

cheers

yoyomonkey

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 2, 2020, 10:54am UTC](https://discuss.elastic.co/t/not-able-to-index-records-which-contain-missing-fields-elasticsearch-7-4/230782/2 "2020-05-02T10:54:02Z")

</div>

What do the events look like? What is the error message? Is there anything in the Elasticsearch logs?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2020, 10:54am UTC](https://discuss.elastic.co/t/not-able-to-index-records-which-contain-missing-fields-elasticsearch-7-4/230782/3 "2020-05-30T10:54:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.