# Elastic search showing partial data

**URL:** <https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567>\
**Category:** Elasticsearch\
**Created:** [May 3, 2019, 5:21pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567 "2019-05-03T17:21:15Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [May 3, 2019, 5:21pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/1 "2019-05-03T17:21:15Z")

</div>

Elastic search is showing partial data for my newly created index. Can you please guide how to fix it.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6cc96747b150ab585eb347901c6d9fe8a7ef5947.png)

I was getting complete results earlier.

```
{
  "_index": "heartbeat-6.7.1-2019.04.15",
  "_type": "_doc",
  "_id": "vteBIWoB5GIIEeSLwiUD",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2019-04-15T14:58:32.124Z",
    "tcp": {
      "port": 7443,
      "rtt": {
        "connect": {
          "us": 113
        }
      }
    },
    "tls": {
      "rtt": {
        "handshake": {
          "us": 12329
        }
      },
      "certificate_not_valid_before": "0001-01-01T00:00:00.000Z"
    },
    "http": {
      "rtt": {
        "validate": {
          "us": 891
        },
        "content": {
          "us": 10
        },
        "total": {
          "us": 28401
        },
        "write_request": {
          "us": 36
        },
        "response_header": {
          "us": 880
        }
      },
      "response": {
        "status_code": 200
      },
      "url": "https://changed.for.security.com:7443/arcgis/home"
    },
    "resolve": {
      "ip": "127.0.0.1",
      "rtt": {
        "us": 213
      },
      "host": "changed.for.security.com"
    },
    "event": {
      "dataset": "uptime"
    },
    "fields": {
      "version": "10.6"
    },
    "beat": {
      "version": "6.7.1",
      "name": "changed.for.security.com",
      "hostname": "changed.for.security.com"
    },
    "monitor": {
      "type": "http",
      "host": "changed.for.security.com",
      "ip": "127.0.0.1",
      "duration": {
        "us": 28718
      },
      "status": "up",
      "scheme": "https",
      "id": "http@https://changed.for.security.com:7443/arcgis/home",
      "name": "http"
    },
    "host": {
      "os": {
        "codename": "Maipo",
        "platform": "rhel",
        "version": "7.5 (Maipo)",
        "family": "",
        "name": "Red Hat Enterprise Linux Server"
      },
      "name": "changed.for.security.com",
      "id": "0709cd0acc86463da7d518c28f145a66",
      "containerized": true,
      "architecture": "x86_64"
    }
  },
  "fields": {
    "tls.certificate_not_valid_before": [
      "0001-01-01T00:00:00.000Z"
    ],
    "@timestamp": [
      "2019-04-15T14:58:32.124Z"
    ]
  },
  "sort": [
    1555340312124
  ]
}

```

Now am seeing only this data in output on kibana.

```
{
  "_index": "heartbeat-2019.05.03-000001",
  "_type": "_doc",
  "_id": "g6SyfmoB5GIIEeSLwzMn",
  "_version": 1,
  "_score": null,
  "fields": {
    "@timestamp": [
      "2019-05-03T17:16:44.700Z"
    ],
    "tls.certificate_not_valid_after": [
      "2020-09-10T12:00:00.000Z"
    ],
    "tls.certificate_not_valid_before": [
      "2017-06-26T00:00:00.000Z"
    ]
  },
  "sort": [
    1556903804700
  ]
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2019, 5:23pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/2 "2019-05-03T17:23:30Z")

</div>

It looks like you may have disabled `_source`. Have you updated any index templates lately? Can you have a look at the index templates matching this index and see if any of them might be the reason?

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [May 3, 2019, 5:42pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/3 "2019-05-03T17:42:14Z")

</div>

@Christian_Dahlqvist Yes, I was using ELK 6.7 them updated my Elastic search and kibana. Every thing was working fine until then.

But I installed beats 7 on a server and now every thing is broken.

I have enabled `_source` now but there is no change.

```
#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  _source.enabled: true

migration.6_to_7.enabled: true

```

I see this notification on kibana  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/2/0220d93e1c98a964d8f97eeec43f2436f75a22e2.png)

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2019, 6:08pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/4 "2019-05-03T18:08:38Z")

</div>

Changes to templates only take effect when new indices are created. If you are setting this through beats config I am not sure it will take effect as templates have already been loaded. You may therefore need to change the templates already loaded.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [May 3, 2019, 6:22pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/5 "2019-05-03T18:22:33Z")

</div>

@Christian_Dahlqvist Thanks for your help.

I used this command to check existing templates

`GET /_template/heartbeat*`

I found three templates.

```
    heartbeat --> this is for 7.0.0 version 
    heartbeat-7.0.0
    heartbeat-6.7.0

```

I have some of the beats 6.7 version still running. Should I only delete template related to 7.0.0 and recreate it or I have to delete all templates and create the latest?

I have deleted indices created with the latest template.

I tried to follow these instructions but I am not very clear on this.  
[https://www.elastic.co/guide/en/beats/libbeat/7.0/troubleshooting-upgrade.html#missing-fields](https://www.elastic.co/guide/en/beats/libbeat/7.0/troubleshooting-upgrade.html#missing-fields)

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [May 3, 2019, 6:32pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/6 "2019-05-03T18:32:49Z")

</div>

@Christian_Dahlqvist

I deleted the heartbeat template related to 7.0.0 which was already created and used `setup --template` to export new template. I am not seeing data in kibana.

I see this mapping conflict in index. Can you please guide me how to resolve it.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/0/d08400994c1248366cd823741c941e72faa6b849.png)

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2019, 6:34pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/7 "2019-05-03T18:34:13Z")

</div>

Have you had any new indices created since you updated the template? This change will not apply to existing indices.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [May 3, 2019, 6:36pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/8 "2019-05-03T18:36:57Z")

</div>

Yes, I have new indices created in kibana and I'm seeing data. Is the conflict because of the old existing indices which were created by prior version.

heartbeat-7.0.0-2019.05.03-000001

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 31, 2019, 6:37pm UTC](https://discuss.elastic.co/t/elastic-search-showing-partial-data/179567/9 "2019-05-31T18:37:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.