# Duplication data in the event

**URL:** <https://discuss.elastic.co/t/duplication-data-in-the-event/289567>\
**Category:** Elasticsearch\
**Created:** [November 18, 2021, 10:09am UTC](https://discuss.elastic.co/t/duplication-data-in-the-event/289567 "2021-11-18T10:09:41Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![EugeneZ](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eugenez/32/78447_2.png) [@EugeneZ](https://discuss.elastic.co/u/EugeneZ)\
**Post date:** [November 18, 2021, 10:09am UTC](https://discuss.elastic.co/t/duplication-data-in-the-event/289567/1 "2021-11-18T10:09:41Z")

</div>

Hi guys  
I have some log from filebeat--logstash--\> elastic  
I see logs in kibana and it is strange  
in JSON for example:

```auto
{
  "_index": "test-fbt-2021.11",
  "_type": "_doc",
  "_id": "dvMV_3wBOQF85F2Xl-Y4",
  "_score": 1,
  "_source": {
    "endOfBatch": false,
    "agent": {
      "ephemeral_id": "44437e45-9644-4524-b63a-2d708bc20cee",
      "version": "7.13.2",
      "hostname": "c-t-sandbox-k8s-wrk",
      "name": "c-t-sandbox-k8s-wrk",
      "type": "filebeat",
      "id": "68056566-714e-4566-9108-5713735a413f"
    },
    "ecs": {
      "version": "1.8.0"
    },
    
  },
  "fields": {
    "agent.version.keyword": [
      "7.13.2"
    ],
    "endOfBatch": [
      false
    ],
    "agent.hostname.keyword": [
      "c-t-sandbox-k8s-wrk"
    ],
    "ecs.version.keyword": [
      "1.8.0"
    ],
    "agent.name": [
      "c-t-sandbox-k8s-wrk"
    ],
    "agent.id.keyword": [
      "68056566-714e-4566-9108-5713735a413f"
    ],
    "agent.hostname": [
      "c-t-sandbox-k8s-wrk"
    ],
    "agent.id": [
      "68056566-714e-4566-9108-5713735a413f"
    ],
    "ecs.version": [
      "1.8.0"
    ],
    "agent.version": [
      "7.13.2"
    ],
    "agent.type": [
      "filebeat"
    ],
    "agent.type.keyword": [
      "filebeat"
    ],
    "agent.ephemeral_id.keyword": [
      "44437e45-9644-4524-b63a-2d708bc20cee"
    ],
    "agent.name.keyword": [
      "c-t-sandbox-k8s-wrk"
    ],
   "agent.ephemeral_id": [
      "44437e45-9644-4524-b63a-2d708bc20cee"
    ],
  }
}

```

data duplicate, index size bigger  
I check filebeat and logstash it's OK

Please help. Can I drop this duplicate field "fields"?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 16, 2021, 10:10am UTC](https://discuss.elastic.co/t/duplication-data-in-the-event/289567/2 "2021-12-16T10:10:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.