# Packetbeat- Duplicated flow records even intermediate report is disabled

**URL:** <https://discuss.elastic.co/t/packetbeat-duplicated-flow-records-even-intermediate-report-is-disabled/251107>\
**Category:** Beats\
**Tags:** packetbeat\
**Created:** [October 6, 2020, 11:12am UTC](https://discuss.elastic.co/t/packetbeat-duplicated-flow-records-even-intermediate-report-is-disabled/251107 "2020-10-06T11:12:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![wphromma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wphromma/32/76715_2.png) [@wphromma](https://discuss.elastic.co/u/wphromma)\
**Post date:** [October 6, 2020, 11:12am UTC](https://discuss.elastic.co/t/packetbeat-duplicated-flow-records-even-intermediate-report-is-disabled/251107/1 "2020-10-06T11:12:19Z")

</div>

Hi,  
I am just curious why we are getting duplicated logs of a flow.

 ![Screen Shot 2563-10-06 at 16.59.59](https://us1.discourse-cdn.com/elastic/original/3X/f/7/f76510512c5d22bead3e1c307659ea1f78ea9ef9.png)

As I understand, the flow id should be uniqued and when `flow.final` is to true, doesn't that mean there should be no more data for this flow?

This is the configuration

```auto
# ================================== General ===================================
fields.dc: {{ dc }}
fields_under_root: true

# =============================== Network device ===============================
packetbeat.interfaces.device: any
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.ignore_outgoing: true
packetbeat.interfaces.buffer_size_mb: 100
packetbeat.interfaces.bpf_filter: "tcp port 8500 or tcp port 8300"

# =================================== Flows ====================================
packetbeat.flows:
  timeout: 30s
  period: -1s

# =================================== Internal queue ====================================
# queue:
# mem:
# events: 4096
# flush.min_events: 2048
# flush.timeout: 0s

# =========================== Transaction protocols ============================
packetbeat.protocols:
  - type: http
    ports: [8500]

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  enabled: true
  hosts: {{elasticsearch_hosts}}
  index: "unsafe-it-devops-services-consul-packetbeat-{{dc}}-%{+yyyy.MM.dd}"
  #bulk_max_size: 100

# ======================= Elasticsearch template setting =======================
setup.template.name: "unsafe-it-devops-services-consul-packetbeat"
setup.template.pattern: "unsafe-it-devops-services-consul-packetbeat-*"
setup.template.settings:
    index.number_of_shards: 1

# ====================== Index Lifecycle Management (ILM) ======================
setup.ilm.enabled: false

# ================================== Logging ===================================
logging.level: info
logging.to_files: true
logging.to_syslog: false
logging.files:
  path: /var/log/packetbeat
  name: packetbeat
  keepfiles: 10
  permissions: 0644

# ================================= Processors =================================
processors:
   - include_fields:
      fields: ["dc", "destination.ip", "flow.final", "flow.id", "destination.port", "http.request.method", "http.response.status_code", "source.ip", "url.path"]
   - drop_event:
        when: 
          or:
            - and:
              - not.equals:
                  destination.port: 8300
              - not.equals:
                  destination.port: 8500
            - and: 
              - equals:
                  type: flow
              - equals:
                  destination.port: 8500
            - and:
              - equals:
                  typs: flow
              - equals:
                  destination.port: 8300
              - not.equals:
                  destination.ip: "{{ ansible_default_ipv4.address }}"

```

I put `flow.period to -1s` to disable intermediate report

Best regards,  
Wathanyu

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 3, 2020, 1:12pm UTC](https://discuss.elastic.co/t/packetbeat-duplicated-flow-records-even-intermediate-report-is-disabled/251107/2 "2020-11-03T13:12:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.