# Packetbeat Flows and Kubernetes not working fine

**URL:** <https://discuss.elastic.co/t/packetbeat-flows-and-kubernetes-not-working-fine/149725>\
**Category:** Beats\
**Tags:** packetbeat\
**Created:** [September 24, 2018, 8:00pm UTC](https://discuss.elastic.co/t/packetbeat-flows-and-kubernetes-not-working-fine/149725 "2018-09-24T20:00:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rikatz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rikatz/32/35825_2.png) [@rikatz](https://discuss.elastic.co/u/rikatz)\
**Post date:** [September 24, 2018, 8:00pm UTC](https://discuss.elastic.co/t/packetbeat-flows-and-kubernetes-not-working-fine/149725/1 "2018-09-24T20:00:38Z")

</div>

I've been trying to use Packetbeat to map my Kubernetes Cluster traffic without success.

The idea here is to use Packetbeat to map all the flows from the Cluster and store them into Elasticsearch, enriching those events with Kubernetes PODs Metadata (namespace and name, mainly).

But when trying to do this, no metadata is inserted. The following is my packetbeat.yml config:

```auto
packetbeat.interfaces.device: any
output.elasticsearch:
  hosts: ["REDACTED:9200"]
setup.kibana:
  host: "REDACTED:5601"
packetbeat.interfaces.device: any
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 100
packetbeat.flows:
  timeout: 30s
  period: 10s
  enabled: true
  processors:
  - add_kubernetes_metadata:
      host: REDACTED
      in_cluster: false
      kube_config: /etc/kubernetes/node-kubeconfig.yaml

```

When trying to start packetbeat, I get the following messages:

```auto
|2018-09-24T16:53:35.937-0300|INFO|kubernetes/watcher.go:180|kubernetes: Performing a resource sync for *v1.PodList|
|---|---|---|---|
|2018-09-24T16:53:35.955-0300|INFO|kubernetes/watcher.go:194|kubernetes: Resource sync done|
|2018-09-24T16:53:35.955-0300|INFO|kubernetes/watcher.go:238|kubernetes: Watching API for resource events|

```

But the event only stand as the following JSON (no Metadata):

```auto
{
  "_index": "packetbeat-6.4.1-2018.09.24",
  "_type": "doc",
  "_id": "UiAoDWYB5pf0lEE4guaU",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2018-09-24T19:57:30.000Z",
    "transport": "tcp",
    "source": {
      "ip": "10.79.REDACTED",
      "port": 56386,
      "stats": {
        "net_bytes_total": 296,
        "net_packets_total": 4
      },
      "mac": "72:dd:61:ad:eb:67"
    },
    "flow_id": "EQQA////DP//////FP8BAAFy3WGt62fu7u7u7u4KTxiqrBDYlkLc0gQ",
    "beat": {
      "version": "6.4.1",
      "name": "REDACTED",
      "hostname": "REDACTED"
    },
    "final": false,
    "dest": {
      "mac": "ee:ee:ee:ee:ee:ee",
      "ip": "172.16.216.150",
      "port": 1234
    },
    "start_time": "2018-09-24T19:57:18.090Z",
    "last_time": "2018-09-24T19:57:25.218Z",
    "type": "flow",
    "host": {
      "name": "REDACTED"
    }
  },
  "fields": {
    "start_time": [
      "2018-09-24T19:57:18.090Z"
    ],
    "@timestamp": [
      "2018-09-24T19:57:30.000Z"
    ],
    "last_time": [
      "2018-09-24T19:57:25.218Z"
    ]
  },
  "highlight": {
    "dest.port": [
      "@kibana-highlighted-field@1234@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1537819050000
  ]
}

```

Any idea how to improve this?

EDIT: Got the following message when enabling Debug "kubernetes":

```auto
2018-09-24T17:13:10.007-0300	DEBUG	[kubernetes]	add_kubernetes_metadata/matchers.go:167	Unable to apply field format pattern on event

```

Thanks!

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [October 1, 2018, 10:24am UTC](https://discuss.elastic.co/t/packetbeat-flows-and-kubernetes-not-working-fine/149725/2 "2018-10-01T10:24:52Z")

</div>

Hi @rikatz,

Packetbeat is configured to enrich events based on `ip` & `port` fields. I see in your event that you would be interested on using `source.ip` and `source.port` this time. Probably something like this could work:

```auto
  - add_kubernetes_metadata:
      host: REDACTED
      in_cluster: false
      kube_config: /etc/kubernetes/node-kubeconfig.yaml
      matchers:
        - field_format:
            format: '%{[source.ip]}:%{[source.port]}'

```

(Take into account that ports need to be declared in the Pods so Packetbeat can enrich based on that info)

Let us know how it goes! we may consider adding these as defaults

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2018, 10:24am UTC](https://discuss.elastic.co/t/packetbeat-flows-and-kubernetes-not-working-fine/149725/3 "2018-10-29T10:24:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.