The `network.protocol: http` returns nothing for packetbeat

[learningelastic]

I installed packetbeat on a server and have it publish event data to elasticsearch version 8.5 and kibana version 8.5. It seems to be working. I set up a simple Rest API on the server with packetbeat and made a few http requests to the Rest API to see if packetbeat will capture the packet data.

The API works fine and I can see that packetbeat is detecting traffic to the rest api on destination.port: 80 as shown by this image:

However, the query network.protocol: http returns nothing. Why does network.protocol: http return nothing? Many of the dashboards/visualizations depend on the query network.protocol: http.

Here is my /etc/packetbeat/packetbeat.yml

packetbeat.interfaces.device: any
packetbeat.interfaces.poll_default_route: 1m
packetbeat.interfaces.internal_networks:
  - private
packetbeat.flows:
  timeout: 30s
  period: 10s
packetbeat.protocols:
- type: icmp
  enabled: true
- type: dhcpv4
  ports: [67, 68]
- type: dns
  ports: [53]
- type: http
  ports: [80, 8080, 8000, 5000, 8002]
  enabled: true
- type: nfs
  ports: [2049]
- type: tls
  ports:
    - 443   # HTTPS
    - 993   # IMAPS
    - 995   # POP3S
    - 5223  # XMPP over SSL
    - 8443
    - 8883  # Secure MQTT
    - 9243  # Elasticsearch
- type: sip
  ports: [5060]
setup.template.settings:
  index.number_of_shards: 1
setup.dashboards.enabled: true
setup.kibana:
  host: "https://kibana.example.com:5601"
output.elasticsearch:
  hosts: ["elastic.example.com:9200"]
  protocol: "https"
  api_key: "${ES_API_KEY}"
processors:
  - # Add forwarded to tags when processing data from a network tap or mirror.
    if.contains.tags: forwarded
    then:
      - drop_fields:
          fields: [host]
    else:
      - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - detect_mime_type:
      field: http.request.body.content
      target: http.request.mime_type
  - detect_mime_type:
      field: http.response.body.content
      target: http.response.mime_type

Why does network.protocol: http return nothing?

---

Incase it matters, here is the golang code for my rest api

package main

import (
        "fmt"
        "net/http"
        "github.com/gorilla/mux"
)

func main() {
        router := mux.NewRouter()
        router.HandleFunc("/", homePage)
        http.ListenAndServe(":80", router)

}
func homePage(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Welcome to the HomePage!")
}

[stephenb (Stephen Brown)]

Hi @learningelastic

First I suspect are you running packetbeat and the webserver and your client all the the same server...

The most important is the client and the webserver on the same server..

I tested all this with the following on a Mac with Elastic Stack in including packetbeat 8.5.3

Here is the issue I believe ... and I tested and saw the same behavior.

If you are testing the HTTP traffic between a client and the http server on port 80 on the same server the traffic never actually goes through the interface that packetbeat is monitoring because the OS is smart enough to know the traffic is "Loop Back" or 127.0.0.1 so the HTTP traffic will not show up ... the http traffic will only be monitored IF it actually passes through the interface being monitored.

So in order to test...

1. you can test by setting the device to loopback

packetbeat.interfaces.device: lo0

And now I hit my local Kibana which is http Traffic on 5601 and I see the http Traffic... in fact I also see the packebeat talking HTTP to elasticsearch.

2. Now I set the interface to my actual network interface.
   packetbeat.interfaces.device: en0
   Which happens to be 192.168.86.29

And even if I use the Server's IP Address so hit Kibana
http://192.168.86.29:5601

it still does not show it was the the OS recognizes it as loopback and therefore it does not pass through the interface! (Yess Grrrrr)

BUT as soon as I access my Kibana from my other Laptop and thus the traffic actually passes through the interface!!! Whalluh! it shows up!

And the Dashboard

[learningelastic]

Thanks for the reply Stephenb

My server set up is:

• packetbeat and the web server on the same server in Australia with IP address 194.195.120.250
• i have a computer in my office in North America. Let's say my office has a public IPv4 address is 1.1.1.1. The computer uses promox hypervisor to spin up elastic server on a local ip address of 192.168.0.33 and kibana server on a local ip address of 192.168.0.34
• I have another computer in my office 1.1.1.1 using Windows and Firefox with local ip address of 192.168.0.10, which I use as my client to ping the http://194.195.120.250 url

I tried changing packetbeat.interfaces.device: eth0 to use eth0 in my /etc/packetbeat/packetbeat.yml. I then systemctl restart packetbeat

Doing a query for destination.port: 80 did yield results like this:

(assume the black box shows 1.1.1.1 because i redacted the ip address of my office)

But doing a query for network.protocol: http still does not yield any results.

Let me know if I misunderstood something?

[learningelastic]

Confirmed!!!! This was not an issue with packetbeat! It had something to do with my router/dhcp server/port forwarding/traffic forwarding or something else in my office network.

I tried spinning up a new VM in my office with 192.168.0.35 and put packetbeat and a webserver on it. When I ping my web server from 192.168.0.10 with my Firefox browser, the kibana query network.protocol: http now shows results.

I lost so much time on this, but your answer pointed me in the right direction, thank you!