How to implement tcp state transition information with metricbeat

[st1t (Shota Ito)]

Hello everyone
I'd like to display TCP state transitions on metricbeat using "/proc/net/tcp" on Linux. (Established, Listen, SYN-SENT ... etc.)

• example

•   "system": {
        "network": {
            "in": {
                "bytes": 0,
                "dropped": 0,
                "errors": 0,
                "packets": 0
            },
            "name": "sit0",
            "out": {
                "bytes": 0,
                "dropped": 0,
                "errors": 0,
                "packets": 0
            },
            "tcp": {
                "established": 110,
                "syn_sent": 3,
                "syn_recv": 0,
                "fin_wait1": 0,
                "fin_wait2": 0,
                "time_wait": 0,
                "close": 0,
                "close_wait": 0,
                "last_ack": 3,
                "listen": 10,
                "closing": 0
            }
        }
    }
  

Should I implement the function as a network package of system module if I implement it?
Or should you add it as a tcpstat module?
Also, could you tell me if there is any other good way?

[ruflin]

Interesting question. Looking at /proc/net there is a lot more stuff that could be potentially added in the future. Directly related to this would be /proc/net/udp and /proc/net/icmp.

As I assume most users will need the in/out stats I would opt for an additional metricset. This is like the save bet for the moment. I would be more then happy if you could open a PR with this.

An other option could be to have a config option network.tcp to enable it but we should discuss this if we have too many separate metricsets.

[andrewkroh]

So if I understand correctly you want to summarize the TCP connection table to the number of sockets that are in each TCP state? And the values reported will be gauges (they go up and down)?

The data in the system network metricset is sent per interface. So you could only add this new data to the system network metricset if you are able to associate each of the sockets in the connection table to an interface. I don't think this is possible. Therefore this will need to go into its own metricset and now we have a naming problem.

The system socket metricset reads the TCP connection table. You could have a look at it's implementation to see what it does. It does not read from /proc/self/net/tcp because that is less efficient. I would probably group the stats into tcp and tcp6 categories.

[st1t (Shota Ito)]

Hello, @ruflin , @andrewkroh
Thank you for your early reply!

I'm sorry, I misunderstood the information in /proc/self/net/tcp. . .

I saw the socket module.
Information that I wanted can be done by enabling the socket module and aggregating it on Elasticsearch side.
If I can not do it well please let me discuss again.

Thank you for a wonderful module!

[andrewkroh]

The socket module won't allow you to get the information you need because it does not report the current state of the socket. It reports the socket information only the first time it is observed and therefore it doesn't make a lot of sense to report the TCP state.

So I think you will need a new MetricSet to summarize the TCP states and send an event.