Packetbeat:How to add a new protocol?

billzy · May 26, 2016, 1:18pm

I had the same problem and after a few research of the existing http protocol, I found following steps were needed when developing a new protocol. This simple guide is based on packetbeat version 1.2.1

Import your package
In beat/packetbeat/main.go, add a line in the import section to import your protocol
for example

// Add newProtocol line in import section
import (
_ "github.com/elastic/beats/packetbeat/protos/http"
_ "github.com/elastic/beats/packetbeat/protos/memcache"
_ "github.com/elastic/beats/packetbeat/protos/mongodb"
```
  _ "github.com/elastic/beats/packetbeat/protos/newProtocol"
```
)
Create a new folder beats/packetbeat/protos/newProtocol and start writing you go file e.g. newProtocol.go
Register your plugin
in the newProtocol.go file, create method init() which will be called for initialization

// When package is imported, function init will be called
func init() {
protos.Register("newProtocol", New)
}

The Register function will register your protocol and call function New which is the second parameter, so you need to implement the function New in next step

Create method New or make a copy from http.go in http package, you can find the prototype of function New which is type ProtocolPlugin in beats/packetbeat/protos/registry.go

type ProtocolPlugin func(
testMode bool,
results publish.Transactions,
cfg *common.Config,
) (Plugin, error)

Implement plugin interface
If your protocol is based on TCP, you need to implement the TcpPlugin interface (TcpPlugin is defined in beats/packetbeat/protos/registry.go)
If your case is UDP, implement UdpPlugin

type TcpPlugin interface {
Plugin

  // Called when TCP payload data is available for parsing.
  Parse(pkt *Packet, tcptuple *common.TcpTuple,
  dir uint8, private ProtocolData) ProtocolData

  // Called when the FIN flag is seen in the TCP stream.
  ReceivedFin(tcptuple *common.TcpTuple, dir uint8,
  private ProtocolData) ProtocolData

  // Called when a packets are missing from the tcp stream.
  GapInStream(tcptuple *common.TcpTuple, dir uint8, nbytes int,
  private ProtocolData) (priv ProtocolData, drop bool)

  // ConnectionTimeout returns the per stream connection timeout.
  // Return <=0 to set default tcp module transaction timeout.
  ConnectionTimeout() time.Duration

}

type UdpPlugin interface {
Plugin

  // ParseUdp is invoked when UDP payload data is available for parsing.
  ParseUdp(pkt *Packet)

}
Hope this helps