onDropConnection

Elastic Stack Beats
1

Hi,
I'm trying to implement a protocol using the generator and I have some doubts about how the onDropConnection function works.
What I'm trying to achieve is, even if I don't get a response back, I want to publish the transaction.

Thanks in advance,
Duarte

2

The generator is mostly a sample skeleton and you're encouraged to adapt it to your requirements. The onDropConnection method is called, if a TCPs connection application layer state is to be dropped by the protocol plugin due to a parsing error or GapInStream. GapInStream will be called by TCP layer in case of TCP layer detecting a packet loss. For GapInStream some different strategies are possible (dependent on protocol and parser state):

  • ignore the gap if some content(body) is affected, which will not be parsed anyway
  • If connection has some ongoing transaction, try to publish incomplete transaction with warning + drop state
  • Just drop state (default in code generator).

The parser error is more severe, as connection drop due to parser error can happen for a few reasons:

  • incorrect protocol being feed to parser (e.g. wrong port configured) => in this case normally no message can be parsed and connection has no active transaction
  • parser state is not correctly in sync (e.g. packetbeat was started after TCP connection was established). If parser is not in sync with underlying stream, we have to drop state and retry parsing. Alternatively one can try to find a common pattern to sync-up (some protocols like Diameter can be handled better this way), but normally just trying to parse and drop state on error works well enough (unfortunately this means one has to harden the parser from all potential parsing errors)
  • worst case: an actual bug in parser

You are very well encouraged to handle onDropConnection or (if possible) try to recover from GapInStream without having to drop the connection. This might be more difficult or simple, depending on the protocol analysed itself. That's why the default code generator just drops all state.

Every connection object holds the parser state for the TCP stream in each direction (stream field) + the active transaction state (trans field) used for correlating messages into transactions. Some protocols might have to correlate multiple messages in one stream direction into one transaction (e.g. data response stream split into multiple messages) and others might have to deal with pipelining (multiple requests before first response). Again other protocols might require N>2 messages for specific transaction types...

So, onDropConnection might check if there are any complete/incomplete messages in the parser states, which can potentially be published (this step is optional, as ongoing transaction might already be available in trans). Normally messages are passed to trans via trans.onMessage. Either reuse trans.onMessage or introduce trans.onIncompleteMessage if you have to.
Normally onMessage triggers the correlation steps tryMergeRequests, tryMergeResponses and correlate. correlate calls trans.onTransaction(requ, resp) upon having found a complete transaction. For example one might introduce a correlateIncomplete executed by onDropConnection in order to push incomplete transaction to the publisher. The handler for publishing complete transactions can be found in pub.go. When publishing an incomplete transaction, also add a messages to notes and consider adding a flag to your event marking the transaction as incomplete.

As you can see, handling onDropConnection might be not that trivial, depending on the protocol at hand. That's why the default onDropConnection is a no-op.

Personally I'm total in favour of publishing incomplete transactions.
I hope you don't feel discouraged by the amount of work required to handle onDropConnection. PRs to improve the generator (e.g. comments and/or code improvements) are very welcome.

Which protocol are you implementing. By having some more knowledge of the protocol, I can maybe help with ideas of how I would solve it in this particular case.

I hope this helps a little.
Steffen

3

Hi,
Thanks for the fast reply.
I think I'm understanding how things are supposed to work now.

The protocol I'm trying to parse is a very simple one. Is based on the MLLP(MLLPV1, MLLPV2) that is used by healthcare applications.
Since I'm using packetbeat with port mirror I might lose some messages in the communication but I still want to send the parts that I got.

I'll try and maybe add a PR to the generator if it works.

Thanks fot the help.
Duarte

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.