I am trying to write a mapping for a custom metric beat whose goal is to capture the number of network connections for a list of protocols. There will be certain hosts, where a certain protocol will not be supported, while on some hosts all protocols will be supported. I have a couple of options that I'm contemplating:
- Have a separate event (i.e. ES document) for each protocol. In case a host doesn't support that protocol, don't log anything to ES for that protocol.
- Having a single document for all protocols, with # of connections for each. In case the protocol is not supported on this host, the value will be 0.
- Have the same structure as (2), however, in case a protocol is not supported on the host, simply remove that key-value pair, instead of adding a 0 value. That is, there will be only one event for all the protocols, but it will only have data for the protocols that this hosts supports. Eventually, there will be variance in the documents across hosts.
The issue with 1 is that it uses more storage, since other meta data in the event document will get duplicated for each event.
Between 2 and 3, I'm trying to understand if there are any performance implications, when doing queries with Kibana. (2) will use more storage, hence 3 seems the most efficient.