Hey there,
Our team would like to make use of Kinesis record aggregation to save on AWS costs (high throughput of <1KiB size logs), but it appears FunctionBeat does not currently handle the Kinesis aggregate record format. I think this will be a desirable feature for others as using the aggregate format seems to be a common way to improve throughput and costs with Kinesis adoption.
I have implemented the needed changes and unit tests in a local fork. I haven't yet done full integration testing in an AWS account (may need to ping our company's cloud team for assistance there due to some guardrails).
The only potential implementation issue is the only available Golang Kinesis deaggregation implementation is using AWS SDK v1, so it would be pulled in as a new Go mod dependency. If that is an issue, it may require working with the AWS maintainers to refactor their Go module to have an AWS SDK v2 flavor.
Let me know how to proceed. If my current solution is acceptable, I'm happy to open an issue and PR on GitHub.
Thanks!
Joey