Hello Kaiyan,
Thank you very much for your reply.
I was first looking in the ECK documentation and saw the bits on the Elastic Agent and it still being in it's early stages. So decided to not pursue that. Hence my thought of needing to use an EC2 instance as harvest system.
The use case I have is somewhat more complex (I think).
The plan is to pull in CloudWatch logs and the sort from multiple AWS accounts, and place it all in one Elasticsearch cluster - the one I have provisioned with ECK.
For compliance reasons I cannot make use of an AWS user access key and token - the valid way would be using an IAM role.
Hence I am thinking an EC2 instance would need to be deployed to each AWS account that can pull the relevant log and metric info. The EC2 instance in this case would have the correct IAM profile attached to be able to read from those sources (e.g. no user key id and token needed).
As I currently only have experience with the 'classic' ELK/Elastic stack, it's unclear to me what Elastic agent configurations should look like.
Would be super good to see some configuration snippets that work with IAM roles only as means to authenticate.
Thanks,
Willem