Good day. I have been working with Functionbeat for the past couple weeks. I was finally able to get logs to my self hosted ELK but I had to modify the IAM role Functionbeat created by adding the following policies:
- AWSLambdaBasicExecutionRole
- AWSLambdaVPCAccessExecutionRole
Once added I was then able to add the VPC setting to the created cloudwatch lambda and I had logs showing up in Kibana. Yay!
I would really like to not have to take the manual steps to get this working. Looking further into functionbeat.yml I see that we have the options:
- role:
- virtual_private_cloud:
Oh goodie I thought, I can create the role with the needed permissions. I created a policy with the below:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogStream",
"Logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:us-east-1:<AccountNumber>:log-group:/aws/lambda/<beatName>:*"
],
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
The above is the combination of the policies I had to add to the previously created lambda to be able to get logs to my hosted ELK.
Using the above policy on a created Role and using said role in functionbeat.yml I am not getting the same results.
functionbeat-6.7.1-linux-x86_64/functionbeat.yml
functionbeat.provider.aws.deploy_bucket: "functionbeat-deploy"
functionbeat.provider.aws.functions:
- name: cw-test-application
enabled: true
type: cloudwatch_logs
description: "lambda function for cloudwatch logs"
role: arn:aws:iam::<AccountNumber>:role/<createdLambdaRole>
virtual_private_cloud:
security_group_ids: ["sg-XXXXXXXXXXX"]
subnet_ids: ["subnet-XXXXXXXXXXX", "subnet-XXXXXXXXXXX"]
triggers:
- log_group_name: dev-application
output.elasticsearch:
hosts: ["10.X.X.X:9200"]
The security group and subnets are what I have working on Functionbeat created Lambda so I know they are good. What do I need to do to make this work without the manual steps? Better yet does anyone have this in terraform yet (I know, asking alot here)?
Thank you in advance.