How to view cloudwatch log in Kibana after deploying via functionbeat

Elastic Stack Beats
1

I am testing how to stream to elastic service cloudwatch log via lambda function.
Deployment seem to be fine but i couldn't create(define) index pattern.
Please let me know what was problem of my tested configuration(functionbeat.yml).

You can find the related ticket here.

All was commented out for Kibana,Elasticsearch Output and Logstash Output because using Elastic Service.

# =============================== Elastic Cloud 
# You can find the `cloud.id` in the Elastic Cloud web UI.
cloud.id: "DEMO:MY ID"
cloud.auth: "elastic:MY Passward"

#==================== Elasticsearch template setting 
setup.template.name: "functionbeat"
setup.template.pattern: "functionbeat-*"
setup.template.settings:
index.number_of_shards: 1

# ================================= Processors 
processors:
   - add_host_metadata: ~
   - add_cloud_metadata: ~

# ================================== Logging 
logging.to_files: true
logging.files:

# ./functionbeat test config -e

[root@ip-172-31-32-236 functionbeat-7.8.1-linux-x86_64]# ./functionbeat test config -e
<snip>
INFO    instance/beat.g o:310    Setup Beat: functionbeat; Version: 7.8.1
INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index to 'functionbeat-7.8.1' as ILM is enabled.
INFO    eslegclient/connection.go:99    elasticsearch url: https://c215xxxxxxxxxxx.ap-northeast-1.aws.found.io:443
INFO    [publisher]     pipeline/module.go:113  Beat name: ip-172-31-32-236.us-east-2.compute.internal

Config OK

The above was referred to below your link.
https://www.elastic.co/guide/en/beats/functionbeat/7.8/functionbeat-reference-yml.html

2

You should be able to see the logs on the Discovery page of Kibana.

If you cannot find it there, could you please share the debug logs of the Functionbeat Lambda deployed on AWS?

3

Hi, Noémi Ványi

Thank you always for your kind support.
I figured this issue out via below implementation.

- Create Trail
	Trail name : <My Trail Name>
	
	Data events (S3) : 
	Add S3 bucket : <Select the Created S3 bucket>
	
	Data events (Lambda) : 
	Add S3 bucket : <Select the Created Lambda function>

	Storage location
	Create a new S3 bucket : Check w/ No
	S3 bucket : <Select the Created S3 bucket>
	
- CloudWatch Logs(Enable after Creating Trail)
	Log group : <Input your created log group> 
	ex) /aws/lambda/my-lambda-function

	IAM role : <Select my IAM role>

After above setting,
I could find to add new log events in Lambda (cloudwatch) Subscripted log group and find log events in auto-created log group by functionbeat deploy.
As a result, I can create an index pattern on Elasticsearch service(Kibana).

But, Unfortunately
I met another issue on Kibana Descovery... :thinking: :thinking:
"No results match your search criteria"

I will create new discuss ticket about this, I would like to ask for your continued support.

Thanks and regards

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.