With functionbeat-kinesis, I have been trying to steam AWS Kinese Data stream to my locally deployed Elastic.
All set-up seems okay. the functionbeat lambda deployed and I don't see any error from the functionbeat lambda.
I created a Foo Kinesis data stream, but the functionbeat is not picking up the data stream. I put number of records into the kinesis stream and uisng AWS CLI, I validated the data is in the stream.
I can't find any reason the functionbeat is not ingesting the kiensis data stream.
Please see my configuration of the functionbeat.yaml.
functionbeat.provider.aws.endpoint: "s3.amazonaws.com" functionbeat.provider.aws.deploy_bucket: "mys3bucket" functionbeat.provider.aws.functions: - name: kinesisfbfoo enabled: true type: kinesis triggers: - event_source_arn: arn:aws:kinesis:us-east-1:xxxxxxxxxxxx:stream/Foo xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.username: beats_system xpack.monitoring.elasticsearch.password: mystrongsecret output.elasticsearch: hosts: ["x.x.x.x"] username: "elastic" password: "mystrongsecret" setup.kibana: host: "localhost"
Thank you for your advice or suggestion in advance.