I am trying to setup AWS integration with Elasticsearch 7.17.3 and have had some success. I am not able to get data for few services and the filebeat.json log shows the below error.
I am seeing the region code twice in the logs , have rechecked few times and not able to figure out where its coming from. Could this be a bug or i am not setting this up right?
{"log.level":"error","@timestamp":"2022-05-20T21:38:42.995+1000","log.logger":"aws-cloudwatch","log.origin":{"file.name":"awscloudwatch/input.go","file.line":171},"message":"getLogEventsFromCl
oudWatch failed: error FilterLogEvents with Paginator: exceeded maximum number of attempts, 3, request send failed, Post \"https://logs.ap-southeast-2.ap-southeast-2.console.aws.amazon.com/\":
dial tcp: lookup logs.ap-southeast-2.ap-southeast-2.console.aws.amazon.com on 1xxxxxxx:53: no such host","service.name":"filebeat","ecs.version":"1.6.0"}
Can u post ur configuration?
1 Like
zmoog
May 20, 2022, 1:38pm
3
As @legoguy1000 already said, the configuration file (with secrets or sensitive information redacted) is probably essential to understand what's going on.
From your error message, I see that the hostname logs.ap-southeast-2.ap-southeast-2.console.aws.amazon.com does not exist.
According to the Amazon CloudWatch Logs endpoints and quotas page, the endpoint for the region ap-southeast-2 is a little different: have you customized the endpoint?
1 Like
You were right, the endpoint was incorrect ( i corrected it, however still see same duplicacy in logs )
id: 2016d7cc-135e-5583-9758-3ba01f5a06e5
revision: 32
outputs:
default:
type: elasticsearch
hosts:
- 'http://localhost:9200'
username: '{ES_USERNAME}'
password: '{ES_PASSWORD}'
output_permissions:
default:
_elastic_agent_monitoring:
indices:
- names:
- logs-elastic_agent.metricbeat-default
privileges: &ref_0
- auto_configure
- create_doc
- names:
- logs-elastic_agent.fleet_server-default
privileges: *ref_0
- names:
- metrics-elastic_agent.fleet_server-default
privileges: *ref_0
- names:
- logs-elastic_agent.auditbeat-default
privileges: *ref_0
- names:
- metrics-elastic_agent.heartbeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.filebeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.endpoint_security-default
privileges: *ref_0
- names:
- metrics-elastic_agent.endpoint_security-default
privileges: *ref_0
- names:
- metrics-elastic_agent.metricbeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.packetbeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.heartbeat-default
privileges: *ref_0
- names:
- logs-elastic_agent-default
privileges: *ref_0
- names:
- metrics-elastic_agent.osquerybeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.osquerybeat-default
privileges: *ref_0
- names:
- metrics-elastic_agent.packetbeat-default
privileges: *ref_0
- names:
- metrics-elastic_agent.filebeat-default
privileges: *ref_0
- names:
- metrics-elastic_agent.auditbeat-default
privileges: *ref_0
- names:
- logs-elastic_agent.apm_server-default
privileges: *ref_0
- names:
- metrics-elastic_agent.apm_server-default
privileges: *ref_0
- names:
- metrics-elastic_agent.elastic_agent-default
privileges: *ref_0
_elastic_agent_checks:
cluster:
- monitor
system-1:
indices:
- names:
- logs-system.auth-default
privileges: *ref_0
- names:
- logs-system.syslog-default
privileges: *ref_0
- names:
- logs-system.application-default
privileges: *ref_0
- names:
- logs-system.security-default
privileges: *ref_0
- names:
- logs-system.system-default
privileges: *ref_0
- names:
- metrics-system.cpu-default
privileges: *ref_0
- names:
- metrics-system.diskio-default
privileges: *ref_0
- names:
- metrics-system.filesystem-default
privileges: *ref_0
- names:
- metrics-system.fsstat-default
privileges: *ref_0
- names:
- metrics-system.load-default
privileges: *ref_0
- names:
- metrics-system.memory-default
privileges: *ref_0
- names:
- metrics-system.network-default
privileges: *ref_0
- names:
- metrics-system.process-default
privileges: *ref_0
- names:
- metrics-system.process.summary-default
privileges: *ref_0
- names:
- metrics-system.socket_summary-default
privileges: *ref_0
- names:
- metrics-system.uptime-default
privileges: *ref_0
POC-syd-aws-integration:
indices:
- names:
- logs-aws.cloudtrail-default
privileges: *ref_0
- names:
- logs-aws.cloudwatch_logs-default
privileges: *ref_0
- names:
- metrics-aws.cloudwatch_metrics-default
privileges: *ref_0
- names:
- metrics-aws.ebs-default
privileges: *ref_0
- names:
- logs-aws.ec2_logs-default
privileges: *ref_0
- names:
- metrics-aws.ec2_metrics-default
privileges: *ref_0
- names:
- metrics-aws.s3_daily_storage-default
privileges: *ref_0
- names:
- metrics-aws.s3_request-default
privileges: *ref_0
- names:
- metrics-aws.s3_storage_lens-default
privileges: *ref_0
- names:
- metrics-aws.sns-default
privileges: *ref_0
- names:
- metrics-aws.transitgateway-default
privileges: *ref_0
- names:
- metrics-aws.usage-default
privileges: *ref_0
- names:
- logs-aws.vpcflow-default
privileges: *ref_0
agent:
monitoring:
enabled: true
use_output: default
namespace: default
logs: true
metrics: true
inputs:
- id: logfile-system-default-system-policy
name: system-1
revision: 1
type: logfile
use_output: default
meta:
package:
name: system
version: 1.11.0
data_stream:
namespace: default
streams:
- id: logfile-system.auth-default-system-policy
data_stream:
dataset: system.auth
type: logs
paths:
- /var/log/auth.log*
- /var/log/secure*
exclude_files:
- .gz$
multiline:
pattern: ^\s
match: after
processors:
- add_locale: null
- id: logfile-system.syslog-default-system-policy
data_stream:
dataset: system.syslog
type: logs
paths:
- /var/log/messages*
- /var/log/syslog*
exclude_files:
- .gz$
multiline:
pattern: ^\s
match: after
processors:
- add_locale: null
- id: winlog-system-default-system-policy
name: system-1
revision: 1
type: winlog
use_output: default
meta:
package:
name: system
version: 1.11.0
data_stream:
namespace: default
streams:
- id: winlog-system.application-default-system-policy
name: Application
data_stream:
dataset: system.application
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.security-default-system-policy
name: Security
data_stream:
dataset: system.security
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: winlog-system.system-default-system-policy
name: System
data_stream:
dataset: system.system
type: logs
condition: '${host.platform} == ''windows'''
ignore_older: 72h
- id: system/metrics-system-default-system-policy
name: system-1
revision: 1
type: system/metrics
use_output: default
meta:
package:
name: system
version: 1.11.0
data_stream:
namespace: default
streams:
- id: system/metrics-system.cpu-default-system-policy
data_stream:
dataset: system.cpu
type: metrics
metricsets:
- cpu
cpu.metrics:
- percentages
- normalized_percentages
period: 10s
- id: system/metrics-system.diskio-default-system-policy
data_stream:
dataset: system.diskio
type: metrics
metricsets:
- diskio
diskio.include_devices: null
period: 10s
- id: system/metrics-system.filesystem-default-system-policy
data_stream:
dataset: system.filesystem
type: metrics
metricsets:
- filesystem
period: 1m
processors:
- drop_event.when.regexp:
system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.fsstat-default-system-policy
data_stream:
dataset: system.fsstat
type: metrics
metricsets:
- fsstat
period: 1m
processors:
- drop_event.when.regexp:
system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
- id: system/metrics-system.load-default-system-policy
data_stream:
dataset: system.load
type: metrics
metricsets:
- load
condition: '${host.platform} != ''windows'''
period: 10s
- id: system/metrics-system.memory-default-system-policy
data_stream:
dataset: system.memory
type: metrics
metricsets:
- memory
period: 10s
- id: system/metrics-system.network-default-system-policy
data_stream:
dataset: system.network
type: metrics
metricsets:
- network
period: 10s
network.interfaces: null
- id: system/metrics-system.process-default-system-policy
data_stream:
dataset: system.process
type: metrics
metricsets:
- process
period: 10s
process.include_top_n.by_cpu: 5
process.include_top_n.by_memory: 5
process.cmdline.cache.enabled: true
process.cgroups.enabled: false
process.include_cpu_ticks: false
processes:
- .*
- id: system/metrics-system.process.summary-default-system-policy
data_stream:
dataset: system.process.summary
type: metrics
metricsets:
- process_summary
period: 10s
- id: system/metrics-system.socket_summary-default-system-policy
data_stream:
dataset: system.socket_summary
type: metrics
metricsets:
- socket_summary
period: 10s
- id: system/metrics-system.uptime-default-system-policy
data_stream:
dataset: system.uptime
type: metrics
metricsets:
- uptime
period: 10s
- id: aws-cloudwatch-cloudtrail-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws-cloudwatch
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws-cloudwatch-aws.cloudtrail-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.cloudtrail
type: logs
log_group_arn: >-
arn:aws:logs:ap-southeast-2:757008298094:log-group:aws-cloudtrail-logs-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
start_position: end
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
api_sleep: 200ms
endpoint: logs.ap-southeast-2.amazonaws.com
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
scan_frequency: 3m
tags:
- forwarded
- aws-cloudtrail
publisher_pipeline.disable_host: true
- id: aws-cloudwatch-cloudwatch-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws-cloudwatch
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: >-
aws-cloudwatch-aws.cloudwatch_logs-baef3b24-99ae-48f0-b48a-a63bdbc82612
dataset: generic
data_stream: null
log_group_arn: >-
arn:aws:logs:ap-southeast-2:757008298094:log-group:aws-cloudtrail-logs-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
start_position: end
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
api_sleep: 200ms
endpoint: logs.ap-southeast-2.amazonaws.com
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
scan_frequency: 1m
tags:
- forwarded
- aws-cloudwatch-logs
publisher_pipeline.disable_host: true
- id: aws/metrics-cloudwatch-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: >-
aws/metrics-aws.cloudwatch_metrics-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.cloudwatch_metrics
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 300s
metricsets:
- cloudwatch
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
metrics:
- name:
- CPUUtilization
- DiskWriteOps
statistic:
- Average
- Maximum
namespace: AWS/EC2
resource_type: 'ec2:instance'
- id: aws/metrics-ebs-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.ebs-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.ebs
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 5m
tags_filter: null
metricsets:
- ebs
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws-cloudwatch-ec2-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws-cloudwatch
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws-cloudwatch-aws.ec2_logs-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.ec2_logs
type: logs
log_group_arn: >-
arn:aws:logs:ap-southeast-2:757008298094:log-group:aws-cloudtrail-logs-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
start_position: end
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
api_sleep: 200ms
endpoint: logs.ap-southeast-2.amazonaws.com
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
scan_frequency: 1m
tags:
- forwarded
- aws-ec2-logs
publisher_pipeline.disable_host: true
- id: aws/metrics-ec2-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.ec2_metrics-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.ec2_metrics
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 5m
tags_filter: null
metricsets:
- ec2
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws/metrics-s3-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.s3_daily_storage-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.s3_daily_storage
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 1m
metricsets:
- s3_daily_storage
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws/metrics-aws.s3_request-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.s3_request
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 1m
metricsets:
- s3_request
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws/metrics-s3_storage_lens-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.s3_storage_lens-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.s3_storage_lens
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 3h
metricsets:
- cloudwatch
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
metrics:
- statistic:
- Average
namespace: AWS/S3/Storage-Lens
processors:
- rename:
ignore_missing: true
fields:
- from: aws.storage-lens.metrics.4xxErrors.avg
to: aws.s3_storage_lens.metrics.4xxErrors.avg
- from: aws.storage-lens.metrics.5xxErrors.avg
to: aws.s3_storage_lens.metrics.5xxErrors.avg
- from: aws.storage-lens.metrics.AllRequests.avg
to: aws.s3_storage_lens.metrics.AllRequests.avg
- from: aws.storage-lens.metrics.BytesDownloaded.avg
to: aws.s3_storage_lens.metrics.BytesDownloaded.avg
- from: aws.storage-lens.metrics.BytesUploaded.avg
to: aws.s3_storage_lens.metrics.BytesUploaded.avg
- from: aws.storage-lens.metrics.CurrentVersionObjectCount.avg
to: aws.s3_storage_lens.metrics.CurrentVersionObjectCount.avg
- from: aws.storage-lens.metrics.CurrentVersionStorageBytes.avg
to: aws.s3_storage_lens.metrics.CurrentVersionStorageBytes.avg
- from: aws.storage-lens.metrics.DeleteMarkerObjectCount.avg
to: aws.s3_storage_lens.metrics.DeleteMarkerObjectCount.avg
- from: aws.storage-lens.metrics.DeleteRequests.avg
to: aws.s3_storage_lens.metrics.DeleteRequests.avg
- from: aws.storage-lens.metrics.EncryptedObjectCount.avg
to: aws.s3_storage_lens.metrics.EncryptedObjectCount.avg
- from: aws.storage-lens.metrics.EncryptedStorageBytes.avg
to: aws.s3_storage_lens.metrics.EncryptedStorageBytes.avg
- from: aws.storage-lens.metrics.GetRequests.avg
to: aws.s3_storage_lens.metrics.GetRequests.avg
- from: aws.storage-lens.metrics.HeadRequests.avg
to: aws.s3_storage_lens.metrics.HeadRequests.avg
- from: >-
aws.storage-lens.metrics.IncompleteMultipartUploadObjectCount.avg
to: >-
aws.s3_storage_lens.metrics.IncompleteMultipartUploadObjectCount.avg
- from: >-
aws.storage-lens.metrics.IncompleteMultipartUploadStorageBytes.avg
to: >-
aws.s3_storage_lens.metrics.IncompleteMultipartUploadStorageBytes.avg
- from: aws.storage-lens.metrics.ListRequests.avg
to: aws.s3_storage_lens.metrics.ListRequests.avg
- from: aws.storage-lens.metrics.NonCurrentVersionObjectCount.avg
to: aws.s3_storage_lens.metrics.NonCurrentVersionObjectCount.avg
- from: aws.storage-lens.metrics.NonCurrentVersionStorageBytes.avg
to: >-
aws.s3_storage_lens.metrics.NonCurrentVersionStorageBytes.avg
- from: aws.storage-lens.metrics.ObjectCount.avg
to: aws.s3_storage_lens.metrics.ObjectCount.avg
- from: aws.storage-lens.metrics.ObjectLockEnabledObjectCount.avg
to: aws.s3_storage_lens.metrics.ObjectLockEnabledObjectCount.avg
- from: aws.storage-lens.metrics.ObjectLockEnabledStorageBytes.avg
to: >-
aws.s3_storage_lens.metrics.ObjectLockEnabledStorageBytes.avg
- from: aws.storage-lens.metrics.PostRequests.avg
to: aws.s3_storage_lens.metrics.PostRequests.avg
- from: aws.storage-lens.metrics.PutRequests.avg
to: aws.s3_storage_lens.metrics.PutRequests.avg
- from: aws.storage-lens.metrics.ReplicatedObjectCount.avg
to: aws.s3_storage_lens.metrics.ReplicatedObjectCount.avg
- from: aws.storage-lens.metrics.ReplicatedStorageBytes.avg
to: aws.s3_storage_lens.metrics.ReplicatedStorageBytes.avg
- from: aws.storage-lens.metrics.SelectRequests.avg
to: aws.s3_storage_lens.metrics.SelectRequests.avg
- from: aws.storage-lens.metrics.SelectReturnedBytes.avg
to: aws.s3_storage_lens.metrics.SelectReturnedBytes.avg
- from: aws.storage-lens.metrics.SelectScannedBytes.avg
to: aws.s3_storage_lens.metrics.SelectScannedBytes.avg
- from: aws.storage-lens.metrics.StorageBytes.avg
to: aws.s3_storage_lens.metrics.StorageBytes.avg
- drop_fields:
ignore_missing: true
fields:
- aws.storage-lens
- id: aws/metrics-sns-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.sns-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.sns
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 5m
tags_filter: null
metricsets:
- sns
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws/metrics-transitgateway-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.transitgateway-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.transitgateway
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 1m
metricsets:
- transitgateway
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws/metrics-usage-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws/metrics
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws/metrics-aws.usage-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.usage
type: metrics
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
period: 1m
metricsets:
- usage
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
- id: aws-cloudwatch-vpcflow-baef3b24-99ae-48f0-b48a-a63bdbc82612
name: POC-syd-aws-integration
revision: 11
type: aws-cloudwatch
use_output: default
meta:
package:
name: aws
version: 1.16.0
data_stream:
namespace: default
streams:
- id: aws-cloudwatch-aws.vpcflow-baef3b24-99ae-48f0-b48a-a63bdbc82612
data_stream:
dataset: aws.vpcflow
type: logs
start_position: end
access_key_id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
api_sleep: 200ms
endpoint: logs.ap-southeast-2.amazonaws.com
secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
region_name: null
scan_frequency: 1m
tags:
- forwarded
- aws-vpcflow
publisher_pipeline.disable_host: true
Checking if anyone has any idea
zmoog
May 25, 2022, 1:52pm
6
I see you have a lot of AWS integrations enabled: which log type is getting duplicates?
system
June 22, 2022, 3:52pm
7
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.