# Cannot connect "packetbeat-7.6.1-1.x86\_64"\[oss version\] to AWS Managed ES \[v7.4.2\] **URL:** https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553 **Category:** Beats **Tags:** packetbeat **Created:** [March 22, 2020, 11:25am UTC](https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553 "2020-03-22T11:25:55Z") **Posts on this page:** 4 **Page:** 1
### Author: ![Yessen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yessen/32/51497_2.png) [@Yessen](https://discuss.elastic.co/u/Yessen) #### Post date: [March 22, 2020, 11:25am UTC](https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553/1 "2020-03-22T11:25:55Z")
output of curl from the box where packetbeat is installed : ```auto curl https://aws-es-domain-endpoint.com:443 { "name" : "99abf5c22ca3a14294d52af72a17df86", "cluster_name" : "528130383285:test-infra-ek", "cluster_uuid" : "-P1lge8AQAOBr6wdEINnvg", "version" : { "number" : "7.4.2", "build_flavor" : "oss", "build_type" : "tar", "build_hash" : "unknown", "build_date" : "2020-02-06T10:08:47.217314Z", "build_snapshot" : false, "lucene_version" : "8.2.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` packetbeat.yml config : ```auto #################### Packetbeat Configuration Example ######################### # This file is an example configuration file highlighting only the most common # options. The packetbeat.reference.yml file from the same directory contains all the # supported options with more comments. You can use it as a reference. # # You can find the full configuration reference here: # https://www.elastic.co/guide/en/beats/packetbeat/index.html #============================== Network device ================================ # Select the network interface to sniff the data. On Linux, you can use the # "any" keyword to sniff on all connected interfaces. packetbeat.interfaces.device: any #================================== Flows ===================================== # Set `enabled: false` or comment out all options to disable flows reporting. packetbeat.flows: # Set network flow timeout. Flow is killed if no packet is received before being # timed out. timeout: 30s # Configure reporting period. If set to -1, only killed flows will be reported period: 10s #========================== Transaction protocols ============================= packetbeat.protocols: - type: icmp # Enable ICMPv4 and ICMPv6 monitoring. Default: false enabled: true - type: amqp # Configure the ports where to listen for AMQP traffic. You can disable # the AMQP protocol by commenting out the list of ports. ports: [5672] - type: cassandra #Cassandra port for traffic monitoring. ports: [9042] - type: dhcpv4 # Configure the DHCP for IPv4 ports. ports: [67, 68] - type: dns # Configure the ports where to listen for DNS traffic. You can disable # the DNS protocol by commenting out the list of ports. ports: [53] - type: http # Configure the ports where to listen for HTTP traffic. You can disable # the HTTP protocol by commenting out the list of ports. ports: [80, 8080, 8000, 5000, 8002] - type: memcache # Configure the ports where to listen for memcache traffic. You can disable # the Memcache protocol by commenting out the list of ports. ports: [11211] - type: mysql # Configure the ports where to listen for MySQL traffic. You can disable # the MySQL protocol by commenting out the list of ports. ports: [3306,3307] - type: pgsql # Configure the ports where to listen for Pgsql traffic. You can disable # the Pgsql protocol by commenting out the list of ports. ports: [5432] - type: redis # Configure the ports where to listen for Redis traffic. You can disable # the Redis protocol by commenting out the list of ports. ports: [6379] - type: thrift # Configure the ports where to listen for Thrift-RPC traffic. You can disable # the Thrift-RPC protocol by commenting out the list of ports. ports: [9090] - type: mongodb # Configure the ports where to listen for MongoDB traffic. You can disable # the MongoDB protocol by commenting out the list of ports. ports: [27017] - type: nfs # Configure the ports where to listen for NFS traffic. You can disable # the NFS protocol by commenting out the list of ports. ports: [2049] - type: tls # Configure the ports where to listen for TLS traffic. You can disable # the TLS protocol by commenting out the list of ports. ports: - 443 # HTTPS - 993 # IMAPS - 995 # POP3S - 5223 # XMPP over SSL - 8443 - 8883 # Secure MQTT - 9243 # Elasticsearch #==================== Elasticsearch template setting ========================== setup.template.settings: index.number_of_shards: 1 #index.codec: best_compression #_source.enabled: false setup.template: name: "client-box" pattern: "client-box-*" #================================ General ===================================== # The name of the shipper that publishes the network data. It can be used to group # all the transactions sent by a single shipper in the web interface. #name: # The tags of the shipper are included in their own field with each # transaction published. #tags: ["service-X", "web-tier"] # Optional fields that you can specify to add additional information to the # output. #fields: # env: staging #============================== Dashboards ===================================== # These settings control loading the sample dashboards to the Kibana index. Loading # the dashboards is disabled by default and can be enabled either by setting the # options here or by using the `setup` command. #setup.dashboards.enabled: false # The URL from where to download the dashboards archive. By default this URL # has a value which is computed based on the Beat name and version. For released # versions, this URL points to the dashboard archive on the artifacts.elastic.co # website. #setup.dashboards.url: #============================== Kibana ===================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: #============================= Elastic Cloud ================================== # These settings simplify using Packetbeat with the Elastic Cloud (https://cloud.elastic.co/). # The cloud.id setting overwrites the `output.elasticsearch.hosts` and # `setup.kibana.host` options. # You can find the `cloud.id` in the Elastic Cloud web UI. #cloud.id: # The cloud.auth setting overwrites the `output.elasticsearch.username` and # `output.elasticsearch.password` settings. The format is `:`. #cloud.auth: #================================ Outputs ===================================== # Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------ output.elasticsearch: # Array of hosts to connect to. hosts: ["https://aws-es-domain-endpoint.com:443"] index: "client-box-%{+yyyy.MM.dd}" # Protocol - either `http` (default) or `https`. #protocol: "https" # Authentication credentials - either API key or username/password. #api_key: "id:api_key" #username: "elastic" #password: "changeme" #----------------------------- Logstash output -------------------------------- #output.logstash: # The Logstash hosts #hosts: ["localhost:5044"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key" #================================ Processors ===================================== # Configure processors to enhance or manipulate events generated by the beat. processors: - add_host_metadata: ~ - add_cloud_metadata: ~ - add_docker_metadata: ~ #================================ Logging ===================================== # Sets log level. The default log level is info. # Available log levels are: error, warning, info, debug #logging.level: debug # At debug level, you can selectively enable logging only for some components. # To enable all selectors use ["*"]. Examples of other selectors are "beat", # "publish", "service". #logging.selectors: ["*"] #============================== X-Pack Monitoring =============================== # packetbeat can export internal metrics to a central Elasticsearch monitoring # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The # reporting is disabled by default. # Set to true to enable the monitoring reporter. #monitoring.enabled: false # Sets the UUID of the Elasticsearch cluster under which monitoring data for this # Packetbeat instance will appear in the Stack Monitoring UI. If output.elasticsearch # is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch. #monitoring.cluster_uuid: # Uncomment to send the metrics to Elasticsearch. Most settings from the # Elasticsearch output are accepted here as well. # Note that the settings should point to your Elasticsearch *monitoring* cluster. # Any setting that is not set is automatically inherited from the Elasticsearch # output configuration, so if you have the Elasticsearch output configured such # that it is pointing to your Elasticsearch monitoring cluster, you can simply # uncomment the following line. #monitoring.elasticsearch: #================================= Migration ================================== # This allows to enable 6.7 migration aliases #migration.6_to_7.enabled: true ``` errors : ```auto Mar 22 11:23:02 client-box packetbeat[8926]: 2020-03-22T11:23:02.392Z INFO elasticsearch/client.go:757 Attempting to connect to Elasticsearch version 7.4.2 Mar 22 11:23:17 client-box packetbeat[8926]: 2020-03-22T11:23:17.649Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":320},"total":{"ticks":2560,"time":{"ms":4},"value":2560},"user":{"ticks":2240,"time":{"ms":4}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":8},"info":{"ephemeral_id":"6807605d-45bf-4247-a964-8c84b1e0fe0e","uptime":{"ms":2400020}},"memstats":{"gc_next":75363648,"memory_alloc":38625472,"memory_total":162505992},"runtime":{"goroutines":40}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":996},"write":{"bytes":370}},"pipeline":{"clients":15,"events":{"active":4117,"retry":50}}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}} Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(https://aws-es-domain-endpoint.com:443)): Connection marked as failed because the onConnect callback failed: request checking for ILM availability failed: 401 Unauthorized: {"Message":"Your request: '/_xpack' is not allowed."} Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(https://aws-es-domain-endpoint.com:443)) with 57 reconnect attempt(s) Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z INFO [publisher] pipeline/retry.go:196 retryer: send unwait-signal to consumer Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z INFO [publisher] pipeline/retry.go:198 done Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z INFO [publisher] pipeline/retry.go:173 retryer: send wait signal to consumer Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.522Z INFO [publisher] pipeline/retry.go:175 done Mar 22 11:23:45 client-box packetbeat[8926]: 2020-03-22T11:23:45.524Z INFO elasticsearch/client.go:757 Attempting to connect to Elasticsearch version 7.4.2 Mar 22 11:23:47 client-box packetbeat[8926]: 2020-03-22T11:23:47.649Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":320,"time":{"ms":3}},"total":{"ticks":2610,"time":{"ms":52},"value":2610},"user":{"ticks":2290,"time":{"ms":49}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":8},"info":{"ephemeral_id":"6807605d-45bf-4247-a964-8c84b1e0fe0e","uptime":{"ms":2430020}},"memstats":{"gc_next":75363584,"memory_alloc":37790800,"memory_total":162906880},"runtime":{"goroutines":40}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":996},"write":{"bytes":370}},"pipeline":{"clients":15,"events":{"active":4117,"retry":4}}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}} ``` Can someone please help ? ---
### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist) #### Post date: [March 22, 2020, 12:27pm UTC](https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553/3 "2020-03-22T12:27:10Z")
Try [disabling ILM](https://www.elastic.co/guide/en/beats/packetbeat/7.6/ilm.html#setup-ilm-option). ---
### Author: ![Yessen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yessen/32/51497_2.png) [@Yessen](https://discuss.elastic.co/u/Yessen) #### Post date: [April 2, 2020, 2:48pm UTC](https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553/4 "2020-04-02T14:48:10Z")
Worked, Thanks 😃 ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [April 30, 2020, 2:48pm UTC](https://discuss.elastic.co/t/cannot-connect-packetbeat-7-6-1-1-x86-64-oss-version-to-aws-managed-es-v7-4-2/224553/5 "2020-04-30T14:48:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.