Fleet Synthetics 7.14 - pipeline with id [synthetics-http-0.2.1] does not exist

Elastic Stack Elasticsearch
,
1

Using Fleet 7.14 and after upgrading the Integration - Elastic Synthetics, i cannot see any http Monitors and when i look at the Elastic-Agent logs, i get the following message that it cannot index the document because a pipeline is missing -> synthetics-http-0.2.1

All other monitors work -> ICMP & TCP (albeit a known issue with the HOST:PORT configuration which makes the Agent show Unhealthy).

Also what works if I run the docker container for ELastic Synthetics and add http type monitors via the docker container.

Is seems i am missing the pipeline in my host configuration, where i need it. Where can i find the configuration for this pipeline or resolve the issue at hand?

2

Sorry to hear about these issues. Can you share where that error is coming from? I assume from the heartbeat logs in the elastic-agent container? Can you share the full log lines verbatim?

3

{"log.level":"warn","@timestamp":"2021-08-09T15:59:23.214+0200","log.logger":"elasticsearch","log.origin":{"file.name":"elasticsearch/client.go","file.line":408},"message":"Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xc03c6b0e89dd986c, ext:2759716019152356, loc:(time.Location)(0x55f8594f6300)}, Meta:{"raw_index":"synthetics-http-default"}, Fields:{"agent":{"ephemeral_id":"5d47729e-e84f-4ac2-94fa-327ac21817cf","hostname":"IsaacAsimov","id":"b8679bdd-1484-4081-99c8-b84313f36253","name":"IsaacAsimov","type":"heartbeat","version":"7.13.3"},"ecs":{"version":"1.9.0"},"event":{"dataset":"http"},"http":{"response":{"body":{"bytes":3525,"hash":"a0c8935fdf5fd4f640f4e136e7cce5847bb3258ef58fce53e313d757133365c1"},"headers":{"Content-Length":"3525","Content-Type":"text/html; charset=utf-8","Date":"Mon, 09 Aug 2021 13:59:22 GMT","Server":"nginx/1.14.2","Strict-Transport-Security":"max-age=31536000; includeSubdomains","Vary":"Accept-Encoding"},"mime_type":"text/html; charset=utf-8","status_code":200},"rtt":{"content":{"us":713},"response_header":{"us":5660},"total":{"us":24111},"validate":{"us":6374},"write_request":{"us":90}}},"monitor":{"check_group":"dbd18751-f919-11eb-8021-ecb1d75ce34d","duration":{"us":46312},"fleet_managed":true,"id":"75c66e6c-74ec-4ef1-8a1b-0d36e269b635","ip":"212.187.124.237","name":"Home Assistant Service - Siobytes","status":"up","timespan":{"gte":"2021-08-09T13:59:22.211Z","lt":"2021-08-09T14:00:22.211Z"},"type":"http"},"observer":{"geo":{"name":"Fleet managed"},"hostname":"IsaacAsimov","ip":["192.168.0.158","192.168.0.3","fe80::59d7:2569:2afb:4d13","fe80::eeb1:d7ff:fe5c:e34d"],"mac":["ec:b1:d7:5c:e3:4d"]},"resolve":{"ip":"212.187.124.237","rtt":{"us":22088}},"summary":{"down":0,"up":1},"tcp":{"rtt":{"connect":{"us":4004}}},"tls":{"certificate_not_valid_after":"2021-12-02T23:59:59.000Z","certificate_not_valid_before":"2020-12-02T00:00:00.000Z","cipher":"ECDHE-RSA-AES-256-GCM-SHA384","established":true,"rtt":{"handshake":{"us":13440}},"server":{"hash":{"sha1":"6a0d32f3cb168c3d11b1fa41901d3f6d30467710","sha256":"2d63cd48a0a4f4295ad7945a08513d4424a6aba71d3fe2ee133760a21d018d0f"},"x509":{"issuer":{"common_name":"RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1","distinguished_name":"CN=RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1,O=DigiCert Inc,C=US"},"not_after":"2021-12-02T23:59:59.000Z","not_before":"2020-12-02T00:00:00.000Z","public_key_algorithm":"RSA","public_key_exponent":65537,"public_key_size":2048,"serial_number":"20745371972699712190373077745864910861","signature_algorithm":"SHA256-RSA","subject":{"common_name":".siobytes.com","distinguished_name":"CN=*.siobytes.com"}}},"version":"1.2","version_protocol":"tls"},"url":{"domain":"home.siobytes.com","full":"https://home.siobytes.com/\",\"path\":\"/\",\"port\":443,\"scheme\":\"https\"}}, Private:interface {}(nil), TimeSeries:false}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"illegal_argument_exception","reason":"pipeline with id [synthetics-http-0.2.1] does not exist"}","ecs.version":"1.6.0"}

4

Would you also be able to share with us the agent policy YAML running this? I'm wondering if it's configuring an extra pipeline somehow. It looks like somehow an agent pipeline was created. I tried to repro this but couldn't.

You can find the policy by clicking on the view policy button on the screen shown below.

image
image2458×1058 169 KB

5

The whole policy is too long.... this is the snippet of the synthetics part

  - id: 93ab26df-d5ad-4fa7-b1a0-38fc415cc7bc
    name: Synthetics - Elasticsearch Service
    revision: 1
    type: synthetics/http
    use_output: default
    meta:
      package:
        name: synthetics
        version: 0.2.1
    data_stream:
      namespace: default
    streams:
      - id: synthetics/http-http-93ab26df-d5ad-4fa7-b1a0-38fc415cc7bc
        name: Synthetics - Elasticsearch Service
        type: http
        data_stream:
          dataset: http
          type: synthetics
        urls: 'https://elasticsearch.siobytes.com/_cluster/health'
        schedule: '@every 1m'
        timeout: 16s
        max_redirects: 0
        tags:
          - elastic
        username: redacted
        password: redacted
        response.include_headers: true
        response.include_body: always
        check.request.method: GET
        processors:
          - add_observer_metadata:
              geo:
                name: Fleet managed
          - add_fields:
              target: ''
              fields:
                monitor.fleet_managed: true```
6

created a new policy and assigned to a single agent, so running just two integrations

id: 2702a800-d6f1-11eb-aa9f-898b32a11cf9
revision: 15
outputs:
  default:
    type: elasticsearch
    hosts:
      - 'https://elasticsearch.siobytes.com:443'
output_permissions:
  default:
    system-1:
      indices:
        - names:
            - logs-system.auth-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.syslog-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.application-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.security-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - logs-system.system-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.core-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.cpu-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.diskio-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.filesystem-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.fsstat-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.load-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.memory-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.network-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.process.summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.socket_summary-default
          privileges:
            - auto_configure
            - create_doc
        - names:
            - metrics-system.uptime-default
          privileges:
            - auto_configure
            - create_doc
    Elastic:
      indices:
        - names:
            - synthetics-http-default
          privileges:
            - auto_configure
            - create_doc
    _elastic_agent_checks:
      cluster:
        - monitor
      indices:
        - names:
            - logs-elastic_agent-default
            - logs-elastic_agent.elastic_agent-default
            - logs-elastic_agent.apm_server-default
            - logs-elastic_agent.filebeat-default
            - logs-elastic_agent.fleet_server-default
            - logs-elastic_agent.metricbeat-default
            - logs-elastic_agent.osquerybeat-default
            - logs-elastic_agent.packetbeat-default
            - logs-elastic_agent.endpoint_security-default
            - logs-elastic_agent.auditbeat-default
            - metrics-elastic_agent-default
            - metrics-elastic_agent.elastic_agent-default
            - metrics-elastic_agent.apm_server-default
            - metrics-elastic_agent.filebeat-default
            - metrics-elastic_agent.fleet_server-default
            - metrics-elastic_agent.metricbeat-default
            - metrics-elastic_agent.osquerybeat-default
            - metrics-elastic_agent.packetbeat-default
            - metrics-elastic_agent.endpoint_security-default
            - metrics-elastic_agent.auditbeat-default
          privileges:
            - auto_configure
            - create_doc
agent:
  monitoring:
    enabled: true
    use_output: default
    namespace: default
    logs: true
    metrics: true
inputs:
  - id: 6276156f-3761-422e-85b2-da303436122d
    name: system-1
    revision: 3
    type: logfile
    use_output: default
    meta:
      package:
        name: system
        version: 0.12.7
    data_stream:
      namespace: default
    streams:
      - id: logfile-system.auth-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.auth
          type: logs
        exclude_files:
          - .gz$
        paths:
          - /var/log/auth.log*
          - /var/log/secure*
        multiline:
          pattern: ^\s
          match: after
        processors:
          - add_locale: null
          - add_fields:
              fields:
                ecs.version: 1.9.0
              target: ''
      - id: logfile-system.syslog-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.syslog
          type: logs
        exclude_files:
          - .gz$
        paths:
          - /var/log/messages*
          - /var/log/syslog*
        multiline:
          pattern: ^\s
          match: after
        processors:
          - add_locale: null
          - add_fields:
              fields:
                ecs.version: 1.9.0
              target: ''
  - id: 6276156f-3761-422e-85b2-da303436122d
    name: system-1
    revision: 3
    type: winlog
    use_output: default
    meta:
      package:
        name: system
        version: 0.12.7
    data_stream:
      namespace: default
    streams:
      - id: winlog-system.application-6276156f-3761-422e-85b2-da303436122d
        name: Application
        data_stream:
          dataset: system.application
          type: logs
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
        tags: null
      - id: winlog-system.security-6276156f-3761-422e-85b2-da303436122d
        name: Security
        data_stream:
          dataset: system.security
          type: logs
        condition: '${host.platform} == ''windows'''
        tags: null
      - id: winlog-system.system-6276156f-3761-422e-85b2-da303436122d
        name: System
        data_stream:
          dataset: system.system
          type: logs
        condition: '${host.platform} == ''windows'''
        tags: null
  - id: 6276156f-3761-422e-85b2-da303436122d
    name: system-1
    revision: 3
    type: system/metrics
    use_output: default
    meta:
      package:
        name: system
        version: 0.12.7
    data_stream:
      namespace: default
    streams:
      - id: system/metrics-system.core-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.core
          type: metrics
        metricsets:
          - core
        core.metrics:
          - percentages
      - id: system/metrics-system.cpu-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.cpu
          type: metrics
        period: 10s
        cpu.metrics:
          - percentages
          - normalized_percentages
        metricsets:
          - cpu
      - id: system/metrics-system.diskio-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.diskio
          type: metrics
        period: 10s
        diskio.include_devices: null
        metricsets:
          - diskio
      - id: system/metrics-system.filesystem-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.filesystem
          type: metrics
        period: 1m
        metricsets:
          - filesystem
        processors:
          - drop_event.when.regexp:
              system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.fsstat-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.fsstat
          type: metrics
        period: 1m
        metricsets:
          - fsstat
        processors:
          - drop_event.when.regexp:
              system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - id: system/metrics-system.load-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.load
          type: metrics
        condition: '${host.platform} != ''windows'''
        period: 10s
        metricsets:
          - load
      - id: system/metrics-system.memory-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.memory
          type: metrics
        period: 10s
        metricsets:
          - memory
      - id: system/metrics-system.network-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.network
          type: metrics
        period: 10s
        network.interfaces: null
        metricsets:
          - network
      - id: system/metrics-system.process-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.process
          type: metrics
        process.include_top_n.by_memory: 5
        period: 10s
        processes:
          - .*
        process.include_top_n.by_cpu: 5
        process.cgroups.enabled: false
        process.cmdline.cache.enabled: true
        metricsets:
          - process
        process.include_cpu_ticks: false
      - id: >-
          system/metrics-system.process.summary-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.process.summary
          type: metrics
        period: 10s
        metricsets:
          - process_summary
      - id: >-
          system/metrics-system.socket_summary-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.socket_summary
          type: metrics
        period: 10s
        metricsets:
          - socket_summary
      - id: system/metrics-system.uptime-6276156f-3761-422e-85b2-da303436122d
        data_stream:
          dataset: system.uptime
          type: metrics
        period: 10s
        metricsets:
          - uptime
  - id: c9807c60-2f44-4126-b844-66238012f778
    name: Elastic
    revision: 1
    type: synthetics/http
    use_output: default
    meta:
      package:
        name: synthetics
        version: 0.2.1
    data_stream:
      namespace: default
    streams:
      - id: synthetics/http-http-c9807c60-2f44-4126-b844-66238012f778
        name: Elastic
        type: http
        data_stream:
          dataset: http
          type: synthetics
        urls: 'https://elasticsearch.siobytes.com/_cluster/health'
        schedule: '@every 1m'
        timeout: 16s
        max_redirects: 0
        username: redacted
        password: redacted
        response.include_headers: true
        response.include_body: always
        check.request.method: GET
        processors:
          - add_observer_metadata:
              geo:
                name: Fleet managed
          - add_fields:
              target: ''
              fields:
                monitor.fleet_managed: true
fleet:
  hosts:
    - 'https://192.168.0.3:8220'![Screen Shot 2021-08-13 at 5.15.25 AM|549x500](upload://50aNQiWWJQpsvKTEMWFeQtiDOIm.jpeg)
7

Just an FYI incase it was missed that this is still not working, the last reply was to isolate the integration to see if the multiple integrations was an issue

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.