API logs not coming after setting up Elastic Agent , Beats logs are coming

It looks like you are configuring this as if it were filebeat but the Elastic Agent standalone configuration is different. The reference configuration for standalone agent looks like this:

inputs:
  # Collect custom data from REST API's: Collect custom data from REST API's
  - id: generic-httpjson
    type: httpjson
    streams:
      # Custom API Input: Collect custom data from REST API's
      - id: httpjson-httpjson.generic
        data_stream:
          dataset: httpjson.generic
        config_version: 2
        interval: 1m
        request.url: 'https://server.example.com:8089/api'
        request.method: GET
        request.tracer.filename: ../../logs/httpjson/http-request-trace-*.ndjson
        request.tracer.maxbackups: 5
        tags:
          - forwarded
        publisher_pipeline.disable_host: true
        # auth.basic.user: <USERNAME> # Basic Auth Username: The username to be used with Basic Auth headers
        # auth.basic.password: <PASSWORD> # Basic Auth Password: The password to be used with Basic Auth headers
        # pipeline: <PIPELINE>
        #  # Ingest Pipeline: The Ingest Node pipeline ID to be used by the integration.
        # request.encode_as: <REQUEST_ENCODE_AS> # Request Encode As: ContentType used for encoding the request body. If set it will force the encoding in the specified format regardless of the Content-Type header value.
        # request.timeout: <REQUEST_TIMEOUT> # Request Timeout: Duration before declaring that the HTTP client connection has timed out. Valid time units are ns, us, ms, s, m, h. Default is "30"s.
        # request.proxy_url: <REQUEST_PROXY_URL> # Request Proxy: This specifies proxy configuration in the form of `http[s]://<user>:<password>@<server name/ip>:<port>`.
        # request.retry.max_attempts: <REQUEST_RETRY_MAX_ATTEMPTS> # Request Retry Max Attempts: The maximum number of retries for the HTTP client. Default is "5".
        # request.retry.wait_min: <REQUEST_RETRY_WAIT_MIN> # Request Retry Wait Min: The minimum time to wait before a retry is attempted. Default is "1s".
        # request.retry.wait_max: <REQUEST_RETRY_WAIT_MAX> # Request Retry Wait Max: The maximum time to wait before a retry is attempted. Default is "60s".
        # request.redirect.forward_headers: <REQUEST_REDIRECT_FORWARD_HEADERS> # Request Redirect Forward Headers: When set to true request headers are forwarded in case of a redirect. Default is "false".
        # request.redirect.headers_ban_list:
        #  - <REQUEST_REDIRECT_HEADERS_BAN_LIST> # Request Redirect Headers Ban List: When Redirect Forward Headers is set to true, all headers except the ones defined in this list will be forwarded. All headers are forwarded by default.
        # request.redirect.max_redirects: <REQUEST_REDIRECT_MAX_REDIRECTS> # Request Redirect Max Redirects: The maximum number of redirects to follow for a request. Default is "10".
        # request.rate_limit.limit: <REQUEST_RATE_LIMIT_LIMIT> # Request Rate Limit: The value of the response that specifies the total limit. It is defined with a Go template value.
        # request.rate_limit.reset: <REQUEST_RATE_LIMIT_RESET> # Request Rate Limit Reset: The value of the response that specifies the epoch time when the rate limit will reset. It is defined with a Go template value.
        # request.rate_limit.remaining: <REQUEST_RATE_LIMIT_REMAINING> # Request Rate Limit Remaining: The value of the response that specifies the remaining quota of the rate limit. It is defined with a Go template value.
        # response.decode_as: <RESPONSE_DECODE_AS>
        #  # Response decode settings: ContentType used for decoding the response body. Supported values: application/json, application/x-ndjson. By default it will use what is in the response Content-Type header.
        # response.request_body_on_pagination: <RESPONSE_REQUEST_BODY_ON_PAGINATION>
        #  # Include request body on Pagination: If set to true, the values in request.body are sent with pagination requests.
        # processors:

If you go into Fleet and make an Agent Policy with the Custom API integration configured how you want it configured you can click Actions -> View Policy to view the configuration that is provided to the agent:

1 Like