The first connection to logstash is always an error

Elastic Stack Beats
1

When running lambda, I always see "retry" first, and then immediately connect to backoff.
How to make logs sent on first connection?
Is there any way to enable more readable logs?

Снимок экрана 2022-02-04 в 13.44.02
Снимок экрана 2022-02-04 в 13.44.022130×1162 453 KB

config functionbeat

functionbeat:
  provider:
    aws:
      deploy_bucket: functionbeat
      endpoint: s3.amazonaws.com
      functions:
      - description: lambda function for cloudwatch logs
        enabled: true
        name: elk-shipper
        triggers:
        - log_group_name: /aws/lambda/
        type: cloudwatch_logs
keystore:
  path: /tmp/functionbeat.keystore
output:
  logstash:
    hosts:
    - logstash.net:1999
    timeout: 8
    pipelining: 0
path:
  config: /etc/functionbeat
  data: /tmp
  home: /etc/functionbeat
  logs: /tmp/logs
processors:
- add_fields:
    fields:
      log_signal: ASDFGH
    target: ""
logging.metrics.period: 1s
2

This is not an error. backoff and retryer are internal elements of the publisher pipeline. All publishers contain a backoff and retry functionality and what you see in the logs are messages from the code files with those names. There is no error at that point.

The logs are just letting you know that Functionbeat is connecting to a URL and it has backoff enabled and it will retry if there is an error during connection.

3

Can you tell me how can I find out why there was a delay? For me, this is important and I would like to make sure that everything is done the first time.

4

There was no delay according to the logs you shared. Are the events not making it into ES or what is the problem you see?

5

I would like to make sure that the log is sent as quickly as possible.
Therefore, I want to make sure that the logs are sent the first time and there are no errors.
Because the words "backoff" and "retryer" indicate that the logs are sent at least the second time.

Can you confirm that the logs are sent the first time, on the first connection, without errors?

6

@kvch I will be glad if you can confirm that there are no errors.

7

There was no delay or error based on the log lines you shared.

8

When creating a load, an error appears in the logs, see the picture

image
image2806×690 200 KB

At the same time, there are no errors on the logstash side.
In this case, all the logs are delivered, where can this error come from?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.