Filebeat Index Name Pattern - timestamp

Elastic Stack Beats
1
  1. Is %{+yyyy.MM.dd} the system time or log file timestamp?
    Configure the Elasticsearch output | Filebeat Reference [7.10] | Elastic
output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "%{[fields.log_type]}-%{[agent.version]}-%{+yyyy.MM.dd}"
  1. What's the difference between YYYY and yyyy?

I've got answer for this. https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
YYYY - week-based-year
yyyy - year-of-era

2

Filebeat uses time series indices, by default, when index lifecycle management is disabled or unsupported. The indices are named filebeat-7.10.2-yyyy.MM.dd, where yyyy.MM.dd is the date when the events were indexed.

What does this mean?

Scenario:
When Filebeat starts, Elasticsearch generates a few indices like filebeat-7.10.2-2022.01.30, filebeat-7.10.2-2022.01.31, filebeat-7.10.2-2022.02.01 at the same time. How does it happen?

3

Filebeat uses time series indices, by default, when index lifecycle management is disabled or unsupported. The indices are named filebeat-7.10.2-yyyy.MM.dd, where yyyy.MM.dd is the date when the events were indexed.

Can anybody elaborate where yyyy.MM.dd is the date when the events were indexed?
Is yyyy.MM.dd the same as system date?

4

Is there an Elastic developer in this forum?
I think a developer may know the answer of this question.

5 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /logs/access_log.*.log

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}"

When Filebeat starts, Elasticsearch generates a few indices like filebeat-7.10.2-2022.01.30, filebeat-7.10.2-2022.01.31, filebeat-7.10.2-2022.02.01 at the same time. How does it happen?

6

output.Elasticsearch:

hosts: ["192.168.0.12:9200"]

enabled: true

index: "test-1-%{[agent.version]}-%{+yyyy.MM.dd}"

setup.template.name: "test-1"

setup.template.pattern: "test-1-*"

setup.template.overwrit: true

setup.template.enabled: true

setup.ilm.enabled: false

7

I've got:
setup.template.enabled: false

8

I suspected that the old registry was the culprit so I've removed the old registry and restarted Filebeat. All of logs goes to the current index now.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.