Hello Guys,
I have an issue with Logstash, it creates index in elasticsearch and archive filename with variable name instead variable value and next creates index and archive filename with correct name.
My logstash config:
input {
tcp {
port => 5000
type => syslog
mode => "server"
ssl_enable => true
ssl_verify => false
ssl_cert => "/etc/logstash/ssl/logstash.crt"
ssl_key => "/etc/logstash/ssl/logstash.key"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "ip", "%{[@metadata][input][tcp][source][ip]}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
date_formatter {
source => "@timestamp"
target => "log_day"
pattern => "YYYY.MM.dd"
}
}
}
output {
elasticsearch {
hosts => ["https://localhost:9200"]
user => "logstash_internal"
password => "password"
ssl => true
ssl_certificate_verification => false
index => "syslog-%{log_day}"
}
file {
path => "/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"
gzip => true
}
}
Created Index:
Logstash log:
Sep 28 12:47:10 log.in.domain.com logstash[2547]: [2022-09-28T12:47:10,893][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
Sep 28 12:47:21 log.in.domain.com logstash[5052]: [2022-09-28T12:47:21,337][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties
Sep 28 12:47:21 log.in.domain.com logstash[5052]: [2022-09-28T12:47:21,344][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.4.2", "jruby.version"=>"jruby 9.3.8.0 (2.6.8) 2022-09-13 98d69c9461 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-linux]"}
Sep 28 12:47:21 log.in.domain.com logstash[5052]: [2022-09-28T12:47:21,346][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
Sep 28 12:47:22 log.in.domain.com logstash[5052]: [2022-09-28T12:47:22,270][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
Sep 28 12:47:23 log.in.domain.com logstash[5052]: [2022-09-28T12:47:23,050][INFO ][org.reflections.Reflections] Reflections took 79 ms to scan 1 urls, producing 125 keys and 434 values
Sep 28 12:47:23 log.in.domain.com logstash[5052]: [2022-09-28T12:47:23,589][INFO ][logstash.codecs.jsonlines] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
Sep 28 12:47:23 log.in.domain.com logstash[5052]: [2022-09-28T12:47:23,606][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,010][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://localhost:9200"]}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,034][WARN ][logstash.outputs.elasticsearch][main] You have enabled encryption but DISABLED certificate verification, to make sure your data is secure remove `ssl_certificate_verification => false`
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,171][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://logstash_internal:xxxxxx@localhost:9200/]}}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,391][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https://logstash_internal:xxxxxx@localhost:9200/"}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,402][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.4.2) {:es_version=>8}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,403][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,434][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,435][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,436][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,453][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,468][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
Sep 28 12:47:24 log.in.domain.com logstash[5052]: [2022-09-28T12:47:24,585][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>100, "pipeline.batch.delay"=>100, "pipeline.max_inflight"=>400, "pipeline.sources"=>["/etc/logstash/conf.d/alerts.conf", "/etc/logstash/conf.d/rsyslog.conf"], :thread=>"#<Thread:0x7b46bade run>"}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,166][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.58}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,359][INFO ][logstash.inputs.file ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sincedb_f5fdf6ea0ea92860c6a6b2b354bfcbbc", :path=>["/var/log/syslog"]}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,371][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,376][INFO ][logstash.inputs.tcp ][main][14159d240d6838d914b60243d0fd4862be6731410144d8d4adb36e30f7b04f36] Starting tcp input listener {:address=>"0.0.0.0:5000", :ssl_enable=>true}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,403][INFO ][filewatch.observingtail ][main][26e5ba772e50eef55a410157b3435109b8c589bb605a5c47f124662d38e8f3ff] START, creating Discoverer, Watch with file and sincedb collections
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,426][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Sep 28 12:47:25 log.in.domain.com logstash[5052]: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/manticore-0.9.1-java/lib/manticore/client.rb:527: warning: already initialized constant Manticore::Client::HttpEntityEnclosingRequestBase
Sep 28 12:47:25 log.in.domain.com logstash[5052]: /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/manticore-0.9.1-java/lib/manticore/client.rb:536: warning: already initialized constant Manticore::Client::StringEntity
Sep 28 12:47:25 log.in.domain.com logstash[5052]: [2022-09-28T12:47:25,869][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:47:35 log.in.domain.com rsyslogd: action 'action-8-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.2112.0 try https://www.rsyslog.com/e/2359 ]
Sep 28 12:47:36 log.in.domain.com logstash[5052]: [2022-09-28T12:47:36,108][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:48:37 log.in.domain.com rsyslogd: action 'action-8-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.2112.0 try https://www.rsyslog.com/e/2359 ]
Sep 28 12:48:37 log.in.domain.com rsyslogd: main Q:Reg: worker thread 55729c6565a0 terminated, now 1 active worker threads [v8.2112.0 try https://www.rsyslog.com/e/2439 ]
Sep 28 12:50:55 log.in.domain.com logstash[5052]: [2022-09-28T12:50:55,474][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:50:55 log.in.domain.com logstash[5052]: [2022-09-28T12:50:55,486][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:50:55 log.in.domain.com logstash[5052]: [2022-09-28T12:50:55,694][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:50:55 log.in.domain.com logstash[5052]: [2022-09-28T12:50:55,783][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:51:20 log.in.domain.com logstash[5052]: [2022-09-28T12:51:20,410][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:51:20 log.in.domain.com logstash[5052]: [2022-09-28T12:51:20,410][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:51:20 log.in.domain.com logstash[5052]: [2022-09-28T12:51:20,619][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:51:20 log.in.domain.com logstash[5052]: [2022-09-28T12:51:20,806][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:51:40 log.in.domain.com logstash[5052]: [2022-09-28T12:51:40,429][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:51:40 log.in.domain.com logstash[5052]: [2022-09-28T12:51:40,429][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:51:40 log.in.domain.com logstash[5052]: [2022-09-28T12:51:40,638][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:51:40 log.in.domain.com logstash[5052]: [2022-09-28T12:51:40,825][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:52:00 log.in.domain.com logstash[5052]: [2022-09-28T12:52:00,447][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:52:00 log.in.domain.com logstash[5052]: [2022-09-28T12:52:00,447][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:52:00 log.in.domain.com logstash[5052]: [2022-09-28T12:52:00,655][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:52:00 log.in.domain.com logstash[5052]: [2022-09-28T12:52:00,843][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:52:25 log.in.domain.com logstash[5052]: [2022-09-28T12:52:25,384][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:52:25 log.in.domain.com logstash[5052]: [2022-09-28T12:52:25,385][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:52:25 log.in.domain.com logstash[5052]: [2022-09-28T12:52:25,593][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:52:25 log.in.domain.com logstash[5052]: [2022-09-28T12:52:25,873][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:52:50 log.in.domain.com logstash[5052]: [2022-09-28T12:52:50,399][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:52:50 log.in.domain.com logstash[5052]: [2022-09-28T12:52:50,399][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:52:50 log.in.domain.com logstash[5052]: [2022-09-28T12:52:50,608][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:52:50 log.in.domain.com logstash[5052]: [2022-09-28T12:52:50,899][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:53:10 log.in.domain.com logstash[5052]: [2022-09-28T12:53:10,420][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:53:10 log.in.domain.com logstash[5052]: [2022-09-28T12:53:10,420][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:53:10 log.in.domain.com logstash[5052]: [2022-09-28T12:53:10,628][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:53:10 log.in.domain.com logstash[5052]: [2022-09-28T12:53:10,918][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:53:30 log.in.domain.com logstash[5052]: [2022-09-28T12:53:30,439][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:53:30 log.in.domain.com logstash[5052]: [2022-09-28T12:53:30,440][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:53:30 log.in.domain.com logstash[5052]: [2022-09-28T12:53:30,720][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:53:30 log.in.domain.com logstash[5052]: [2022-09-28T12:53:30,938][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:53:55 log.in.domain.com logstash[5052]: [2022-09-28T12:53:55,448][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:53:55 log.in.domain.com logstash[5052]: [2022-09-28T12:53:55,449][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:53:55 log.in.domain.com logstash[5052]: [2022-09-28T12:53:55,656][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:53:55 log.in.domain.com logstash[5052]: [2022-09-28T12:53:55,962][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:54:20 log.in.domain.com logstash[5052]: [2022-09-28T12:54:20,387][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:54:20 log.in.domain.com logstash[5052]: [2022-09-28T12:54:20,387][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:54:20 log.in.domain.com logstash[5052]: [2022-09-28T12:54:20,595][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:54:20 log.in.domain.com logstash[5052]: [2022-09-28T12:54:20,984][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:54:40 log.in.domain.com logstash[5052]: [2022-09-28T12:54:40,404][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:54:40 log.in.domain.com logstash[5052]: [2022-09-28T12:54:40,405][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:54:40 log.in.domain.com logstash[5052]: [2022-09-28T12:54:40,613][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:54:41 log.in.domain.com logstash[5052]: [2022-09-28T12:54:41,002][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:55:00 log.in.domain.com logstash[5052]: [2022-09-28T12:55:00,424][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:55:00 log.in.domain.com logstash[5052]: [2022-09-28T12:55:00,424][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:55:00 log.in.domain.com logstash[5052]: [2022-09-28T12:55:00,633][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:55:01 log.in.domain.com logstash[5052]: [2022-09-28T12:55:01,022][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:55:20 log.in.domain.com logstash[5052]: [2022-09-28T12:55:20,446][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:55:20 log.in.domain.com logstash[5052]: [2022-09-28T12:55:20,446][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:55:20 log.in.domain.com logstash[5052]: [2022-09-28T12:55:20,655][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:55:21 log.in.domain.com logstash[5052]: [2022-09-28T12:55:21,040][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
Sep 28 12:55:40 log.in.domain.com logstash[5052]: [2022-09-28T12:55:40,462][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip
Sep 28 12:55:40 log.in.domain.com logstash[5052]: [2022-09-28T12:55:40,462][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Closing file /log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip
Sep 28 12:55:40 log.in.domain.com logstash[5052]: [2022-09-28T12:55:40,671][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/2022.09.28/log.in.domain.com/log.in.domain.com-127.0.0.1.gzip"}
Sep 28 12:55:41 log.in.domain.com logstash[5052]: [2022-09-28T12:55:41,059][INFO ][logstash.outputs.file ][main][d59951eb0edc4a45039c963d8469646951119fceecb121e27584384be9c567d0] Opening file {:path=>"/log/%{log_day}/%{syslog_hostname}/%{syslog_hostname}-%{ip}.gzip"}
As you can see some times it loads data to corect index/file and sometimes not.
Anyone has an idea what is causing this?