# Filebeat not respect order of file?

**URL:** https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011
**Category:** Beats
**Tags:** filebeat
**Created:** [October 19, 2022, 1:54pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011 "2022-10-19T13:54:03Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Nikolas1306](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolas1306/32/111507_2.png) [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)
#### Post date: [October 19, 2022, 1:54pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/1 "2022-10-19T13:54:03Z")

</div>

filebeat at indexed at last to "server.log.2022-07-27"

but the last is server.log why?

not respect the name of file but the size of file?

```auto
-rw-r--r--. 1 wildfly wildfly 0 Apr 20 18:25 audit.log
-rw-r--r--. 1 wildfly wildfly 2308 Oct 12 16:24 server.log
-rw-r--r--. 1 wildfly wildfly 24631718 Jun 23 12:50 server.log.2022-06-23
-rw-r--r--. 1 wildfly wildfly 39545 Jun 24 13:05 server.log.2022-06-24
-rw-r--r--. 1 wildfly wildfly 104888 Jul 8 15:40 server.log.2022-07-08
-rw-r--r--. 1 wildfly wildfly 109697 Jul 11 18:18 server.log.2022-07-11
-rw-r--r--. 1 wildfly wildfly 10299 Jul 14 18:17 server.log.2022-07-14
-rw-r--r--. 1 wildfly wildfly 1349 Jul 15 14:07 server.log.2022-07-15
-rw-r--r--. 1 wildfly wildfly 17535 Jul 18 18:43 server.log.2022-07-18
-rw-r--r--. 1 wildfly wildfly 4047 Jul 19 15:00 server.log.2022-07-19
-rw-r--r--. 1 wildfly wildfly 58817 Jul 21 17:29 server.log.2022-07-21
-rw-r--r--. 1 wildfly wildfly 708 Jul 22 14:49 server.log.2022-07-22
-rw-r--r--. 1 wildfly wildfly 29477939 Jul 27 18:18 server.log.2022-07-27
-rw-r--r--. 1 wildfly wildfly 39149 Jul 29 00:34 server.log.2022-07-29
-rw-r--r--. 1 wildfly wildfly 40050 Aug 23 14:14 server.log.2022-08-23
-rw-r--r--. 1 wildfly wildfly 26700 Aug 25 16:41 server.log.2022-08-25
-rw-r--r--. 1 wildfly wildfly 4203 Aug 26 12:32 server.log.2022-08-26
-rw-r--r--. 1 wildfly wildfly 78902 Aug 30 16:16 server.log.2022-08-30
-rw-r--r--. 1 wildfly wildfly 6300 Aug 31 18:02 server.log.2022-08-31
-rw-r--r--. 1 wildfly wildfly 90223 Sep 2 23:57 server.log.2022-09-02
-rw-r--r--. 1 wildfly wildfly 2041 Sep 12 10:54 server.log.2022-09-12
-rw-r--r--. 1 wildfly wildfly 532268 Sep 30 14:53 server.log.2022-09-30
-rw-r--r--. 1 wildfly wildfly 163192 Oct 4 18:28 server.log.2022-10-04
-rw-r--r--. 1 wildfly wildfly 6720 Oct 8 19:36 server.log.2022-10-08
-rw-r--r--. 1 wildfly wildfly 86430 Oct 11 23:36 server.log.2022-10-11

```

---

<div class="post-metadata">

### Author: ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)
#### Post date: [October 19, 2022, 5:13pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/2 "2022-10-19T17:13:24Z")

</div>

Hi @Nikolas1306 ,

the order Filebeat will ingest the files is not defined, you can use [processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) and [parsers](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#_parsers) on Filebeat, [ingest pipelines](https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ingest-node.html) on Elasticsearch or even define a different field as the timestamp field when creating your [data views on Kibana](https://www.elastic.co/guide/en/kibana/current/data-views.html) to ensure the logs indexed/searched/shown in the correct order.

---

<div class="post-metadata">

### Author: ![Nikolas1306](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolas1306/32/111507_2.png) [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)
#### Post date: [October 19, 2022, 5:16pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/3 "2022-10-19T17:16:48Z")

</div>

ok i need create and other timestamp field with extract datatime from log and use for order

example in logstash

```auto
grok {
   match => ["message", "%{TIMESTAMP_ISO8601:logmessage}"]
 }

      date {
          
              match => ["logmessage", "%{TIMESTAMP_ISO8601}"]
              
              target => "timestamp_log"
           }
}

```

but this cause crash

```auto

[2022-10-19T17:15:14,776][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:terza, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: Illegal pattern component: T", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:85)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:846)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1229)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1202)", "org.jruby.ir.targets.indy.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:29)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:381)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.Block.call(Block.java:143)", "org.jruby.RubyProc.call(RubyProc.java:309)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:107)", "java.base/java.lang.Thread.run(Thread.java:833)"]}

```

---

<div class="post-metadata">

### Author: ![Nikolas1306](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolas1306/32/111507_2.png) [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)
#### Post date: [October 19, 2022, 5:23pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/4 "2022-10-19T17:23:16Z")

</div>

if change the config with

```auto
grok {
   match => ["message", "%{TIMESTAMP_ISO8601:logmessage}"]
 }

      date {
              match => ["logmessage", "yyyy-MM-dd HH:mm:ss%"]
                      
              target => "report_date"
           }
}

```

i have the field logmessage but is string not datatime

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/e/deb2cf19e9acd55b5d0f063be5bfe2506f1d4606.png)

---

<div class="post-metadata">

### Author: ![Nikolas1306](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolas1306/32/111507_2.png) [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)
#### Post date: [October 19, 2022, 7:06pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/5 "2022-10-19T19:06:58Z")

</div>

```auto
filter {

   grok {
           match => ["message", "%{TIMESTAMP_ISO8601:logmessage}"]
     }

   mutate {
            remove_field => ["[event][original]","[agent][id]","[agent][ephemeral_id]"]
    }
     

     date {
	         match => ["logmessage", "ISO8601", "YYYY-MM-dd HH:mm:ss"]
	         target => "logdate"
	         locale => "en"
        }

}

```

this is ok sorry

---

<div class="post-metadata">

### Author: ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)
#### Post date: [October 20, 2022, 11:10am UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/6 "2022-10-20T11:10:02Z")

</div>

If I understood your last message correctly, you got it working, is that right?

---

<div class="post-metadata">

### Author: ![Nikolas1306](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikolas1306/32/111507_2.png) [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)
#### Post date: [October 20, 2022, 12:35pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/7 "2022-10-20T12:35:00Z")

</div>

> [@TiagoQueiroz](#):
>
> Se ho capito bene il tuo ultimo messaggio, hai funzionato, è giusto?

yes sir i change match in date and is ok

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 17, 2022, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-not-respect-order-of-file/317011/8 "2022-11-17T14:35:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
