# How to parse logs in same order as in log files

**URL:** https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982
**Category:** Kibana
**Created:** [May 31, 2018, 6:57am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982 "2018-05-31T06:57:17Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![yoga-sara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoga-sara/32/27541_2.png) [@yoga-sara](https://discuss.elastic.co/u/yoga-sara)
#### Post date: [May 31, 2018, 6:57am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/1 "2018-05-31T06:57:17Z")

</div>

I configured my ELK setup with 1 index.It collects logs from 25 machines. The average log event count is 2.5 lacs/3 hrs. Every 5 min 5000 event parsing sometimes it increase.

My configuration like Filebeat =\> Logstash =\> Elasticsearch(single cluster with 1 node) =\> Kibana

I have event order mismatch problem.How to resolve it.  
Thanks in advance.

This is my actual order of logs:

```
       [[0m^[[0m04:07:34,117 INFO [com. .knowledgebase.ejb.KnowledgeBaseSingleton] (ServerService Thread Pool -- 62) 1 of 3 KnowledgeBase::26_ _UI_WF Created in serverGroup::26_ _RRFW
    
      [[0m^[[0m04:07:36,101 INFO [com. .knowledgebase.ejb.KnowledgeBaseSingleton] (ServerService Thread Pool -- 62) 2 of 3 KnowledgeBase::26_ _FORMS Created in serverGroup::26_ _RRFW
   
       [[0m^[[0m04:07:39,238 INFO [com. .knowledgebase.ejb.KnowledgeBaseSingleton] (ServerService Thread Pool -- 62) 3 of 3 KnowledgeBase::26_ _RATE_RRF Created in serverGroup::26_ _RRFW

```

I have attached screenshot of my misaligned logs.

 ![order-mismatched](https://us1.discourse-cdn.com/elastic/original/3X/6/3/63504bf6fbe7fa5398dfa6ba383e874252d7f70b.png)

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 31, 2018, 8:23am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/2 "2018-05-31T08:23:23Z")

</div>

You need to use the date filter in Logstash to make sure that the event timestamp is the main one in Elasticsearch.

---

<div class="post-metadata">

### Author: ![yoga-sara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoga-sara/32/27541_2.png) [@yoga-sara](https://discuss.elastic.co/u/yoga-sara)
#### Post date: [June 7, 2018, 10:23am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/3 "2018-06-07T10:23:59Z")

</div>

Hi warkolm,  
can you please give me an example?  
my logs time zone is different from my access location.

My event log separated by grok filter including time.

FYR,

```
  grok {
    match => [
                      "message", "%{TIME:time} %{LOGLEVEL:level}\s+\[%{JAVACLASS:class}\] \((?<thread>[^)]+)\) %{GREEDYDATA:message}",
                      "message", "%{NUMBER:procsec:float} %{IP:clientip} %{DATA:logname} %{DATA:user} \[%{HTTPDATE:time}\] %{DATA:method} %{DATA:requestpath} %{DATA:httpversion} %{DATA:respstatus} %{DATA:bytesent} ((- %{GREEDYDATA:cookie})|(%{GREEDYDATA:cookie}))"
                      ]
    overwrite => ["message"]
    tag_on_failure => ["JBOSSLG_grok"]
    add_tag => ["JBOSSLG"]
    }
```

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 7, 2018, 6:06pm UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/4 "2018-06-07T18:06:52Z")

</div>

Check out [https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-date.html](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-date.html), just make sure you set the target field to the `time` one you have groked out.

---

<div class="post-metadata">

### Author: ![yoga-sara](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoga-sara/32/27541_2.png) [@yoga-sara](https://discuss.elastic.co/u/yoga-sara)
#### Post date: [June 8, 2018, 6:25am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/5 "2018-06-08T06:25:06Z")

</div>

Hi Warkolm,  
My actual problem is I am not getting events with logs time order. It shows events with kibana UI's Timestamp.  
After I groked out, I used date filter with target time.

```
date {
   match => ["time", "HH:mm:ss.SSS"]
   target => "time"
      }

```

I have attached my logs. I marked my jboss logs timestamp with red which is in mismatched order.Suggest me to resolve this or give me an example for this.

 ![June-8-ordermismatch-whole](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b7bded486dc9e960922c77abb7cf37bfe1e437ff.png)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2018, 6:25am UTC](https://discuss.elastic.co/t/how-to-parse-logs-in-same-order-as-in-log-files/133982/6 "2018-07-06T06:25:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
