# @timestamp field not matching with the Actual log field

**URL:** https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343
**Category:** Logstash
**Created:** [May 7, 2015, 1:49pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343 "2015-05-07T13:49:05Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 7, 2015, 1:49pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/1 "2015-05-07T13:49:05Z")

</div>

Hi,

I'm new to ELK, Please help me in getting out of this issue.  
I have logs having a "AdmitDate" field. @timestamp field is showing the date and time which I have uploaded the logs.The requirement is, @timestamp field must contain the date and time same as "AdmitDate" field.  
For this I have changed the logstash.conf file as below:  
input  
{  
file  
{  
type =\> "csv"  
tags =\> "csv"  
path =\>["Q:/sanjay/Splunk/DB/out\_hemas\_test.csv"]  
start\_position =\> "beginning"   
}  
}  
filter  
{  
csv  
{  
columns =\> ["AdmitDate","IPID","AdmissionNumber","PatientID","RegCode","FirstName","Middlename","PatientName","Age","Gender"]  
separator =\> ","  
}

date  
{ locale =\> "en"  
match =\> ["AdmitDate", "YYYY-MM-DD HH:mm:ss.SSS"]  
}  
}  
output  
{  
elasticsearch\_http  
{  
host =\> "localhost"  
index =\> "logstash-%{+YYYY.MM.dd}"  
manage\_template =\> false  
template\_name =\> "logstash"   
}  
stdout{codec =\> json}  
}

Is there any modifications required in logstash.conf file??

Please help.

Thanks

---

<div class="post-metadata">

### Author: ![nellicus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nellicus/32/51566_2.png) [@nellicus](https://discuss.elastic.co/u/nellicus)
#### Post date: [May 7, 2015, 2:10pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/2 "2015-05-07T14:10:20Z")

</div>

That looks right to me... are you sure the date format matches?

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 7, 2015, 2:21pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/3 "2015-05-07T14:21:32Z")

</div>

Thanks for the quick response,

I have changed the date format in KIBANA according to the date format I have in the logs.

![](https://sea2.discourse-cdn.com/elastic/uploads/default/146/819665795d01c946.png)

Don't know what to do:(

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 7, 2015, 2:31pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/4 "2015-05-07T14:31:47Z")

</div>

> I have logs having a "AdmitDate" field. @timestamp field is showing the date and time which I have uploaded the logs.The requirement is, @timestamp field must contain the date and time same as "AdmitDate" field.

Please give an example input line.

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 7, 2015, 2:41pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/5 "2015-05-07T14:41:25Z")

</div>

@magnusbaeck

please find the screenshot of the input logs below

![](https://sea2.discourse-cdn.com/elastic/uploads/default/147/73153bbf80353d58.png)

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 8, 2015, 5:22am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/6 "2015-05-08T05:22:06Z")

</div>

@magnusbaeck

The input line from the logs is given below:

AdmitDate, IPID , AdmissionNumber, PatientID , RegCode , FirstName , Middlename   
2014-08-09 17:20:43.530 , 381385, OPD.00144616 , 220946, WTID.0000095414 , MANOJA, UDAYANGANI

![](https://sea2.discourse-cdn.com/elastic/uploads/default/166/29ed2b7c3e47dd26.PNG)

What should I do to get the @timestamp field same as the "AdmitDate"

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 8, 2015, 8:22am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/7 "2015-05-08T08:22:37Z")

</div>

> [@sanju1323](#):
>
> match =\> ["AdmitDate", "YYYY-MM-DD HH:mm:ss.SSS"]

I think the pattern is failing as 'DD' represents day of the year while 'dd' corresponds to day of the month. Try using the pattern 'YYYY-MM-dd HH:mm:ss.SSS' instead and see if that helps.

Christian

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 8, 2015, 9:22am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/8 "2015-05-08T09:22:16Z")

</div>

Hi @Christian_Dahlqvist

Even after changing the format to 'YYYY-MM-dd HH:mm:ss.SSS', facing the same problem. While uploading the logs the following error is popping up

![](https://sea2.discourse-cdn.com/elastic/uploads/default/172/2917e0417d16d3c6.png)

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 8, 2015, 9:55am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/9 "2015-05-08T09:55:01Z")

</div>

The AdmitDate field appears to have a trailing space.

Please copy/paste text when you can instead of inserting screenshots.

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 8, 2015, 10:15am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/10 "2015-05-08T10:15:33Z")

</div>

Hi @magnusbaeck

please find the sample log file.

AdmitDate , IPID , AdmissionNumber , PatientID , RegCode   
2014-08-09 17:20:43.530 , 381385 , OPD.00144616 , 220946, WTID.0000095414   
2014-08-09 17:22:40.820 , 381388, OPD.00144619, 71303, WTID.0000011478   
2014-08-09 17:23:28.113 , 381391 , OPD.00144620 , 217521, WTID.0000093720  
2014-08-09 17:30:39.657 , 381409 , OPD.00144631 , 54810, WTID.0000000270

I'm not able to insert all the fields due to space constraint ☹

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 8, 2015, 10:38am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/11 "2015-05-08T10:38:03Z")

</div>

And what if you remove the trailing space with e.g. the [mutate filter's gsub option](http://logstash.net/docs/1.4.2/filters/mutate#gsub)?

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 8, 2015, 11:21am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/12 "2015-05-08T11:21:45Z")

</div>

@magnusbaeck please let me know, what should I replace with the trailing space in the AdmitDate

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 8, 2015, 11:24am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/13 "2015-05-08T11:24:58Z")

</div>

You should delete the trailing space.

```
mutate {
  gsub => ["AdmitDate", "\s+$", ""]
}
```

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 8, 2015, 12:05pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/14 "2015-05-08T12:05:16Z")

</div>

@magnusbaeck

I have done the same. But the following error came while uploading

←[33mFailed parsing date from field {:field=\>"AdmitDate", :value=\>" 2014-08-09 1  
8:24:20.667 ", :exception=\>java.lang.IllegalArgumentException: Invalid format: "  
2014-08-09 18:24:20.667 ", :level=\>:warn}←[0m  
←[33mFailed parsing date from field {:field=\>"AdmitDate", :value=\>" 2014-08-09 1  
8:31:31.140 ", :exception=\>java.lang.IllegalArgumentException: Invalid format: "  
2014-08-09 18:31:31.140 ", :level=\>:warn}←[0m  
←[33mFailed parsing date from field {:field=\>"AdmitDate", :value=\>" 2014-08-09 1  
8:38:50.040 ", :exception=\>java.lang.IllegalArgumentException: Invalid format: "  
2014-08-09 18:38:50.040 ", :level=\>:warn}←[0m  
←[33mFailed parsing date from field {:field=\>"AdmitDate", :value=\>" 2014-08-09 1  
8:39:36.393 ", :exception=\>java.lang.IllegalArgumentException: Invalid format: "  
2014-08-09 18:39:36.393 ", :level=\>:warn}←[0m  
←[33mFailed parsing date from field {:field=\>"AdmitDate", :value=\>" 2014-08-09 1  
8:52:30.603 ", :exception=\>java.lang.IllegalArgumentException: Invalid format: "  
2014-08-09 18:52:30.603 ", :level=\>:warn}←[0m

Logs are uploaded successfully, but the @timestamp remains same... ☹a

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 8, 2015, 12:16pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/15 "2015-05-08T12:16:17Z")

</div>

But you still have the trailing whitespace, but now I see you also have leading whitespace in the AdmitDate field. I suggest you hand-edit one line in the source file and trim the whitespace and retry. If we can confirm that it fixes the problem with can dig into the whitespace again.

---

<div class="post-metadata">

### Author: ![sanju1323](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@sanju1323](https://discuss.elastic.co/u/sanju1323)
#### Post date: [May 20, 2015, 11:07am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/16 "2015-05-20T11:07:06Z")

</div>

Hi,  
my problem solved by using the following grok command.  
grok  
{  
match =\> ["AdmitDate", "%{YEAR:al\_year}-%{MONTHNUM:al\_month}-%{MONTHDAY:al\_monthday}  
}

thanks @Christian_Dahlqvist @magnusbaeck for the help 😄

---

<div class="post-metadata">

### Author: ![Yashpal\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yashpal_singh/32/43209_2.png) [@Yashpal\_Singh](https://discuss.elastic.co/u/Yashpal_Singh)
#### Post date: [August 29, 2015, 8:49am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/17 "2015-08-29T08:49:33Z")

</div>

Hi All,  
I am also facing the same problem Logstash is not considering my log date rather making the date at which I am forwarding the logs as @timeStamp field.

@sanju1323 could you please help me in generating the filter as you have did for your date

currently I am using

date {  
match =\> ["logdate", "YYYY-MM-dd HH:mm:ss,SSS Z"]

```
    }
```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [August 29, 2015, 3:25pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/18 "2015-08-29T15:25:17Z")

</div>

Please start a new topic when asking a new question.

---

<div class="post-metadata">

### Author: ![Yashpal\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yashpal_singh/32/43209_2.png) [@Yashpal\_Singh](https://discuss.elastic.co/u/Yashpal_Singh)
#### Post date: [August 30, 2015, 4:30pm UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/19 "2015-08-30T16:30:17Z")

</div>

Hi @magnusbaeck ,

Will start new discussion next time onwards but as I am facing the similar problem as @sanju1323 So I have pasted my problem on this thread.

---

<div class="post-metadata">

### Author: ![Mahesh\_Dn](https://avatars.discourse-cdn.com/v4/letter/m/f475e1/32.png) [@Mahesh\_Dn](https://discuss.elastic.co/u/Mahesh_Dn)
#### Post date: [September 10, 2015, 5:59am UTC](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343/20 "2015-09-10T05:59:19Z")

</div>

Solution for this Issue is as shown in below Image.

[Next page](https://discuss.elastic.co/t/timestamp-field-not-matching-with-the-actual-log-field/343.md?page=2)
