# Using my log timestamp in logstash sample log (first-pipeline.conf)

**URL:** <https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854>\
**Category:** Logstash\
**Created:** [April 27, 2017, 12:02pm UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854 "2017-04-27T12:02:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![hasan\_sharekhan](https://avatars.discourse-cdn.com/v4/letter/h/eada6e/32.png) [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\
**Post date:** [April 27, 2017, 12:02pm UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/1 "2017-04-27T12:02:36Z")

</div>

I have just successfully completed first example (first-pipeline.conf) from Logstash tutorial by uploading logs from FileBeat to Logstash to ElasticSearch

In Kibana while creating new Index "logstash-\*" I am not getting timestamp fields from logs instead i am getting @timestamp which actual log uploading time.

Links used

Sample Apache Logs used:  
[https://download.elastic.co/demos/logstash/gettingstarted/logstash-tutorial.log.gz](https://download.elastic.co/demos/logstash/gettingstarted/logstash-tutorial.log.gz)

Parsing Logs with Logstashed:  
[https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html](https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html)

---

<div class="post-metadata">

**Author:** ![Xavy](https://avatars.discourse-cdn.com/v4/letter/x/e0b2c6/32.png) [@Xavy](https://discuss.elastic.co/u/Xavy)\
**Post date:** [April 27, 2017, 2:50pm UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/2 "2017-04-27T14:50:34Z")

</div>

Hello:

This is normal and is seen on the doc you're mentioning.

If you want to match the log date + time fields, I think that you might need using a grok patter to match the date + time piece on each log entry, and map it to a particular field of your choice

---

<div class="post-metadata">

**Author:** ![hasan\_sharekhan](https://avatars.discourse-cdn.com/v4/letter/h/eada6e/32.png) [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\
**Post date:** [April 28, 2017, 7:23am UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/3 "2017-04-28T07:23:05Z")

</div>

Thanks, I am using the first-pipeline.conf as given on  
[https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html](https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html)

input {  
beats {  
port =\> "5043"  
}  
}

filter {  
grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}"}  
}

```
geoip {
    source => "clientip"
}

```

}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
}  
}

In filter section I am using "%{COMBINEDAPACHELOG} as given in example/doc, isn't it enough or do i need to make changes in filter { } section for each and every field, since it is Apache web Logs and logstash understand it the format and it believe it is enough

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2017, 7:32am UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/4 "2017-04-28T07:32:25Z")

</div>

You need a date filter. See [https://www.elastic.co/guide/en/logstash/current/config-examples.html#\_processing\_apache\_logs](https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_apache_logs).

---

<div class="post-metadata">

**Author:** ![hasan\_sharekhan](https://avatars.discourse-cdn.com/v4/letter/h/eada6e/32.png) [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\
**Post date:** [May 2, 2017, 11:37am UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/5 "2017-05-02T11:37:28Z")

</div>

Thanks, its working,  
Now I want to use same first-pipeline.conf file to parse my application logs which looks like this

Each line is divided in 3 parts  
a) Date  
b) LogType  
c) Message

My first query is how to I parse it in 3 parts  
and second is how to I parse message of every line which is different.

2017-04-16 04:17:24.497+05:30 [I] " Data: ABC Server started.."  
2017-04-16 04:17:35.606+05:30 [D] " Data: XYZ List Generation - Start"  
2017-04-16 04:18:15.309+05:30 [D] " Data: Restricted User List Generated. MaxTime [4/3/2017 12:32:24 PM]"  
2017-04-16 04:18:20.106+05:30 [I] " Data: Normal Order Enabled in : MY\_SERVER\_A"  
2017-04-16 04:18:20.841+05:30 [D] " Data: BulkOrderHome is Created Successfully..."  
2017-04-16 04:18:22.778+05:30 [D] " Data: Bulk Report Disabled in : MY\_SERVER\_B"  
2017-04-16 05:46:13.466+05:30 [D] " Data: Logged In Clients,0,Total Clients,0,P.Send Q,0,S.Send Q,0,Ack Size 0"  
2017-04-19 06:12:56.312+05:30 [I] " Data: Holiday Master Loaded from 20170423 to 20171230"  
2017-04-19 10:00:00.609+05:30 [I] " Data: GetLastSentDataSize: 12511212"  
2017-04-19 10:00:02.546+05:30 [I] " Data: General Process Q : R=t|L=INDIA123"

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2017, 11:38am UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854/6 "2017-05-30T11:38:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.