# Filebeat add system time in prospectors field

**URL:** https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913
**Category:** Logstash
**Created:** [September 9, 2017, 3:47am UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913 "2017-09-09T03:47:51Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Gangadhar\_Mahadevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gangadhar_mahadevan/32/62579_2.png) [@Gangadhar\_Mahadevan](https://discuss.elastic.co/u/Gangadhar_Mahadevan)
#### Post date: [September 9, 2017, 3:47am UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/1 "2017-09-09T03:47:51Z")

</div>

Filebeat version 5.2

Is there a way to add system current time (using date command) as a custom field in filebeat prosectors? I am able to add hardcoded values in custom fields but am looking for gathering system time as custom fields to capture at what time filebeat ships the log messages.

Something like

## filebeat\_prospectors:

path : /var/log/syslog  
fields:  
source: filebeat  
time: {current time}

Thanks!!

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [September 11, 2017, 11:38am UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/2 "2017-09-11T11:38:12Z")

</div>

The `@timestamp` field is exactly the time filebeat did read the log line. Lines might be buffered for unknown time, due to backpressure. E.g. in Logstash you can get the arrival time and this way compute how long it took for the event being ready for shipping until the event is finally processed by Logstash.

---

<div class="post-metadata">

### Author: ![Gangadhar\_Mahadevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gangadhar_mahadevan/32/62579_2.png) [@Gangadhar\_Mahadevan](https://discuss.elastic.co/u/Gangadhar_Mahadevan)
#### Post date: [September 11, 2017, 10:26pm UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/3 "2017-09-11T22:26:40Z")

</div>

This is sample message that has @timestamp when I read from kafka topic. I tried to extract @timestamp in different logstash filters but was unsuccessful.  
Sample message:  
{"@timestamp":"2017-09-11T21:17:04.020Z","beat":{"hostname":"ossmlpadc0101a","name":"ossmlpadc0101a","version":"5.2.0"},"fields":{"@timestamp":"2017-09-11T21:17:02.778Z","beat":{"hostname":"xxxx,"name":"xxxx","version":"5.2.0"},"fields":{"format":"syslog","source":"filebeat"},"input\_type":"log","source":"/var/log/syslog","type":"local-dev-vergil-adc01-rg-  
syslog

Tried ruby plugin answer from [Get current time using ruby filter in logstash](https://discuss.elastic.co/t/get-current-time-using-ruby-filter-in-logstash/90084/8)

Also Tried date filter plugin and didn't help.

The error message is  
Caused by: java.lang.IllegalStateException: Pattern does not contain a date  
at org.apache.logging.log4j.core.appender.rolling.PatternProcessor.getNextTime(PatternProcessor.java:119)  
at org.apache.logging.log4j.core.appender.rolling.TimeBasedTriggeringPolicy.initialize(TimeBasedTriggeringPolicy.java:59)  
at org.apache.logging.log4j.core.appender.rolling.CompositeTriggeringPolicy.initialize(CompositeTriggeringPolicy.java:49)  
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.initialize(RollingFileManager.java:82)  
at org.apache.logging.log4j.core.appender.RollingFileAppender.createAppender(RollingFileAppender.java:197)  
... 108 more

2017-09-11 17:14:19,755 main ERROR Null object returned for RollingFile in Appenders.

Am trying to find how long it took filebeat to ship message and logstash to process and forward to Elasticsearch or another kafka topic. Thanks!!

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [September 12, 2017, 12:35pm UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/4 "2017-09-12T12:35:55Z")

</div>

I moved this topic to the Logstash forum. Problem is about configuring logstash to parse and process timestamps.

---

<div class="post-metadata">

### Author: ![Gangadhar\_Mahadevan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gangadhar_mahadevan/32/62579_2.png) [@Gangadhar\_Mahadevan](https://discuss.elastic.co/u/Gangadhar_Mahadevan)
#### Post date: [September 12, 2017, 6:53pm UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/5 "2017-09-12T18:53:11Z")

</div>

Thanks!! Anyone from logstash team can help with this? This seems pretty simple but for some reasons whatever methods I try it says illegal date pattern

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 10, 2017, 6:53pm UTC](https://discuss.elastic.co/t/filebeat-add-system-time-in-prospectors-field/99913/6 "2017-10-10T18:53:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
