# Setting @timestamp in filebeat

**URL:** <https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 17, 2018, 10:49pm UTC](https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224 "2018-06-17T22:49:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![michas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michas/32/32355_2.png) [@michas](https://discuss.elastic.co/u/michas)\
**Post date:** [June 17, 2018, 10:49pm UTC](https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224/1 "2018-06-17T22:49:07Z")

</div>

Recent versions of filebeat allow to [dissect](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html) log messages directly. (Without the need of logstash or an ingestion pipeline.)  
Therefore I would like to avoid any overhead and send the dissected fields directly to ES.

Currently I have two timestamps, @timestamp containing the processing time, and my parsed timestamp containing the actual event time.

Is it possible to set @timestamp directly to the parsed event time?  
(Or is there a good reason, why this would be a bad idea?)

* * *

For reference, this is my current config.

filebeat.yml:

```
filebeat.inputs:
- type: log
  tags: ["ingestion"]
  multiline.pattern: '^\d{4}-\d{2}-\d{2}T'
  multiline.negate: true
  multiline.match: after
  paths:
  - '/druid/var/druid/task/*/log'

processors:
- dissect:
    tokenizer: "%{timestamp} %{loglevel} [%{component}] %{class} - %{message}"
    field: "message"
    target_prefix: "ingest"
- dissect:
    tokenizer: "/druid/var/druid/task/%{task-id}/log"
    field: "source"
    target_prefix: "ingest"
- include_fields:
    fields: ["ingest", "message"]

output.elasticsearch:
  hosts: ["${ES_URL}"]
  index: "ingest-%{+yyyy.MM.dd}"

setup.template.enabled: true
setup.template.overwrite: true
setup.template.name: "ingest"
setup.template.pattern: "ingest-*"
setup.template.fields: "fields.yml"

```

fields.yml:

```
- name: main
  type: group
  description: >
    What is this additional main level good for?
  fields:
  - name: ingest
    type: group
    description: >
      Parsed values of the ingestion tasks.
    fields:
    - name: timestamp
      type: date
      description: >
        The actual timestamp.
    - name: loglevel
      type: keyword
      description: >
        The loglevel.
    - name: message
      type: text
      description: >
        The actual message.
```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [June 19, 2018, 2:45pm UTC](https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224/2 "2018-06-19T14:45:41Z")

</div>

At the current time it's not possible to change the `@timestamp` via `dissect` or even `rename`. See [https://github.com/elastic/beats/issues/7351](https://github.com/elastic/beats/issues/7351).

In the meantime you could use an Ingest Node pipeline to parse the timestamp. See [https://www.elastic.co/guide/en/elasticsearch/reference/master/date-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/date-processor.html). Then once you have created the pipeline in Elasticsearch you will add [`pipeline: my-pipeline-name`](https://www.elastic.co/guide/en/beats/filebeat/6.3/filebeat-input-log.html#_literal_pipeline_literal) to your Filebeat input config so that data from that input is routed to the Ingest Node pipeline.

---

<div class="post-metadata">

**Author:** ![michas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michas/32/32355_2.png) [@michas](https://discuss.elastic.co/u/michas)\
**Post date:** [June 20, 2018, 2:10pm UTC](https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224/3 "2018-06-20T14:10:27Z")

</div>

The charm of the above solution is, that filebeat itself is able to set up everything needed. And all the parsing logic can easily be located next to the application producing the logs.

As soon as I need to reach out and configure logstash or an ingestion node, then I can probably also do dissection there and there. Guess an option to set `@timestamp` directly in filebeat would be really go well with the new dissect processor.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2018, 2:10pm UTC](https://discuss.elastic.co/t/setting-timestamp-in-filebeat/136224/4 "2018-07-18T14:10:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.