Log file's timestamp instead of kibana default timestamp

Elastic Stack Kibana
1

Hi,

I am connecting elastic search with kibana without logstash (elk version 7.2). I want to put the timestamp of my log file's instead of when it parsed in kibana.

Could someone help me to get the proper way to do it?

2

Hi,

Using grok you can parse the date field in the entry of the log and then using logstash you can transform this field to a date field. In this moment you can select this fields in Kibana to show the logs with the date field of the logs.

I hope i explained well :slight_smile:

Aleix

3

Thanks for you quick rply,

You meant to say that i have to use logstash in order to parse my log generated timestamp or in kibana configuration itself i can do it?

4

Hi Shree,

If you want to take the timestamp from the log, you have to parse the log. To parse the log, you have to use Logstash. In Kibana you can't do that sort of things.

Yours sincerely, Aleix.

5

Okay, but i want to try without logstash and i'm using filebeat, elasticsearch and kibana.

is there any way in filebeat that I can control the timestamp?

6

It's impossible to do this without logstash.

Filebeat => sends a copy of the log to the logstash or to the elasticsearch.
Logstash => is responsible of parsing the logs that come from filebeat and it's the only "program" that can parse the logs.
Elasticsearch => BBDD of the logs "parsed" with logstash or directly from filebeat without parse.
Kibana => Visualizer of the data from elasticsearch and you can make graphs or anything.

It's very easy to configure logstash and take the timestamp from the log. If you need help, i can help you :slight_smile:

Yours sincerely, Aleix.

7

Hi @Shree
I dont know if there is a way to do that in filebeat but you can create Index Pattern for your logs in Kibana and add it a time filter field.

8

If your original log files are present in the index, potentially in the _source field, you should be able to reindex using an ingest pipeline. Ingest pipelines use similar extensions as Logstash, but are built in to elasticsearch: Ingest pipelines | Elasticsearch Guide [8.11] | Elastic

The reindex API docs say:

Reindex can also use the Ingest node feature by specifying a pipeline like this:

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "some_ingest_pipeline"
  }
}

If the data is available in Kibana, you may be able to change the Index Pattern by going to Management > Index Patterns

9

This is incorrect, many of the parsing features of Logstash are supported using Ingest pipelines in Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.