# Logstah doesn't split log into separate fields in elastic index

**URL:** https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049
**Category:** Logstash
**Created:** [January 10, 2025, 9:34am UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049 "2025-01-10T09:34:12Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![andp](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@andp](https://discuss.elastic.co/u/andp)
#### Post date: [January 10, 2025, 9:34am UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049/1 "2025-01-10T09:34:12Z")

</div>

Hi everyone!

I am trying to import logs to elastic index using Filebeat and Logstah.  
The index is created but whole log line is present in one field message.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/0/40d5b914805b1b0e689ffa5e053c8da1b21b471e.png)

For start I'm trying to separate datetime to a separate field and most likely make it a timestamp in kibana.

He is an example of my log:

```auto
2024-10-27 22:00:32.289 [DefaultQuartzScheduler_Worker-5] DEBUG c.c.t.s.a.f.t.i.SomeMethod[run()][line 154]: Some log message here "with quatation" or without

```

in logstah.conf I have grok expression like this:

```auto
filter {
  if [type] == "logfile" {
    grok {
      match => { 'message' => "%{DATA:fc_timestamp}" }
    }
	date {
      match => ["fc_timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
	  target => "fc_timestamp"
    }
  }
}

```

Unfortunately it doesn't create fc\_timestamp field in index.

Whole grok which I tested in some grok debbuger and it seems to work is:

```auto
match => { 'message' => "%{DATA:fc_timestamp} \[%{DATA:worker_number}\] %{LOGLEVEL:log_level} %{DATA:method}: %{GREEDYDATA:log}" }

```

But for now I am struggling just to separete log entry date and best to make it a timestamp.  
Please help.

---

<div class="post-metadata">

### Author: ![andp](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@andp](https://discuss.elastic.co/u/andp)
#### Post date: [January 15, 2025, 5:19pm UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049/2 "2025-01-15T17:19:15Z")

</div>

I did some changes to logstah.conf to see if there is any improvement but it there isn't:

```auto
input {
  beats {
    port => 5044
	type => fc_logfile
  }
}

filter {
  if [type] == "fc_logfile" {
    grok {
      match => { 'message' => "%{DATA:fc_timestamp}" }
    }
	date {
      match => ["fc_timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
	  target => "@timestamp"
    }
  }
}

```

Unfortunately @timestamp in kibana is still holding date and time of import and not log record datetime.

![image](https://us1.discourse-cdn.com/elastic/original/3X/f/c/fce1e1654dd145dad64e78cfcea07869bbc51efe.png)

---

<div class="post-metadata">

### Author: ![andp](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@andp](https://discuss.elastic.co/u/andp)
#### Post date: [January 15, 2025, 5:19pm UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049/3 "2025-01-15T17:19:50Z")

</div>

The only change I can see is in type field

![image](https://us1.discourse-cdn.com/elastic/original/3X/f/c/fce1e1654dd145dad64e78cfcea07869bbc51efe.png)

Any help will be much appreciated.

Regards!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 15, 2025, 8:00pm UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049/4 "2025-01-15T20:00:41Z")

</div>

> [@andp](#):
>
> `match => { 'message' => "%{DATA:fc_timestamp}" }`

DATA can match anything, including nothing at all. If you add `keep_empty_captures => true` to the grok filter then you will see that "nothing at all" is exactly what it is matching.

```
"fc_timestamp" => "",

```

So you need to modify the pattern to force DATA to capture something. If you use `%{DATA:fc_timestamp} ` with a trailing space then you will get

```
"fc_timestamp" => "2024-10-27",

```

to force the capture of whole timestamp try

```
grok { match => { 'message' => "%{DATA:fc_timestamp} \[" } }

```

Your date filter will then work.

Building a grok approach incrementally is a great approach to creating complex patterns, you just need to start with enough to get an initial match 🙂

---

<div class="post-metadata">

### Author: ![andp](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@andp](https://discuss.elastic.co/u/andp)
#### Post date: [January 20, 2025, 1:38pm UTC](https://discuss.elastic.co/t/logstah-doesnt-split-log-into-separate-fields-in-elastic-index/373049/5 "2025-01-20T13:38:50Z")

</div>

Thank You for your help!
