# Get filename using beats and filter it in logstash

**URL:** <https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058>\
**Category:** Logstash\
**Created:** [July 11, 2019, 3:21pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058 "2019-07-11T15:21:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![111175](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111175/32/46127_2.png) [@111175](https://discuss.elastic.co/u/111175)\
**Post date:** [July 11, 2019, 3:21pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/1 "2019-07-11T15:21:22Z")

</div>

Hi,  
I have logfile with date as it's name and time inside it. I'm using filebeats to get logs from remote server. Is it possible to combine date with time somehow and put in @timestamp? I tried something like this:  
//  
filter {  
grok {  
match =\> {  
"path" =\> [  
"D:/logs/(?%{YEAR})(?%{MONTHNUM})(?%{MONTHDAY}).log"  
]  
}  
}

```
    mutate {
        #combine them
        add_field => {"timestamp" => "%{year}-%{month}-%{day} %{logtime}"}
           }

    date {
        match => ["timestamp","ISO8601"]
       timezone => "UTC"
        remove_field => ["timestamp"]
          }

    mutate {
        #throw fields away
        remove_field => ["logtime","year","day","month"]
           }

```

}  
//  
It works fine with local file, but not with beats. Any ideas?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 11, 2019, 3:29pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/2 "2019-07-11T15:29:27Z")

</div>

If the file name were in the [source] field I would expect that to work. That suggests that the file name is not in that field. If you are using Kibana, what does an event look like on the JSON tab in Discover?

---

<div class="post-metadata">

**Author:** ![111175](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111175/32/46127_2.png) [@111175](https://discuss.elastic.co/u/111175)\
**Post date:** [July 11, 2019, 3:31pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/3 "2019-07-11T15:31:54Z")

</div>

Sorry, it's not "source", I used "path".  
When it's local file then everything is fine, when I'm using beats it shows like %{year}-%{month}-%{day} 15:23:56:122

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 11, 2019, 3:42pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/4 "2019-07-11T15:42:18Z")

</div>

OK, so what does the event look like when you are using beats?

---

<div class="post-metadata">

**Author:** ![111175](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111175/32/46127_2.png) [@111175](https://discuss.elastic.co/u/111175)\
**Post date:** [July 12, 2019, 7:58am UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/5 "2019-07-12T07:58:29Z")

</div>

{  
"\_index": "srv01",  
"\_type": "\_doc",  
"\_id": "VtYu5WsBAHAKgvZocFpf",  
"\_version": 1,  
"\_score": 1,  
"\_source": {  
"agent": {  
"ephemeral\_id": "984319d7-95d2-481a-90b2-44fb1ced0214",  
"id": "da2fd808-a61c-46f9-b1e5-89d1912ef38c",  
"version": "7.1.0",  
"hostname": "srv01",  
"type": "filebeat"  
},  
"@timestamp": "2019-07-12T07:55:52.409Z",  
"ecs": {  
"version": "1.0.0"  
},  
"host": {  
"hostname": "srv01",  
"architecture": "x86\_64",  
"name": "srv01",  
"os": {  
"name": "Windows Server 2012 R2 Datacenter",  
"family": "windows",  
"kernel": "6.3.9600.19228 (winblue\_ltsb.181208-0600)",  
"version": "6.3",  
"platform": "windows",  
"build": "9600.19235"  
},  
"id": "d9d9c909-610c-4089-a277-43e5c3e9e82e"  
},  
"@version": "1",  
"log": {  
"offset": 914869,  
"file": {  
"path": "D:\logs\20190104.log"  
}  
},  
"input": {  
"type": "log"  
},  
"severity": "1",  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"\_grokparsefailure",  
"\_dateparsefailure"  
],  
"message": [  
"some message"  
],  
"timestamp": **"%{year}-%{month}-%{day}** 00:03:16.551"  
},  
"fields": {  
"@timestamp": [  
"2019-07-12T07:55:52.409Z"  
]  
}  
}

---

<div class="post-metadata">

**Author:** ![111175](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111175/32/46127_2.png) [@111175](https://discuss.elastic.co/u/111175)\
**Post date:** [July 12, 2019, 2:00pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/6 "2019-07-12T14:00:16Z")

</div>

Problem solved. Correct filter:  
grok {  
match =\> {  
"[log][file][path]" =\> [  
"(?%{YEAR})(?%{MONTHNUM})(?%{MONTHDAY})"  
]  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 9, 2019, 2:10pm UTC](https://discuss.elastic.co/t/get-filename-using-beats-and-filter-it-in-logstash/190058/7 "2019-08-09T14:10:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.