# Grok Help Request - Get FileDate from File Name into new field FileDate
**URL:** https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404
**Category:** Logstash
**Created:** [December 5, 2017, 6:13pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404 "2017-12-05T18:13:25Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 5, 2017, 6:13pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/1 "2017-12-05T18:13:25Z")
Hi, I have a lot of csv files I am importing using the CSV filter but they have not timestamp included in the logs but I do have the date the log was produced in the filename, therefore, I want to create a new filedate field and then I want to grok out the date from the filename and send it to the filedate field but having problems getting this working.
here is my file name(there are just two columns in these files a name and a number:
Cangenbus-17-10-20.csv
here is my index and mapping:
PUT cangenbus-11
{
"mappings": {
"doc": {
"properties": {
"Name": { "type": "text" },
"Number": { "type": "integer","ignore\_malformed": true},
"FileDate": { "type": "date" },
"Path": { "type": "text" }
}
}
}
}
===========================================================
here is my configuration file:
input {
file {
path =\> "/opt/sample-data/cangenbus-csv/\*.csv"
start\_position =\> "beginning"
sincedb\_path =\> "/dev/null"
}
}
filter {
csv {
separator =\> ","
columns =\> ["Name","Number"]
}
grok {
match =\> (?[%{YEAR-}%{MONTHNUM-}%{MONTHDAY-}])
add\_field =\> ["filedate", "%{year-}%{month-}{day-}"]
}
date{
match =\> ["temptimestamp", "[yyyy-MM-dd]"]
target =\> "filedate"
}
}
output {
elasticsearch {
hosts =\> "[http://10.0.2.15:9200](http://10.0.2.15:9200)"
index =\> "cangenbus-v9"
}
stdout {}
}
=====================================================
here is the error:
[ERROR] 2017-12-05 10:03:51.463 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, ", ', -, [, { at line 15, column 10 (byte 240) after filter {\n csv {\n separator =\> ","\n columns =\> ["Name","Number"]\t\n }\n \ngrok {\nmatch =\> ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_ast'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:54:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in`block in compile\_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in`compile\_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:107:in `compile_lir'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:215:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:35:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:362:in`block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
Help would be appreciated.
Thank you.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2017, 7:19pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/2 "2017-12-05T19:19:12Z")
The file input puts the filename in path. This seems to work.
```
filter {
grok {
match => { "path" => "-%{INT:year}-%{INT:month}-%{INT:day}\.csv" }
add_field => ["filedate", "%{year}-%{month}-%{day}"]
}
}
```
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 5, 2017, 9:32pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/3 "2017-12-05T21:32:19Z")
Thank you this works! I really appreciate it. In case anyone else is trying to do this here is my config file:
root@ubuntu-16:~# clear
root@ubuntu-16:~# tmux
input {
file {
path =\> "/opt/sample-data/cangenbus-csv/\*.csv"
start\_position =\> "beginning"
sincedb\_path =\> "/dev/null"
}
}
filter {
csv {
separator =\> ","
columns =\> ["Name","Number"]
}
grok {
match =\> {"path" =\> "-%{INT:year}-%{INT:month}-%{INT:day}.csv"}
add\_field =\> ["filedate", "%{year}-%{month}-%{day}"]
}
}
output {
elasticsearch {
hosts =\> "[http://10.0.2.15:9200](http://10.0.2.15:9200)"
index =\> "cangenbus-v13"
}
stdout {}
}
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 5, 2017, 9:34pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/4 "2017-12-05T21:34:05Z")
similarly, if I want to grok out the filename and set is a new filed called filename would it be like this?:
grok {
match =\> {"path" =\> "{WORD:filename}-%.csv"}
add\_field =\> ["filename", "%keyword"]
}
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 7, 2017, 9:59pm UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/5 "2017-12-07T21:59:34Z")
Thanks can you help me with one more please? I am trying to make another grok filter statement to createa new field filename and populate it with the filename in the path field. It is not working for me.
grok {
match =\> {"path" =\> "{WORD:filename}-%{INT:year}-%{INT:month}-%{INT:day}.csv"}
add\_field =\> ["filename", "%{keyword}"]
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 8, 2017, 12:00am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/6 "2017-12-08T00:00:11Z")
@sconrod, the match is missing % before {WORD:filename}. I do not know what you are trying to do with the add\_field. If the match pattern is corrected then the filename field gets added.
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 8, 2017, 12:06am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/7 "2017-12-08T00:06:25Z")
Hi thanks I am trying to add a new field which will also show up in the index called "filename" and I want just the first part of the filename without the date in that filename field.
so my actual filename or logname is: Cangenbus-17-10-20.csv
and I want to populate the new field I am creating called 'filename' with just the first part of the name which is Cangenbus
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 8, 2017, 12:08am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/8 "2017-12-08T00:08:11Z")
I tried this as well and it isn't working:
grok {
match =\> {"path" =\> " "%{WORD:filename}-%{GREEDYDATA}.csv"}
add\_field =\> ["filename", "%{filename}"]
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 8, 2017, 12:16am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/9 "2017-12-08T00:16:51Z")
Use the first one, with the % added
```
match => {"path" => "%{WORD:filename}-%{INT:year}-%{INT:month}-%{INT:day}.csv"}
```
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 8, 2017, 12:41am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/10 "2017-12-08T00:41:25Z")
Thanks that works, but I am getting the name twice in the new filename field.
This is my entire grok filter...do I need the second line then?
grok {
match =\> {"path" =\> "%{WORD:filename}-%{INT:year}-%{INT:month}-%{INT:day}.csv"}
add\_field =\> ["filename", "%{filename}"]
}
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 8, 2017, 12:42am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/11 "2017-12-08T00:42:42Z")
No, you do not need the second line.
---
### Author:  [@sconrod](https://discuss.elastic.co/u/sconrod)
#### Post date: [December 8, 2017, 1:01am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/12 "2017-12-08T01:01:47Z")
Thank you.
I will remove it. I appreciate the help. If you have time I have another one open on a log with binary data in it that will not parse.....:0)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 5, 2018, 1:02am UTC](https://discuss.elastic.co/t/grok-help-request-get-filedate-from-file-name-into-new-field-filedate/110404/13 "2018-01-05T01:02:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.