# Can't create a field with a variable from a grok match regex **URL:** \ **Category:** Logstash\ **Created:** [August 1, 2018, 3:35pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613 "2018-08-01T15:35:12Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![TheSmartMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thesmartmonkey/32/33965_2.png) [@TheSmartMonkey](https://discuss.elastic.co/u/TheSmartMonkey)\ **Post date:** [August 1, 2018, 3:35pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/1 "2018-08-01T15:35:12Z")
Hi all, I am currently using logstash, elasticsearch and kibana 6.3.0 My log are generated at a unique id path : /tmp/USER\_DATA/FactoryContainer/images/(my unique id)/oar/oar\_image\_job(my unique id).stdout What i want to do is to match this unique id and to create a field with this id. I m a bit novice to logstash filter but I don't know why it doesn't want to use my uid and always return me %{uid} in my field or this Failed to execute action error my filter : ``` input { file { path => "/tmp/USER_DATA/FactoryContainer/images/*/oar/oar_image_job*.stdout" start_position => "beginning" add_field => { "data_source" => "oar-image-job" } } } filter { grok { match => ["path","%{UNIXPATH}%{NUMBER:uid}%{UNIXPATH}"] } mutate { add_field => ["unique_id" => "%{uid}"] } } output { if [data_source] == "oar-image-job" { elasticsearch { index => "oar-image-job-%{+YYYY.MM.dd}" hosts => ["localhost:9200"] } } } ``` the data\_source field is to avoid this issue : When you put multiple config files in a directory for Logstash to use, they will all be concatenated in the grok debugger %{UNIXPATH}%{NUMBER:uid}%{UNIXPATH} my path return me the good value ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 1, 2018, 4:52pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/2 "2018-08-01T16:52:24Z")
Avoid UNIXPATH. It is extremely [expensive](https://github.com/logstash-plugins/logstash-patterns-core/issues/159). You know what the path looks like. Why not reference it? ``` grok { match => { "path" => ["/images/%{DATA:id1}/oar/oar_images_job%{DATA:id2}.stdout"] } } ``` ---
**Author:** ![TheSmartMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thesmartmonkey/32/33965_2.png) [@TheSmartMonkey](https://discuss.elastic.co/u/TheSmartMonkey)\ **Post date:** [August 2, 2018, 2:41pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/3 "2018-08-02T14:41:09Z")
Good point but it don't resolve my issue, other ideas ? ``` input { file { path => "/tmp/USER_DATA/FactoryContainer/images/*/oar/oar_image_job*.stdout" start_position => "beginning" add_field => { "data_source" => "oar-image-job" } } } filter { grok { match => { "path" => ["/tmp/USER_DATA/FactoryContainer/images/%{DATA:id1}/oar/oar_image_job%{DATA:id2}.stdout"] } } mutate { add_field => ["unique_id" => "%{id1}"] } } output { if [data_source] == "oar-image-job" { elasticsearch { index => "oar-image-job-%{+YYYY.MM.dd}" hosts => ["localhost:9200"] } } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 2, 2018, 3:00pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/4 "2018-08-02T15:00:19Z")
> [@TheSmartMonkey](#): > > Good point but it don't resolve my issue What is your issue? ---
**Author:** ![TheSmartMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thesmartmonkey/32/33965_2.png) [@TheSmartMonkey](https://discuss.elastic.co/u/TheSmartMonkey)\ **Post date:** [August 2, 2018, 3:02pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/5 "2018-08-02T15:02:09Z")
[ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, {, ,, ] at line 64, column 36 (byte 1305) after filter {\n grok {\n match =\> { "[path]" =\> ["/tmp/USER\_DATA/FactoryContainer/images/%{DATA:id1}/oar/oar\_image\_job%{DATA:id2}.stdout"] }\n }\n mutate {\n add\_field =\> ["unique\_id" ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:167:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:305:in `block in converge\_state'"]} ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 2, 2018, 3:05pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/6 "2018-08-02T15:05:57Z")
``` mutate { add_field => { "unique_id" => "%{id1}" } } ``` Use braces, not brackets. It wants a hash, not an array. ---
**Author:** ![TheSmartMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thesmartmonkey/32/33965_2.png) [@TheSmartMonkey](https://discuss.elastic.co/u/TheSmartMonkey)\ **Post date:** [August 2, 2018, 3:18pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/7 "2018-08-02T15:18:00Z")
Thanks a lot @Badger I just removed the mutate and changed the data variable because it add a field automatically don't need add\_field the correct filter: ``` input { file { path => "/tmp/USER_DATA/FactoryContainer/images/*/oar/oar_image_job*.stdout" start_position => "beginning" add_field => { "data_source" => "oar-image-job" } } } filter { grok { match => { "path" => ["/tmp/USER_DATA/FactoryContainer/images/%{DATA:unique_id}/oar/oar_image_job%{DATA}.stdout"] } } } output { if [data_source] == "oar-image-job" { elasticsearch { index => "oar-image-job-%{+YYYY.MM.dd}" hosts => ["localhost:9200"] } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 30, 2018, 3:18pm UTC](https://discuss.elastic.co/t/cant-create-a-field-with-a-variable-from-a-grok-match-regex/142613/8 "2018-08-30T15:18:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.