# Logstash does not create fields using grok **URL:** \ **Category:** Logstash\ **Created:** [April 19, 2019, 12:25pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615 "2019-04-19T12:25:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Sam\_Kisada](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_kisada/32/44452_2.png) [@Sam\_Kisada](https://discuss.elastic.co/u/Sam_Kisada)\ **Post date:** [April 19, 2019, 12:25pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/1 "2019-04-19T12:25:50Z")
Hi! This is another "help me with grok" topic, so please help... 😊 Source Log (simple f2b): > ``` > 2019-04-18 16:39:03,454 fail2ban.actions [25673]: NOTICE [postfix-sasl] Ban 185.234.218.237 > 2019-04-18 19:19:20,432 fail2ban.filter [25673]: INFO [postfix-sasl] Found 82.140.81.38 > 2019-04-18 19:19:21,270 fail2ban.actions [25673]: NOTICE [postfix-sasl] Ban 82.140.81.38 > 2019-04-19 04:09:51,432 fail2ban.actions [25673]: NOTICE [postfix-sasl] Unban 170.80.134.250 > 2019-04-19 04:50:22,040 fail2ban.filter [25673]: INFO [postfix-sasl] Found 141.98.80.32 > 2019-04-19 04:50:31,258 fail2ban.filter [25673]: INFO [postfix-sasl] Found 141.98.80.32 > 2019-04-19 04:50:31,582 fail2ban.actions [25673]: NOTICE [postfix-sasl] Ban 141.98.80.32 > 2019-04-19 14:10:08,320 fail2ban.filter [25673]: INFO [postfix-sasl] Found 23.254.231.87 > > ``` This log captured by filebeat and go to logstash: ``` - type: log enabled: true paths: - /var/log/fail2ban.log tags: ["fail2ban"] output.logstash: # The Logstash hosts hosts: ["hostname:5044"] ``` Logstash makes its work: ``` input { beats { port => 5044 } } filter { if [tags] == "fail2ban" { grok { match => { "message" => "%{TIMESTAMP_ISO8601:date}\s%{WORD:log_src}.%{WORD:src_action}\s*\[%{INT:fail2ban_pid}\]:\s%{LOGLEVEL:level}\s*\[%{NOTSPACE:service}\]\s%{WORD:action}\s%{IP:ip}"} } geoip { source => "ip" } } } output { if "fail2ban" in [tags] { elasticsearch { hosts => "10.1.1.11:9200" index => "fail2ban-%{+YYYY.MM}" } file { path => "/var/log/logstash/debug.log" codec=>rubydebug } } } ``` I add index fail2ban-%{+YYYY.MM} to kibana, but no additional fields appear: ``` @timestamp Apr 19, 2019 @ 14:35:53.878 t @version 1 t _id pQpiNWoBiVr8gQ-rCo3X t _index fail2ban-2019.04 # _score - t _type _doc t agent.ephemeral_id c6bbe7df-cb34-43be-a81e-b17eb52ad2eb t agent.hostname ihostname t agent.id 86e5338e-c3f6-414b-9b0e-63359cbe016c t agent.type filebeat t agent.version 7.0.0 t ecs.version 1.0.0 t host.architecture x86_64 host.containerized true t host.hostname ihostname t host.id ddac9954b8134d2999ae8149db6365ce t host.name ihostname t host.os.codename Core t host.os.family redhat t host.os.kernel 3.10.0-957.5.1.el7.x86_64 t host.os.name CentOS Linux t host.os.platform centos t host.os.version 7 (Core) t input.type log t log.file.path /var/log/fail2ban.log # log.offset 10,645 t message 2019-04-18 16:39:03,454 fail2ban.actions [25673]: NOTICE [postfix-sasl] Ban 185.234.218.237 t tags fail2ban, beats_input_codec_plain_applied ``` Grok debugger show me parsed fields: ``` { "date": "2019-04-18 16:39:03,454", "level": "NOTICE", "service": "postfix-sasl", "ip": "185.234.218.237", "action": "Ban", "log_src": "fail2ban", "src_action": "actions", "fail2ban_pid": "25673" } ``` So what is wrong? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 19, 2019, 12:39pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/2 "2019-04-19T12:39:22Z")
> [@Sam\_Kisada](#): > > if [tags] == "fail2ban" { I suspect this should instead be: > [@Sam\_Kisada](#): > > if "fail2ban" in [tags] { ---
**Author:** ![Sam\_Kisada](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_kisada/32/44452_2.png) [@Sam\_Kisada](https://discuss.elastic.co/u/Sam_Kisada)\ **Post date:** [April 19, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/3 "2019-04-19T12:56:22Z")
> [@Christian\_Dahlqvist](#): > > if "fail2ban" in [tags] Thanks for response , but no effect. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 19, 2019, 12:58pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/4 "2019-04-19T12:58:11Z")
What does the output to your debug log file look like? ---
**Author:** ![Sam\_Kisada](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_kisada/32/44452_2.png) [@Sam\_Kisada](https://discuss.elastic.co/u/Sam_Kisada)\ **Post date:** [April 19, 2019, 1:00pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/5 "2019-04-19T13:00:50Z")
``` "input" => { "type" => "log" }, "log" => { "offset" => 11337, "file" => { "path" => "/var/log/fail2ban.log" } }, "@version" => "1", "ecs" => { "version" => "1.0.0" }, "@timestamp" => 2019-04-19T12:44:53.571Z, "tags" => [ [0] "fail2ban", [1] "beats_input_codec_plain_applied" ], "host" => { "os" => { "kernel" => "3.10.0-957.5.1.el7.x86_64", "codename" => "Core", "name" => "CentOS Linux", "platform" => "centos", "version" => "7 (Core)", "family" => "redhat" }, "containerized" => true, "name" => "ihostname", "id" => "ddac9954b8134d2999ae8149db6365ce", "architecture" => "x86_64", "hostname" => "ihostname" }, "agent" => { "id" => "86e5338e-c3f6-414b-9b0e-63359cbe016c", "type" => "filebeat", "version" => "7.0.0", "ephemeral_id" => "2901a7c6-38a2-4772-bd98-39d9cdb8fc25", "hostname" => "ihostname" }, "message" => "2019-04-19 14:10:08,320 fail2ban.filter [25673]: INFO [postfix-sasl] Found 23.254.231.87" ``` ---
**Author:** ![Sam\_Kisada](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_kisada/32/44452_2.png) [@Sam\_Kisada](https://discuss.elastic.co/u/Sam_Kisada)\ **Post date:** [April 19, 2019, 1:10pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/6 "2019-04-19T13:10:32Z")
After second filebeat regestry clean and restart logstash+filebeat needed data have apeared Thanks for help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 17, 2019, 1:10pm UTC](https://discuss.elastic.co/t/logstash-does-not-create-fields-using-grok/177615/7 "2019-05-17T13:10:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.