# Empty fields in input - grok failure
**URL:** https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483
**Category:** Logstash
**Created:** [May 19, 2017, 7:10pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483 "2017-05-19T19:10:57Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)
#### Post date: [May 19, 2017, 7:10pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483/1 "2017-05-19T19:10:57Z")
Hi ,
I am trying to use grok for input lines like ;
**INPUT DATA**
[2017-05-15 00:00:07,751] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 26baee2d30164c2083e380b40b2a3b15 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212154&s2=306903392384](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212154&s2=306903392384) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: [ET: DLPInfo, IP: 81.2.24.161]
[2017-05-15 00:00:07,877] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 22ac17317b93484c89a3da74ff72845b :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212151&s2=322766181292](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212151&s2=322766181292) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: [ET: DLPInfo, IP: 81.2.54.161]
[2017-05-15 00:00:07,884] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 2dbcaad4295249cc87e6e5793c9fa909 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212155&s2=306903392824](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212155&s2=306903392824) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: [ET: DLPInfo, IP: 81.2.25.161]
[2017-05-15 00:00:07,965] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 5941264e1dfb4711862f5106e9f28087 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212147&s2=306903393464](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212147&s2=306903393464) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: [ET: DLPInfo, IP: 81.2.258.161]
[2017-05-15 00:00:07,988] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 26baee2d30164c2083e380b40b2a3b15 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212154&s2=306903392384](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212154&s2=306903392384) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: [ET: SplashPageServed, IP: 81.2.254.161]
**GROK works fine with**
input {
beats {
port =\> 5044
}
}
filter {
grok {
match =\> { "message" =\> "(?[[^]]_])%{SPACE}:|:%{SPACE}%{WORD:level}%{SPACE}:|:%{SPACE}%{USERNAME:hostname}%{SPACE}:|:%{SPACE}%{GREEDYDATA:coidkey}%{SPACE}:|:%{SPACE}%{GREEDYDATA:clientinfo}%{SPACE}:|:%{SPACE}%{IP:clientIP}%{SPACE}:|:%{SPACE}%{GREEDYDATA:Url}%{SPAC
E}:|:%{SPACE}%{JAVACLASS:class}%{SPACE}:|:%{SPACE}(?[[^]]_])"}
}
}
output {
stdout { codec =\> rubydebug }
}
**but if any field is missing ( Empty field ) in input like ,**
[2017-05-11 10:13:33,203] :|: INFO :|: [dfprdsndlbfe24.df.jabodo.com](http://dfprdsndlbfe24.df.jabodo.com) **:|: :|:** [BT:CHROME, BV:58, BL:en, CC:IN] :|: 117.221.64.97 :|: :|: c.m.w.d.m.UnifiedLoggerWrapper :|: - [ET: PageView, IP: 117.221.64.97]
[2017-05-11 10:13:33,308] :|: INFO :|: [dfprdsndlbfe24.df.jabodo.com](http://dfprdsndlbfe24.df.jabodo.com) :|: :|: [BT:CHROME, BV:58, BL:en, CC:IN] :|: 117.221.64.97 :|: :|: c.m.w.d.m.UnifiedLoggerWrapper :|: - [ET: PageView, IP: 117.221.64.97]
[2017-05-11 10:05:00,000] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) **:|: :|: :|: :|: :|:** c.m.w.d.s.InMemoryDataRefreshService :|: - Starting testAllocation refresh
[2017-05-11 10:05:00,006] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.w.d.s.InMemoryDataRefreshService :|: - Done refreshing testAllocation. It took 6 milliseconds
[2017-05-11 10:05:29,408] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.s.c.h.HazelcastCacheService :|: - Cache client is connected!
[2017-05-11 10:05:29,651] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.s.c.h.HazelcastCacheService :|: - Cache client is connected!
[2017-05-11 10:05:33,432] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.c.PeriodicProductDataRefresher :|: - Beginning Product data refresh
[2017-05-11 10:05:34,492] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.c.PeriodicProductDataRefresher :|: - Product data is up to date, no refresh was conducted
[2017-05-11 10:10:29,408] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.s.c.h.HazelcastCacheService :|: - Cache client is connected!
**Grok does not work , can someone help on this issue ?**
Thanks,
Nikhil
---
### Author:  [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)
#### Post date: [May 19, 2017, 8:21pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483/2 "2017-05-19T20:21:17Z")
I checked **(?:%{IP:clientip})?** or **(?:\s+%{IP:ip})** grok it works but it skips the fields name which is defined . I am trying to display the fieldname even though its empty(i.e there is no value)
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [May 20, 2017, 10:46am UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483/3 "2017-05-20T10:46:16Z")
This seems to be a csv style format with `:|:` as a separator. Have you tried using the [csv filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv) instead of grok?
---
### Author:  [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)
#### Post date: [May 22, 2017, 2:55pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483/4 "2017-05-22T14:55:09Z")
I am not familiar of CSV filter.
my grok is working fine but only issues im having is - at the end and the different last field
following grok works fine for input without dash near end field
INPUT :- [2017-05-15 00:00:07,751] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 26baee2d30164c2083e380b40b2a3b15 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212154&s2=306903392384](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212154&s2=306903392384) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: **[ET: DLPInfo, IP: 81.2.24.161]**
^(?[[^]]_])%{SPACE}:|:%{SPACE}(?:\s+%{WORD:level})?%{SPACE}:|:%{SPACE}(?:\s+%{USERNAME:hostname})?%{SPACE}:|:%{SPACE}(?:\s+%{GREEDYDATA:coidkey})?%{SPACE}:|:%{SPACE}(?:\s+%{GREEDYDATA:clientinfo})?%{SPACE}:|:%{SPACE}(?:\s+%{IP:clientip})?%{SPACE}:|:%{SPACE}(?:\s+%{GREEDYDATA:Url})?%{SPACE}:|:%{SPACE}(?:\s+%{JAVACLASS:class})?%{SPACE}:|:%{SPACE}(?\\[[^]]_])$
but there is a - near last field and last field varies , please check following
[2017-05-15 00:00:07,751] :|: INFO :|: [dubprdsndlbfe33.dub.jabodo.com](http://dubprdsndlbfe33.dub.jabodo.com) :|: 26baee2d30164c2083e380b40b2a3b15 :|: [BT:CHROME, BV:55, BL:en, CC:CZ] :|: 81.2.254.161 :|: [http://free.internetspeedtracker.com/index.jhtml?partner=^BBQ^xpt184&s1=1212154&s2=306903392384](http://free.internetspeedtracker.com/index.jhtml?partner=%5EBBQ%5Expt184&s1=1212154&s2=306903392384) :|: c.m.w.d.m.UnifiedLoggerWrapper :|: **- [ET: DLPInfo, IP: 81.2.24.161]**
[2017-05-09 10:22:00,000] :|: INFO :|: [dfprdsndlbfe1.df.jabodo.com](http://dfprdsndlbfe1.df.jabodo.com) :|: :|: :|: :|: :|: c.m.w.d.s.InMemoryDataRefreshService :|: **- Starting messages refresh**
can someone please advise ?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 19, 2017, 2:55pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483/5 "2017-06-19T14:55:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.