# Match complete line after some regex pattern **URL:** \ **Category:** Logstash\ **Created:** [September 16, 2021, 11:31am UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385 "2021-09-16T11:31:37Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 16, 2021, 11:31am UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/1 "2021-09-16T11:31:37Z")
I have lines in document like this. ```auto *** Begin time: Sat Jun 26 21:11:14 AEST 2019 ``` Want to assign new field **begin\_time = Sat Jun 26 21:11:14 AEST 2019** This is what I tried: ```auto if x =~ /(\bBegin time:\s+(.*))/ begin_time = x ``` return empty string ```auto if x =~ /(\b?.:?Begin time:\s+(\S+.*))/ begin_time = x ``` also returns empty string. Trying it out in rubular, it works (mathes the new line content) - [Rubular: \bBegin time:\s+(.\*)](https://rubular.com/r/1wR4BQiGymjcEL) ---
**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\ **Post date:** [September 16, 2021, 3:51pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/2 "2021-09-16T15:51:27Z")
Hi, You can use grok for that ```auto grok { match { "x" => "Begin time:\s+%{GREEDYDATA:begin_time}$" } } ``` But you can't create and edit a field like you try to (with only an association). Cad. ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 16, 2021, 9:05pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/3 "2021-09-16T21:05:38Z")
> [@Cad](#): > > ```auto > grok { > match { > "x" => "Begin time:\s+%{GREEDYDATA:begin_time}$" > } > } > > ``` Unfotunately, this block does not work (I keep getting errors), this is "expanded" code ```auto filter { ruby { code => ' lines = event.get("message").lines(chomp: true) begin_time = "" end_time = "" lines.each { |x| if x =~ /Begin time:)/ begin_time = x elsif x =~ /(End time)/ end_time = x ... ``` In the same file (log) I have begin and end time timestamps, and also couple of other variables.. and all of them works ok if they are on the same line, but cannot succeed to map value from next row, like in example. ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 17, 2021, 12:34pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/4 "2021-09-17T12:34:54Z")
`[2021-09-17T14:34:16,392][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 118, column 8 (byte 4435) after filter {\n\truby {\n..` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 17, 2021, 12:42pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/5 "2021-09-17T12:42:18Z")
This is a pipeline configuration error, something is missing, maybe a curly bracket or double quotes was not closed. It says where the error is: `Message=>"Expected one of #, => at line 118, column 8` ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 17, 2021, 12:48pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/6 "2021-09-17T12:48:32Z")
I know, but it appears when i add grok in the filter, otherwise it does not.. ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 17, 2021, 2:31pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/7 "2021-09-17T14:31:57Z")
This is the code, I don't really see anything wrong.. ```auto input { file { path => "/etc/logstash/files/*" codec => multiline { pattern => "^$" negate => true what => next max_lines => 14000 auto_flush_interval => 5 } start_position => "beginning" sincedb_path => "/dev/null" } } filter { ruby { code => ' lines = event.get("message").lines(chomp: true) begin_time = "" end_time = "" logData = "" user = "" lines.each { |x| if x =~ /(Begin time)/ begin_time = x elsif x =~ /(End time)/ end_time = x #end_time = end_time * "" elsif x =~ /^(USER)/ user = x.scan(/(?:.?USER=)(.*)/ )[0] user = user * "" else unless x =~ /^(\s|sending|total|rsl|sent|\[)/ logData += x + "," end end } event.set("begin_time", begin_time) event.set("end_time", end_time) event.set("logData", logData) event.set("user", user) ' } // I PUT GROK HERE AND GET AN ERROR // grok { // match { // "x" => "Begin time:\s+%{GREEDYDATA:begin_time}$" // } //} mutate { remove_field => ["tags", "message", "@version", "@timestamp", "host", "path"] } } output{ elasticsearch { hosts => ["XXXX"] index => "log_logs" } stdout { codec => rubydebug } } ``` ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 17, 2021, 3:37pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/8 "2021-09-17T15:37:12Z")
I managed to do something and grok is now working, but since my file has multiple lines after "begin time", it saves under "begin time" field all the rest of the document, how can I make him to store only that one (next) line where the date is? Thank you, now it looks like: ```auto "begin_time" : """ Sat Jun 26 16:56:16 AEST 2021 rm: cannot remove '/scrXXXX/003': No such file or directory IDS=4947802324992 ENV=BATCH LD_LIBRARY_PATH=/apps/ncl/6.6.2/l ... ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 17, 2021, 9:02pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/9 "2021-09-17T21:02:44Z")
> [@ansamHox](#): > > I have lines in document like this. > > ```auto > *** Begin time: > Sat Jun 26 21:11:14 AEST 2019 > > ``` That's not a line, it is two lines. You use a multiline codec to combine multiple lines into a single [message] field, but then you are using ``` lines = event.get("message").lines(chomp: true) ``` in order to process the lines one at a time. You could try something like ``` message = event.get("message") mdata = message.match(/Begin Time:[^\n]*\n([^\n]*)\n/) if mdata event.set("begin_time", mdata[0]) end ``` ---
**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Post date:** [September 17, 2021, 9:57pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/10 "2021-09-17T21:57:55Z")
I'm using chomp true because I'm extracting 30 more fields which are in the same line ( field: value, eg.). Ony **begin\_time** and **end\_time** has value in new line. I added new message variable and tried out this match, and the result is: ```auto "start_time" : """ Starting time: Sat Jun 26 16:56:16 AEST 2021 """ ``` Looks like it should be `event.set("begin_time", mdata[1])` Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 15, 2021, 9:57pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385/11 "2021-10-15T21:57:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.