Formatting data as a block in logstash

Elastic Stack Logstash
1

Below is my sample data. How can it be filter based on start time and end time using grok pattern. Any idea friends since the data came as a stream of blocks. Can you guys suggest and provid example. I need to know start time , end time, success or fail. if fail the messge

++++++++++++++++++++++++++++++++++++++++++++++
Name: My file name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Begin Date:	MM/DD/YYYY
Begin Time:	HH:MM:SS

Activity
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
YYYY-MM-DD HH:MM:SS message
YYYY-MM-DD HH:MM:SS message
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Result
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
messge Completed
End Date:	MM/DD/YYYY
End Time:	HH:MM:SS
Executed by: Name
++++++++++++++++++++++++++++++++++++++++++++++

++++++++++++++++++++++++++++++++++++++++++++++
Name: My file name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Begin Date:	MM/DD/YYYY
Begin Time:	HH:MM:SS

Activity
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
YYYY-MM-DD HH:MM:SS message
YYYY-MM-DD HH:MM:SS message
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Result
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
message Failed
ERROR:	message
ERROR:	message
  at stacktrace
  at stacktrace
Caused by: message
  at stacktrace
  at stacktrace
	
End Date:	MM/DD/YYYY
End Time:	HH:MM:SS
Executed by: Name
++++++++++++++++++++++++++++++++++++++++++++++
2

A couple of hints I can think of.
First you need to use a multiline input and capture everything between the "++++++" lines.
Like this:
++++++++++++++++++++++++++++++++++++++++++++++ Name: My file name ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Begin Date: 10/13/1987 Begin Time: HH:MM:SS Activity ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ YYYY-MM-DD HH:MM:SS message YYYY-MM-DD HH:MM:SS message ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Result ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ messge Completed End Date: MM/DD/YYYY End Time: HH:MM:SS Executed by: Name ++++++++++++++++++++++++++++++++++++++++++++++

Then use a grok pattern (going to be ugly) to cut out the pieces that you need.

I started playing around with it a little bit (assuming that all newline characters were removed and the entire thing could be handled as a single line event.

\+ Name: %{DATA:my_file_name} \~%{NOTSPACE} Begin Date: %{NUMBER:Month}

This will get you this:


  "my_file_name": [
    [
      "My file name"
    ]
  ],
  "NOTSPACE": [
    [
      "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
    ]
  ],
  "Month": [
    [
      "10"
    ]
  ],
  "BASE10NUM": [
    [
      "10"
    ]
  ]
}

Check out https://grokdebug.herokuapp.com/ to help you figure out the rest.
Good luck.