GROK Parse failure for log file pattern

  1. 2019-05-12 15:57:15.446 [localhost] [Services] [Request Id : RT1242124202120519027049] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [1242,124202,RT1242124202120519027049] Exit Add_SaleLineItem: Output [" Transaction_Status=In process Transaction_Total=�0.00 Item_Count=1 RefundExpiryDate=Wed Jun 26 23:59:59 BST 2019 ageRestrictedItemDeclinedByCashier=false ageRestrictionByPassByCashier= promotionId={rrrLog.transaction.customerOrderTransaction.PromotionSavingsSummary.savingsList.0-promotionId}"]
  2. 2019-05-12 15:57:15.788 [localhost] [Services] [Request Id : 98626a7b-cbce-40d3-9fff-31455aba7f53] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [,,] Enter ActiveTransaction: Input [""]
  3. 2019-05-12 15:57:15.805 [localhost] [Services] [Request Id : 98626a7b-cbce-40d3-9fff-31455aba7f53] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [,,] Exit ActiveTransaction: Output ["TransactionId =RT1242124202120519027049 SequenceNumber =270 TransactionCurrencyCode =GBP TransactionBeginDateTime =Sun May 12 15:57:14 BST 2019" ]
  4. 2019-05-12 15:57:18.884 [localhost] [Services] [Request Id : dc89f23d-7c43-4cc3-b47b-0f9c6392b690] [User Id : ] ERROR Log:104 - Servicesstore.heartbeat.serviceimpl.DbServerHealthCheckServiceImpl : in DbServerHealthCheckServiceImpl class: checkHealth method errorjdbc/PrimaryDataSource -- service jboss.naming.context.java.jboss.jdbc.PrimaryDataSource
  5. 2019-05-12 15:59:49.415 [localhost] [Services] [Request Id : 1531e5bd-6e67-439e-9b9a-a44f7687cab8] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [1242,124202,] Exit LookupLastTransaction: Output ["Transaction_id =RT1242124202120519027049 TransactionDateTime =rrrlog.DateTimeCommonData@55eb23bb TotalQuantity =4 TenderTotal = GrandTotal = TotalSalesAmount =�10.99 TotalReturnAmount =�15.00"]
  6. 2019-05-12 15:59:49.415 [localhost] [Services] [Request Id : 1531e5bd-6e67-439e-9b9a-a44f7687cab8] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [1242,124202,] Exit LookupLastTransaction: Output ["Transaction_id =RT1242124202120519027049 TransactionDateTime =rrrlog.DateTimeCommonData@55eb23bb TotalQuantity =10 TenderTotal = GrandTotal = TotalSalesAmount =�152.99 TotalReturnAmount =�86.00"]
  7. 2019-05-12 16:01:15.767 [localhost] [Services] [Request Id : 91406c10-6abd-493b-99ba-f2acad4c947f] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [,,] Enter ActiveTransaction: Input [""]
  8. 2019-05-13 07:27:34.311 [localhost] [Services] [Request Id : c3489430-78ff-4db4-bf05-0539187d582d] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [1242,124202,RT1242124202130519000589] Enter Add_Cash_Tender: Input ["Transaction_Type =RetailTransaction Transaction_Status= Operator_ID = TEST@sales.com Tender_line_no = 0 Tender_type = Cash tenderAmount = 36.52 Currency_Code = GBP"]
  9. 2019-05-12 16:01:15.785 [localhost] [Services] [Request Id : 91406c10-6abd-493b-99ba-f2acad4c947f] [User Id : TEST@sales.com] INFO Services.aspect.AspectController:120 - [,,] Exit ActiveTransaction: Output ["TransactionId =RT1242124202120519027293 SequenceNumber =272 TransactionCurrencyCode =GBP TransactionBeginDateTime =Sun May 12 16:01:15 BST 2019" ]

Hi All

Above is my log message printed from server.I need to slice the data from line numbers 5,6 & 8.

This is my GROK filter
filter {
grok {
match => [
"message" , "%{GREEDYDATA}TotalQuantity%{SPACE}=%{DATA}%{NUMBER:TotalQuantity:int}%{GREEDYDATA}TotalSalesAmount%{SPACE}=%{DATA}%{NUMBER:TotalSalesAmount:float}%{SPACE}TotalReturnAmount%{SPACE}=%{DATA}%{NUMBER:TotalReturnAmount:float}",
"message" , "%{GREEDYDATA}Cash tenderAmount%{SPACE}=%{DATA}%{NUMBER:CashTenderAmount:float}" ]
remove_field => ["message"]
}
}

and this works fine only if log message exactly like line numbers 5,6 & 8 are printed or else I am getting "tags" => [
[0] "_grokparsefailure"
],
message in logstash .Since they keywords strings will be printed in between the logs I am not sure how to use this filters to parse the entire logs and extract the value which I want.And my log file will not be in exact pattern as like GROK.

Any help is much appreciated

you should parse the data from beginning, you are ignoring beginning data as GREEDYDATA and you are losing the information.
I observed your logs, that contains the information how they are different from each others
for example: at line no. 5 and 6 it contains "Exit LookupLastTransaction" and line 8 has "Enter Add_Cash_Tender" may be these are the types, so first parse the data till this point and keep remaining data in a single field.

afterwards you make condition based on the types and then parse the remaining filed(by using kv filter or grok filter).