Hi All,
I have the log line like
2015-06-09 00:00:07,947 [11] DEBUG NCR.XE.Component.MessageHandler.HistoryEventMessageProcessor - Starting RentalTransactionMessageProcessor message type with the data <invoice iid="80000000-d050-163a-1428-ffff08d27045" id="635694191783997533" dt="2015-06-09T03:59:38" dt_local="2015-06-08T20:59:38" site_iid="bd20166f-b304-43bd-9ba3-56df4a429b74" user_id="" cust_id="805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325" cust_name=" " membership_iid="d230ef2c-3a92-4841-a1b6-5816f8b36cf8" sub_total="15" tax_total="0" total="15" status_iid="1" session_id="80000000-d050-163a-fd18-ffff08d27044" loyalty_id="" loyalty_units="0" loyalty_amount="15" created_offline="False" resolution_date="" email_receipt_sent="False"><customer id="805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325" cust_iid="80000000-d14a-163a-4dad-ffff08d1fa67" card_id="bncvfr123456" expiration="2019-04-30T00:00:00" date_last_transaction="2015-01-10T05:07:32" cc_digits="1234" gov_id="" email="test@test.com" f_name="" l_name="" m_name="" dob="" addr1="" addr2="" city="" state="" zip="99577" country="" phone1="" phone2="" cc_type="0" AllowSpecialOffers="true" AllowReceipts="true" /><line iid="80000000-d050-163a-2ab3-ffff08d27045" line_number="0" type="1" product_type="2" product_iid="37180094-996c-4336-93a8-42e87d4ffd92" product_id="D000198" piece_id="E00401007579A27B" description="¡Mujeres Al Ataque!" units="1" unit_price="15" days_out="1" due_dt="2015-06-09T19:00:00" rent_dt="2015-06-08T20:59:00" cost="0" sub_total="15" tax_total="0" line_total="15" orig_days="1" orig_due_dt="2015-06-09T19:00:00" orig_units="1" orig_unit_price="15" parent_invoice="" payment_iid="80000000-d050-163a-2b00-ffff08d27045" days_charged="1" price_desc="Default" price_days_out="1" price_is_partial_day="false" price_config_amount="15" price_group_id="0" return_site_iid=""><tax iid="80000001-d050-163a-2ab3-ffff08d27045" tax_rate_iid="5f454d5f-2948-419c-b3fd-573035be6952" tax_zone_iid="1e52aa32-5ddb-4d28-886d-5e9583b3686a" desc="" rate="0" taxable_amount="15" non_taxable_amount="0" tax_amount="0" /></line><payment amount="15" cc_digits="1234" cc_expiration="2011-02-10T00:00:00" cc_name=" " cc_trans_id="ns8B3e56P5jFH" cc_zip="99577" cust_id="805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325" payment_iid="80000000-d050-163a-2b00-ffff08d27045" payment_dt="2015-06-09T03:59:38" payment_dt_local="2015-06-08T20:59:38" method_iid="3e9547cd-f8d1-41e4-811f-0cbf175f4a15" card_id="ns8B3e56P5jFH" /></invoice>
grok matching pattern is:
grok {
match => { "msg" => '%{GREEDYDATA:text} [pP]iece[_]*?[iI]*[dD]* ?[=]* ?"?(?<PieceID>[A-Z0-9]{16})"?' }
}
loading json format to ES:
{
"message" => "2015-06-09 00:00:07,947 [11] DEBUG NCR.XE.Component.MessageHandler.HistoryEventMessageProcessor - Starting RentalTransactionMessageProcessor message
type with the data <invoice iid=\\\"80000000-d050-163a-1428-ffff08d27045\\\" id=\\\"635694191783997533\\\" dt=\\\"2015-06-09T03:59:38\\\" dt_local=\\\"2015-06-08T20:59:
38\\\" site_iid=\\\"bd20166f-b304-43bd-9ba3-56df4a429b74\\\" user_id=\\\"\\\" cust_id=\\\"805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" cust_name=
\\\" \\\" membership_iid=\\\"d230ef2c-3a92-4841-a1b6-5816f8b36cf8\\\" sub_total=\\\"15\\\" tax_total=\\\"0\\\" total=\\\"15\\\" status_iid=\\\"1\\\" session_id=\\\"80000
000-d050-163a-fd18-ffff08d27044\\\" loyalty_id=\\\"\\\" loyalty_units=\\\"0\\\" loyalty_amount=\\\"15\\\" created_offline=\\\"False\\\" resolution_date=\\\"\\\" email_re
ceipt_sent=\\\"False\\\"><customer id=\\\"805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" cust_iid=\\\"80000000-d14a-163a-4dad-ffff08d1fa67\\\" card
_id=\\\"ns8B3e56P5jFH\\\" expiration=\\\"2019-04-30T00:00:00\\\" date_last_transaction=\\\"2015-01-10T05:07:32\\\" cc_digits=\\\"0976\\\" gov_id=\\\"\\\" email=\\\"AGB.0
2@LIVE.COM\\\" f_name=\\\"\\\" l_name=\\\"\\\" m_name=\\\"\\\" dob=\\\"\\\" addr1=\\\"\\\" addr2=\\\"\\\" city=\\\"\\\" state=\\\"\\\" zip=\\\"99577\\\" country=\\\"\\\"
phone1=\\\"\\\" phone2=\\\"\\\" cc_type=\\\"0\\\" AllowSpecialOffers=\\\"true\\\" AllowReceipts=\\\"true\\\" /><line iid=\\\"80000000-d050-163a-2ab3-ffff08d27045\\\" li
ne_number=\\\"0\\\" type=\\\"1\\\" product_type=\\\"2\\\" product_iid=\\\"37180094-996c-4336-93a8-42e87d4ffd92\\\" product_id=\\\"D000198\\\" piece_id=\\\"E00401007579A2
7B\\\" description=\\\"\\xA1Mujeres Al Ataque!\\\" units=\\\"1\\\" unit_price=\\\"15\\\" days_out=\\\"1\\\" due_dt=\\\"2015-06-09T19:00:00\\\" rent_dt=\\\"2015-06-08T20:
59:00\\\" cost=\\\"0\\\" sub_total=\\\"15\\\" tax_total=\\\"0\\\" line_total=\\\"15\\\" orig_days=\\\"1\\\" orig_due_dt=\\\"2015-06-09T19:00:00\\\" orig_units=\\\"1\\\"
orig_unit_price=\\\"15\\\" parent_invoice=\\\"\\\" payment_iid=\\\"80000000-d050-163a-2b00-ffff08d27045\\\" days_charged=\\\"1\\\" price_desc=\\\"Default\\\" price_days_
out=\\\"1\\\" price_is_partial_day=\\\"false\\\" price_config_amount=\\\"15\\\" price_group_id=\\\"0\\\" return_site_iid=\\\"\\\"><tax iid=\\\"80000001-d050-163a-2ab3-ff
ff08d27045\\\" tax_rate_iid=\\\"5f454d5f-2948-419c-b3fd-573035be6952\\\" tax_zone_iid=\\\"1e52aa32-5ddb-4d28-886d-5e9583b3686a\\\" desc=\\\"\\\" rate=\\\"0\\\" taxable_a
mount=\\\"15\\\" non_taxable_amount=\\\"0\\\" tax_amount=\\\"0\\\" /></line><payment amount=\\\"15\\\" cc_digits=\\\"0976\\\" cc_expiration=\\\"2019-04-30T00:00:00\\\" c
c_name=\\\" \\\" cc_trans_id=\\\"ns8B3e56P5jFH\\\" cc_zip=\\\"99577\\\" cust_id=\\\"805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" payment_iid=\\\"
80000000-d050-163a-2b00-ffff08d27045\\\" payment_dt=\\\"2015-06-09T03:59:38\\\" payment_dt_local=\\\"2015-06-08T20:59:38\\\" method_iid=\\\"3e9547cd-f8d1-41e4-811f-0cbf1
75f4a15\\\" card_id=\\\"ns8B3e56P5jFH\\\" /></invoice>\\r",
"@version" => "1",
"@timestamp" => "2015-07-17T06:43:09.842Z",
"host" => "MSSPC036",
"path" => "\\\\192.168.1.5\\logs/AKPOS_MessageWS_Log4Net.mss.20150414_bak",
"tags" => [
[0] "multiline",
[1] "_grokparsefailure"
],
"time" => "2015-06-09 00:00:07,947",
"thread" => "11",
"loglevel" => "DEBUG",
"class" => "NCR.XE.Component.MessageHandler.HistoryEventMessageProcessor",
"msg" => "Starting RentalTransactionMessageProcessor message type with the data <invoice iid=\\\"80000000-d050-163a-1428-ffff08d27045\\\" id=\\\"6356941917839
97533\\\" dt=\\\"2015-06-09T03:59:38\\\" dt_local=\\\"2015-06-08T20:59:38\\\" site_iid=\\\"bd20166f-b304-43bd-9ba3-56df4a429b74\\\" user_id=\\\"\\\" cust_id=\\\"805be797
27bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" cust_name=\\\" \\\" membership_iid=\\\"d230ef2c-3a92-4841-a1b6-5816f8b36cf8\\\" sub_total=\\\"15\\\" tax_tot
al=\\\"0\\\" total=\\\"15\\\" status_iid=\\\"1\\\" session_id=\\\"80000000-d050-163a-fd18-ffff08d27044\\\" loyalty_id=\\\"\\\" loyalty_units=\\\"0\\\" loyalty_amount=\\\
"15\\\" created_offline=\\\"False\\\" resolution_date=\\\"\\\" email_receipt_sent=\\\"False\\\"><customer id=\\\"805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5
fb66e325\\\" cust_iid=\\\"80000000-d14a-163a-4dad-ffff08d1fa67\\\" card_id=\\\"ns8B3e56P5jFH\\\" expiration=\\\"2019-04-30T00:00:00\\\" date_last_transaction=\\\"2015-01
-10T05:07:32\\\" cc_digits=\\\"0976\\\" gov_id=\\\"\\\" email=\\\"AGB.02@LIVE.COM\\\" f_name=\\\"\\\" l_name=\\\"\\\" m_name=\\\"\\\" dob=\\\"\\\" addr1=\\\"\\\" addr2=\
\\"\\\" city=\\\"\\\" state=\\\"\\\" zip=\\\"99577\\\" country=\\\"\\\" phone1=\\\"\\\" phone2=\\\"\\\" cc_type=\\\"0\\\" AllowSpecialOffers=\\\"true\\\" AllowReceipts=\
\\"true\\\" /><line iid=\\\"80000000-d050-163a-2ab3-ffff08d27045\\\" line_number=\\\"0\\\" type=\\\"1\\\" product_type=\\\"2\\\" product_iid=\\\"37180094-996c-4336-93a8-
42e87d4ffd92\\\" product_id=\\\"D000198\\\" piece_id=\\\"E00401007579A27B\\\" description=\\\"\\xA1Mujeres Al Ataque!\\\" units=\\\"1\\\" unit_price=\\\"15\\\" days_out=
\\\"1\\\" due_dt=\\\"2015-06-09T19:00:00\\\" rent_dt=\\\"2015-06-08T20:59:00\\\" cost=\\\"0\\\" sub_total=\\\"15\\\" tax_total=\\\"0\\\" line_total=\\\"15\\\" orig_days=
\\\"1\\\" orig_due_dt=\\\"2015-06-09T19:00:00\\\" orig_units=\\\"1\\\" orig_unit_price=\\\"15\\\" parent_invoice=\\\"\\\" payment_iid=\\\"80000000-d050-163a-2b00-ffff08d
27045\\\" days_charged=\\\"1\\\" price_desc=\\\"Default\\\" price_days_out=\\\"1\\\" price_is_partial_day=\\\"false\\\" price_config_amount=\\\"15\\\" price_group_id=\\\
"0\\\" return_site_iid=\\\"\\\"><tax iid=\\\"80000001-d050-163a-2ab3-ffff08d27045\\\" tax_rate_iid=\\\"5f454d5f-2948-419c-b3fd-573035be6952\\\" tax_zone_iid=\\\"1e52aa32
-5ddb-4d28-886d-5e9583b3686a\\\" desc=\\\"\\\" rate=\\\"0\\\" taxable_amount=\\\"15\\\" non_taxable_amount=\\\"0\\\" tax_amount=\\\"0\\\" /></line><payment amount=\\\"15
\\\" cc_digits=\\\"0976\\\" cc_expiration=\\\"2019-04-30T00:00:00\\\" cc_name=\\\" \\\" cc_trans_id=\\\"ns8B3e56P5jFH\\\" cc_zip=\\\"99577\\\" cust_id=\\\"805be79727bff9
fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" payment_iid=\\\"80000000-d050-163a-2b00-ffff08d27045\\\" payment_dt=\\\"2015-06-09T03:59:38\\\" payment_dt_local=\
\\"2015-06-08T20:59:38\\\" method_iid=\\\"3e9547cd-f8d1-41e4-811f-0cbf175f4a15\\\" card_id=\\\"ns8B3e56P5jFH\\\" /></invoice>\\r",
"text" => [
[0] "Starting RentalTransactionMessageProcessor message type with the data",
[1] "Starting RentalTransactionMessageProcessor message type with the data <invoice iid=\\\"80000000-d050-163a-1428-ffff08d27045\\\" id=\\\"635694191783997533\\\
" dt=\\\"2015-06-09T03:59:38\\\" dt_local=\\\"2015-06-08T20:59:38\\\" site_iid=\\\"bd20166f-b304-43bd-9ba3-56df4a429b74\\\" user_id=\\\"\\\" cust_id=\\\"805be79727bff9fc
46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325\\\" cust_name=\\\" \\\" membership_iid=\\\"d230ef2c-3a92-4841-a1b6-5816f8b36cf8\\\" sub_total=\\\"15\\\" tax_total=\\\"0
\\\" total=\\\"15\\\" status_iid=\\\"1\\\" session_id=\\\"80000000-d050-163a-fd18-ffff08d27044\\\" loyalty_id=\\\"\\\" loyalty_units=\\\"0\\\" loyalty_amount=\\\"15\\\"
created_offline=\\\"False\\\" resolution_date=\\\"\\\" email_receipt_sent=\\\"False\\\"><customer id=\\\"805be79727bff9fc46b7cad519ca298cccc531a69c03ba00835c73e5fb66e325
\\\" cust_iid=\\\"80000000-d14a-163a-4dad-ffff08d1fa67\\\" card_id=\\\"ns8B3e56P5jFH\\\" expiration=\\\"2019-04-30T00:00:00\\\" date_last_transaction=\\\"2015-01-10T05:0
7:32\\\" cc_digits=\\\"XXXX\\\" gov_id=\\\"\\\" email=\\\"test@test.com\\\" f_name=\\\"\\\" l_name=\\\"\\\" m_name=\\\"\\\" dob=\\\"\\\" addr1=\\\"\\\" addr2=\\\"\\\"
city=\\\"\\\" state=\\\"\\\" zip=\\\"99577\\\" country=\\\"\\\" phone1=\\\"\\\" phone2=\\\"\\\" cc_type=\\\"0\\\" AllowSpecialOffers=\\\"true\\\" AllowReceipts=\\\"true\
\\" /><line iid=\\\"80000000-d050-163a-2ab3-ffff08d27045\\\" line_number=\\\"0\\\" type=\\\"1\\\" product_type=\\\"2\\\" product_iid=\\\"37180094-996c-4336-93a8-42e87d4f
fd92\\\" product_id=\\\"D000198\\\" piece_id=\\\"E00401007579A27B\\\" description=\\\"\\xA1Mujeres Al Ataque!\\\" units=\\\"1\\\" unit_price=\\\"15\\\" days_out=\\\"1\\\
" due_dt=\\\"2015-06-09T19:00:00\\\" rent_dt=\\\"2015-06-08T20:59:00\\\" cost=\\\"0\\\" sub_total=\\\"15\\\" tax_total=\\\"0\\\" line_total=\\\"15\\\" orig_days=\\\"1\\\
" orig_due_dt=\\\"2015-06-09T19:00:00\\\" orig_units=\\\"1\\\" orig_unit_price=\\\"15\\\" parent_invoice=\\\"\\\" payment_iid=\\\"80000000-d050-163a-2b00-ffff08d27045\\\
" days_charged=\\\"1\\\" price_desc=\\\"Default\\\" price_days_out=\\\"1\\\" price_is_partial_day=\\\"false\\\" price_config_amount=\\\"15\\\" price_group_id=\\\"0\\\" r
eturn_site_iid=\\\"\\\"><tax iid=\\\"80000001-d050-163a-2ab3-ffff08d27045\\\" tax_rate_iid=\\\"5f454d5f-2948-419c-b3fd-573035be6952\\\" tax_zone_iid=\\\"1e52aa32-5ddb-4d
28-886d-5e9583b3686a\\\" desc=\\\"\\\" rate=\\\"0\\\" taxable_amount=\\\"15\\\" non_taxable_amount=\\\"0\\\" tax_amount=\\\"0\\\" /></line><payment amount=\\\"15\\\""
],
"InvoiceIID" => "80000000-d050-163a-1428-ffff08d27045",
"ccdigits" => "XXXX"
}
When loading this log line, PieceID field is not creating. First i have a doubt with GREEDYDATA length.Then i checked with this below line(delete some content from that line).It' s working perfectly(means PieceID field created successfully).
2015-06-09 00:00:07,947 [11] DEBUG NCR.XE.Component.MessageHandler.HistoryEventMessageProcessor - Starting RentalTransactionMessageProcessor message type with the data <invoice iid="80000000-d050-163a-1428-ffff08d27045" cc_digits="0976" piece_id="E00401007579A27B" text/></invoice>
Can anybody tell me the problem ?? I am unable to find out the probelm ??
Please Thanks in advance ....