Error parsing json and truncated error in JSON logs

Elastic Stack Logstash
1

Hi,

I am pushing logs through Filebeat and using logstash into Elastic. I have JSON logs and for some of the logs I am getting the error Error parsing json , [truncated 4301 bytes]; line: 1, column: 16] but some of the them are getting parsed. I have checked the JSON log and can confirm that JSON is correct. Although I have tried in my local machine and it is working perfectly with the same confile and version.

Kindly advice.

Regards

2

Thanks for reaching out, @Seemant_Bind. I have a few follow up questions:

  • What version are you using?
  • Do you have sample of the JSON you are using?
3

Hi,

I am using version 7.11.1

Please see the sample JSON below

{"ModifiedDate":"2024-11-04T00:30:44","PaymentRequestId":000001,"PaymentRequestDate":"2024-11-04T00:11:49","ABCId":"0xxxxxf8-2000-4xxx-9003-4xxxxxxxxxx7","CollectionRequestId":000001,"CurrencyCodeAlpha":"USD","CustomerId":11100000,"BankId":"XXX ","ProgramId":00,"PaymentStatus":"Completed","TransferAmount":1,"MobilePhone":"xxxxxxxxxxx","CallbackDate":"2024-11-04T00:30:44","CallbackStatus":"xxx","CallbackBody":{"thirdPartyTransId":"0xxxxxf8-2000-4xxx-9003-4xxxxxxxxxx7","ABCTransId":"000000000000000000","eTransId":"000000000000","fxRate":1.0,"status":{"code":"xxx","message":"Success"},"sendAmount":{"amount":1.0,"currencyCode":"USD"},"receiveAmount":{"amount":1.0,"currencyCode":"USD"}},"ResponseDate":"2024-11-04T00:11:50","ResponseHttpStatus":200,"RequestBody":{"CorporateCode":"XXX","Password":"************************","UniqueKey":"**********","ABCSign":null,"BatchId":"0xxxxxf8-200-4xxxx-9xxx-4xxxxxxxxxxx","RequestBody":[{"InstructionType":{"DestAcctType":"xxxxxxxxxxxx","AmountType":"ReceiveAmount","$type":"InstructionType"},"Amount":{"Amount":1.0,"CurrencyCode":"USD","$type":"InstructionAmount"},"SendFee":null,"Sender":{"Msisdn":"*********","FromCountry":"US","Name":"XXX","Surname":"XXX ","Address":"*********","City":"********","State":"AA","PostalCode":"","Email":null,"DateOfBirth":"*********","Document":null,"$type":"Sender"},"Recipient":{"Msisdn":"*********","ToCountry":"US","Name":"*********","Surname":"*********","Address":"*********","City":"*******","State":"AA","PostalCode":"","Email":null,"DateOfBirth":"*********","Document":null,"DestinationAccount":null,"$type":"Recipient"},"ThirdPartyTransId":"0xxxxxf8-2000-4xxx-9003-4xxxxxxxxxx7","Reference":"xxxPaymentRequestId:000001,ABCAccountId:11100000","$type":"RequestBody"}],"$type":"xxxxxxxxxxxxxxxxxxxxx"},"CorrelationId":"051xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx55","TransactionId":000000000000000000,"ReverseTransactionId":0,"ResponseBody":{"totalTxSent":1,"noTxAccepted":1,"noTxRejected":0,"details":{"transResponse":[{"thirdPartyId":"0a0xxxxxf8-xxxx-4cv8-9003-4xxxxxxxxxx7","status":{"code":"100","message":"Accepted","messageDetail":null}}]},"timestamp":"2024-11-04 05:11:50"},"Otp":"Your verification code is xxxxxx. It expires in 5 minutes.","PhoneCountryCode":"xxx","MobileNetworkPrefix":"xx","MobileNetworkOperator":"xxx","Country":"xxxxx","CreateDate":"2024-11-04T00:11:49"}
4

Thanks @Seemant_Bind, I have another follow up question, how are you configuring Filebeat?

5

I realize you have redacted a lot of data in that, but if you really do have leading zeroes on numbers then the json filter will fail to parse it, since JSON does not allow them.

1 Like