Dear community,
We have a requirement where we will determine if the file is successfully uploaded or not based on the doc id, if doc id =-1 then the document is not uploaded and if doc id is other than that document is successfully uploaded.
we are taking number of documents that are uploaded successfully and number of documents that failed while uploading.
The users will upload the files,if the file is not uploaded successfully in such cases we are getting two logs(one with the success count and other as a failed count)
if we fetch the data we are getting one log file with doc id =-1 and other records with doc id not equal to -1.So if document is uploaded successfully after failure we should not include that document in the failure count
please find the sample logs
[07 Jul 2023 10:05:39,947 : DEBUG] - Docupload PC_98000031230391117794.pdf, File Size is: 0 bytes, and Document ID is: -1
[07 Jul 2023 10:06:19,947 : DEBUG] - Docupload PC_98000031230391117794.pdf, File Size is: 42753 bytes, and Document ID is: 842650961
[07 Jul 2023 10:10:45,947 : DEBUG] - Docupload PC_98000031278787878787.pdf, File Size is: 0 bytes, and Document ID is: -1
[07 Jul 2023 10:15:39,947 : DEBUG] - Docupload PC_98000031278787878787.pdf, File Size is: 42753 bytes, and Document ID is: 898989787
[07 Jul 2023 10:20:39,947 : DEBUG] - Docupload PC_98000031230391117794.pdf, File Size is: 42753 bytes, and Document ID is: 787878765
We have tried using unique count in metrics to remove the duplicate records and added the filters to exclude the records that are successfully uploaded but we could not get the proper result.
Is there any way in Kibana to get the failed upload count without the file records that are successfully uploaded after some time.