Message gets split

Elastic Stack Beats
1

Hi,

I have a log file with a single hit as given below.

2018-01-23 05:13:52,995 INFO org.service.impl.TemplateService - template data after update for templateId: 501 and case-type 1 is {"regionalAnesthesiaValues":{"150":{"valuesArray":["150","114","116","1057","1055","122","124","117","1152","143","1060","1066","1067","1069","1071","1072","1074","1077","1075","1547","1085","1080","1095","1088","1125","1124","1126","1151","1150","326"],"valuesDict":{"143":"1","1151":"Bupivicaine 0.5% 2 cc
Sterile technique","1126":"1","1080":"25","1088":"4.0"},"timestampDict":{}}},"anesthesiaProcedureValues":{"3029":{"outcomeNotes":["3055","3030"],"equipmentNotes":["3049","3045","3031"],"processNotes":["3040","3032"]}},"patientPositioningExtremities":{"PositionExtremities":["3099","3096","56","3098","3095","3092","3091","3089","3090","3082","3088","3079","3068","3078","3065","3064","3060","3063","3061"]},"3059":{"3059":"Verified"},"patientPositionVerified":{"866":"Verified"},"airwaysValues":{"434":{"valuesArray":["911","432","910","919","935","433","994","434","600"],"valuesDict":{},"startTimeEndTimeDict":{}}},"vitalsValues":{"Fluids":[{"defaultFluidID":"2"},{"defaultFluidID":"15"},{"defaultFluidID":"16"},{"fluidID":"1"}],"Monitor":["31"],"PreOPMedications":[{"medID":"98","medsRouteID":"5","medsUnitID":"1"},{"customMedName":"Duramorph","medsRouteID":"5","medsUnitID":"1"},{"customMedName":"Midazolam","medsRouteID":"1","medsUnitID":"1"},{"medID":"30","medsRouteID":"1","medsUnitID":"2"},{"medID":"56","medsRouteID":"1","medsUnitID":"1"},{"medID":"103","medsRouteID":"1","medsUnitID":"1"},{"medID":"357","medsRouteID":"1","medsUnitID":"1"}],"Gases":["defaultOxygen"]}}

But when filebeat sends this to logstash, it comes as two hits(message get split).

First Hit
2018-01-23 05:13:52,995 INFO org.service.impl.TemplateService - template data after update for templateId: 501 and case-type 1 is {"regionalAnesthesiaValues":{"150":{"valuesArray":["150","114","116","1057","1055","122","124","117","1152","143","1060","1066","1067","1069","1071","1072","1074","1077","1075","1547","1085","1080","1095","1088","1125","1124","1126","1151","1150","326"],"valuesDict":{"143":"1","1151":"Bupivicaine 0.5% 2 cc

Second Hit
Sterile technique","1126":"1","1080":"25","1088":"4.0"},"timestampDict":{}}},"anesthesiaProcedureValues":{"3029":{"outcomeNotes":["3055","3030"],"equipmentNotes":["3049","3045","3031"],"processNotes":["3040","3032"]}},"patientPositioningExtremities":{"PositionExtremities":["3099","3096","56","3098","3095","3092","3091","3089","3090","3082","3088","3079","3068","3078","3065","3064","3060","3063","3061"]},"3059":{"3059":"Verified"},"patientPositionVerified":{"866":"Verified"},"airwaysValues":{"434":{"valuesArray":["911","432","910","919","935","433","994","434","600"],"valuesDict":{},"startTimeEndTimeDict":{}}},"vitalsValues":{"Fluids":[{"defaultFluidID":"2"},{"defaultFluidID":"15"},{"defaultFluidID":"16"},{"fluidID":"1"}],"Monitor":["31"],"PreOPMedications":[{"medID":"98","medsRouteID":"5","medsUnitID":"1"},{"customMedName":"Duramorph","medsRouteID":"5","medsUnitID":"1"},{"customMedName":"Midazolam","medsRouteID":"1","medsUnitID":"1"},{"medID":"30","medsRouteID":"1","medsUnitID":"2"},{"medID":"56","medsRouteID":"1","medsUnitID":"1"},{"medID":"103","medsRouteID":"1","medsUnitID":"1"},{"medID":"357","medsRouteID":"1","medsUnitID":"1"}],"Gases":["defaultOxygen"]}}

How can I rectify this issue so that it should come as a single Hit? I use multiline in filebeat but in this case the template data portion(4th field in the hit) can be anything.

Thanks,

2

Looking at the data above it seems you could have a timestamp regexp for the multiline as the second like does not start with a timestamp?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.