Every line of Logs is not passing through

Elastic Stack Logstash
1

Hello All ,
I have below sample line in from the log file which is not passing through logstash. I am not sure where the problem is , I have the below log line

2021-02-24 10:46:29,827 INFO [5b5021c6-6af1-4e70-9cfb-f099386a8890] AzureServiceBus.Processors.MessageProcessor - Processing message {
"Body": "PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48RmxlZXRWZWhpY2xlQ2hhbmdlRXZlbnQgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSI+PEZsZWV0PjxGbGVldElEPjQ1Mjc0MDAwMDM8L0ZsZWV0SUQ+PFZlaGljbGU+PFZJTj5XMVQ5NjM0MjMyMDQzOTAxMDwvVklOPjxDdXN0b21lclZlaGljbGVOYW1lPkJWNzAgQVlVPC9DdXN0b21lclZlaGljbGVOYW1lPjxTZXJ2aWNlRGVhbGVyT3V0bGV0SWQ+R1MwMDEzOTQ1PC9TZXJ2aWNlRGVhbGVyT3V0bGV0SWQ+PC9WZWhpY2xlPjwvRmxlZXQ+PGFjdGlvbj5BZGQ8L2FjdGlvbj48L0ZsZWV0VmVoaWNsZUNoYW5nZUV2ZW50Pg==",
"MessageId": "ID:414d5120585a3046303031312020202032883560ecbc0a20",
"ExpiresAtUtc": "9999-12-31T23:59:59.9999999",
"TimeToLive": "10675199.02:48:05.4775807",
"CorrelationId": "10fb5c17-bfbd-40a3-985c-52b157a0c94f",
"Label": "FleetVehicleChangeEvent",
"ContentType": "application/xml",
"ScheduledEnqueueTimeUtc": "0001-01-01T00:00:00",
"Size": 361,
"UserProperties": {},
"SystemProperties": {
"IsLockTokenSet": true,
"LockToken": "f2b26158-5a76-411d-bf14-49074d4d320b",
"IsReceived": true,
"DeliveryCount": 1,
"LockedUntilUtc": "2021-02-24T09:47:29.728Z",
"SequenceNumber": 212214,
"EnqueuedSequenceNumber": 934441,
"EnqueuedTimeUtc": "2021-02-24T09:46:22.805Z"
},
"BodyAsString": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><FleetVehicleChangeEvent xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance\">4527400003W1T96342320439010BV70 AYUGS0013945Add"
}

Grok pattern :

%{TIMESTAMP_ISO8601:EventDate}%{SPACE}%{GREEDYDATA:EventType}%{SPACE}\[%{NOTSPACE:MessageId}\]%{GREEDYDATA:LogMessage}

Multiline pattern from filebeat :

   - //XXXX/XX/Log.txt
    - //XXXX/XX/Log.txt
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  # Defines if the pattern set under pattern should be negated or not. Default is false.
  multiline.negate: true
  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
  # that was (not) matched before or after or as long as a pattern is not matched based on negate.
  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
  multiline.match: after
  tail_files: false

Kindly help me with the debugging steps , FYI , I am getting all the other lines from the same log line.

Thanks and Regards

2

I'm not sure what the problem is either. Are you getting error messages from logstash? If so what are they? What makes you think the events are "not passing through logstash"?

3

Hello @Badger ,

I don't see any error in the logs. I confirmed that they are not passing because i don't see this particular line in Kibana, where as i can see all the other lines .

For example , below event i can see in Kibana .

2021-03-10 06:22:17,471 INFO [997c7dad-c95c-45ea-a62f-14a1f1279364] FleetboardListener.MessageReceiverService - Handling message with Label 'FleetVehicleChangeEvent

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.