# Split message in Logstash does not display complete message **URL:** \ **Category:** Logstash\ **Created:** [October 27, 2023, 3:44pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936 "2023-10-27T15:44:30Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [October 27, 2023, 3:44pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/1 "2023-10-27T15:44:30Z")
Hi All, The stdout log of an application is tokenized with a delimiter `~|~`. There are a total of 5 delimiters. An excerpt from log is as below: ```auto 2023-10-27 11:03:41,294~|~INFO~|~host.com~|~com.controller.LoginController~|~Key, Value : cookie = BCSID=9f347f4e-475b-45b1-b2d5-8d94cd65e608; _cls_v=f492d25b-c0ff-4417-bb45-eee8eb12ff45; _cls_s=f4c06851-b23f-4ae4-8b1a-2febe4c48ae4:0; TU7NGTeXTlLCIxKTEy3M4kU4uGl58QAwyCc2Dmks/SHnWUcy5b6YYm0UWtyV5ipSjCAgA==~-1~-1~-1; ak_bmsc=7ADB4FB55525821AD4469FC6CC9D83BB~000000000000000000000000000000~YAAQcpAFt0LXawYuYXNe9/pvebJHLWHHgLgOqLlaUjwa9nDr5J5tXryUbHFCDyagXeR/kr657VDa3MrckLiKVx5kg==; bm_sz=7CBFD8FECC1D9F4611FCCA8BB0B3D629~YAAQczsvF4kmtlSLAQAAdSOfcRUTDvX9C6Djc4BmniUser=%D2g%u0118%A9%u0122%B7%D8i%B1z%98h%B1%7D%7EJ; prtlnkt=20ec3ef6-161a-43e3-ab35-c2f31a82bce0; AuditUserId=koko7044; E2eeUserID=a29rbzcwNDQ; AuthType=fam; ADRUM_BTa=R:46|g:aec13a90-53ed-496d-93cc-4d781100b238|n:customer1_a6584e1e-55e3-4797-bf22-e75ff520b393; ADRUM_BT1=R:46|i:60813|e:34; bm_sv=C893149D924163E571F206802682573D~YAAQjXjOF1dK4mqLAQAANpGpcRXSejUixVg2sTzxdKRU5D03ZviQBFm26TgT7YWtNv1Eb8EETlC3n8mVVWtQ7u56Cj7EkY41oKafhXvEp1uc=~1~|~ ``` Logstash receives message from filebeat and its config file splits the message in 5 tokens as follows: ```auto mutate { split => ["message", "~|~"] add_field =>{ "createdTime" => "%{[message][0]}" "logLevel" => "%{[message][1]}" "hostname" => "%{[message][2]}" "class" => "%{[message][3]}" "Message" => "%{[message][4]}" ``` I understand multiline option could be exercised at filebeat level, but my question is when we are tokenizing the entire message in 5 distinct tokens, shouldn't the complete message get displayed as %{[message][4]}" ? Because %{[message][4]}" is long text with a number of lines, it results in it not showing completely. Also the lines are not the same in number and pattern every time. Please guide ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [October 27, 2023, 4:32pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/2 "2023-10-27T16:32:35Z")
What is the output you expect? From what you shared the value `[message][4]` will be: > Key, Value : cookie = BCSID=9f347f4e-475b-45b1-b2d5-8d94cd65e608; \_cls\_v=f492d25b-c0ff-4417-bb45-eee8eb12ff45; \_cls\_s=f4c06851-b23f-4ae4-8b1a-2febe4c48ae4:0; TU7NGTeXTlLCIxKTEy3M4kU4uGl58QAwyCc2Dmks/SHnWUcy5b6YYm0UWtyV5ipSjCAgA==~-1~-1~-1; ak\_bmsc=7ADB4FB55525821AD4469FC6CC9D83BB~000000000000000000000000000000~YAAQcpAFt0LXawYuYXNe9/pvebJHLWHHgLgOqLlaUjwa9nDr5J5tXryUbHFCDyagXeR/kr657VDa3MrckLiKVx5kg==; bm\_sz=7CBFD8FECC1D9F4611FCCA8BB0B3D629~YAAQczsvF4kmtlSLAQAAdSOfcRUTDvX9C6Djc4BmniUser=%D2g%u0118%A9%u0122%B7%D8i%B1z%98h%B1%7D%7EJ; prtlnkt=20ec3ef6-161a-43e3-ab35-c2f31a82bce0; AuditUserId=koko7044; E2eeUserID=a29rbzcwNDQ; AuthType=fam; ADRUM\_BTa=R:46|g:aec13a90-53ed-496d-93cc-4d781100b238|n:customer1\_a6584e1e-55e3-4797-bf22-e75ff520b393; ADRUM\_BT1=R:46|i:60813|e:34; bm\_sv=C893149D924163E571F206802682573D~YAAQjXjOF1dK4mqLAQAANpGpcRXSejUixVg2sTzxdKRU5D03ZviQBFm26TgT7YWtNv1Eb8EETlC3n8mVVWtQ7u56Cj7EkY41oKafhXvEp1uc=~1 This is the output that the filters you shared will created based on that sample message: ```auto [2023-10-27T13:30:11,122][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} { "logLevel" => "INFO", "Message" => "Key, Value : cookie = BCSID=9f347f4e-475b-45b1-b2d5-8d94cd65e608; _cls_v=f492d25b-c0ff-4417-bb45-eee8eb12ff45; _cls_s=f4c06851-b23f-4ae4-8b1a-2febe4c48ae4:0; TU7NGTeXTlLCIxKTEy3M4kU4uGl58QAwyCc2Dmks/SHnWUcy5b6YYm0UWtyV5ipSjCAgA==~-1~-1~-1; ak_bmsc=7ADB4FB55525821AD4469FC6CC9D83BB~000000000000000000000000000000~YAAQcpAFt0LXawYuYXNe9/pvebJHLWHHgLgOqLlaUjwa9nDr5J5tXryUbHFCDyagXeR/kr657VDa3MrckLiKVx5kg==; bm_sz=7CBFD8FECC1D9F4611FCCA8BB0B3D629~YAAQczsvF4kmtlSLAQAAdSOfcRUTDvX9C6Djc4BmniUser=%D2g%u0118%A9%u0122%B7%D8i%B1z%98h%B1%7D%7EJ; prtlnkt=20ec3ef6-161a-43e3-ab35-c2f31a82bce0; AuditUserId=koko7044; E2eeUserID=a29rbzcwNDQ; AuthType=fam; ADRUM_BTa=R:46|g:aec13a90-53ed-496d-93cc-4d781100b238|n:customer1_a6584e1e-55e3-4797-bf22-e75ff520b393; ADRUM_BT1=R:46|i:60813|e:34; bm_sv=C893149D924163E571F206802682573D~YAAQjXjOF1dK4mqLAQAANpGpcRXSejUixVg2sTzxdKRU5D03ZviQBFm26TgT7YWtNv1Eb8EETlC3n8mVVWtQ7u56Cj7EkY41oKafhXvEp1uc=~1", "createdTime" => "2023-10-27 11:03:41,294", "@timestamp" => 2023-10-27T16:30:11.114680581Z, "host" => "lab", "message" => [ [0] "2023-10-27 11:03:41,294", [1] "INFO", [2] "host.com", [3] "com.controller.LoginController", [4] "Key, Value : cookie = BCSID=9f347f4e-475b-45b1-b2d5-8d94cd65e608; _cls_v=f492d25b-c0ff-4417-bb45-eee8eb12ff45; _cls_s=f4c06851-b23f-4ae4-8b1a-2febe4c48ae4:0; TU7NGTeXTlLCIxKTEy3M4kU4uGl58QAwyCc2Dmks/SHnWUcy5b6YYm0UWtyV5ipSjCAgA==~-1~-1~-1; ak_bmsc=7ADB4FB55525821AD4469FC6CC9D83BB~000000000000000000000000000000~YAAQcpAFt0LXawYuYXNe9/pvebJHLWHHgLgOqLlaUjwa9nDr5J5tXryUbHFCDyagXeR/kr657VDa3MrckLiKVx5kg==; bm_sz=7CBFD8FECC1D9F4611FCCA8BB0B3D629~YAAQczsvF4kmtlSLAQAAdSOfcRUTDvX9C6Djc4BmniUser=%D2g%u0118%A9%u0122%B7%D8i%B1z%98h%B1%7D%7EJ; prtlnkt=20ec3ef6-161a-43e3-ab35-c2f31a82bce0; AuditUserId=koko7044; E2eeUserID=a29rbzcwNDQ; AuthType=fam; ADRUM_BTa=R:46|g:aec13a90-53ed-496d-93cc-4d781100b238|n:customer1_a6584e1e-55e3-4797-bf22-e75ff520b393; ADRUM_BT1=R:46|i:60813|e:34; bm_sv=C893149D924163E571F206802682573D~YAAQjXjOF1dK4mqLAQAANpGpcRXSejUixVg2sTzxdKRU5D03ZviQBFm26TgT7YWtNv1Eb8EETlC3n8mVVWtQ7u56Cj7EkY41oKafhXvEp1uc=~1" ], "@version" => "1", "hostname" => "host.com", "class" => "com.controller.LoginController" } [2023-10-27T13:30:11,289][INFO][logstash.javapipeline][main] Pipeline terminated {"pipeline.id"=>"main"} ``` ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [October 27, 2023, 5:14pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/3 "2023-10-27T17:14:53Z")
thanks. So in Kibana it does not show ` [message][4]` as a complete message. It seems to break it down into different sub-messages. Also as `[message][4]` is non-standard so different responses are seen. In the case below, it seems that it has taken the message (which is well within the token) as `new line` and lost formatting. I expect the entire message (regardless of complexity) to appear completely as `[message][4]` ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/8/e/8edd53c1ab95500983bd25647219095d8e16407f.png) ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [October 27, 2023, 5:30pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/4 "2023-10-27T17:30:45Z")
This image shows that your `message` is completely different from the one you shared in your first post. As you see the value for the `message` field is just `"name": "displayPassword",`, so there is nothing to split because the string `~|~` does not exist in your message, so you will not have `[message][X]`. It seems that your source message is a multiline message, you would need to configure filebeat to consume the multiline message as one single message. ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [October 27, 2023, 5:59pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/5 "2023-10-27T17:59:42Z")
> [@leandrojmp](#): > > configure filebeat to consume the multiline message as one single message thanks @leandrojmp Yes the content of message field was different the second time around. The message comes in various sizes and formats, because of which I thought I would be able to encapsulate the entire message within one delimiter i.e. ~|~ Also you are right that the message in stdout is a multiline message. But as this message does not conform to a set format, it is hard to configure multiline message pattern in filebeat. Is there a "one size fits all" kind of multiline pattern that I could follow at the filebeat level? TIA ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 24, 2023, 6:00pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936/6 "2023-11-24T18:00:31Z")
Closed on November 24, 2023, 6:00pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.