# Logstash Filter to convert comma separated syslog message body to JSON format **URL:** \ **Category:** Logstash\ **Created:** [March 26, 2019, 3:26pm UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959 "2019-03-26T15:26:11Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![akj01](https://avatars.discourse-cdn.com/v4/letter/a/bcef8e/32.png) [@akj01](https://discuss.elastic.co/u/akj01)\ **Post date:** [March 26, 2019, 3:26pm UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959/1 "2019-03-26T15:26:12Z")
**We have one of the application server forwarding syslog to logstash. The "message" body of the log contains few unwanted fields inserted by the application server and remaining required fields as well. Providing below the sample log format.** { "\_index": “applogs", "\_type": "doc", "\_id": "CKsxa2kB00P1Rp\_OBCpg", "\_score": 1, "\_source": { "@timestamp": "2019-03-11T05:19:28.236Z", "message": "2019-03-11T05:19:28.004Z 10.63.1.223 \<54\>Mar 11 00:18:38 server06 Action: Virus found,IP Address: 10.1.1.1,Computer name: client01,Intensive Protection Level: 0,Certificate issuer: ,Certificate signer: ,Certificate thumbprint: ,Signing timestamp: 0,Certificate serial number: ,Source: Virus scan,Risk name: EICAR Test String,Occurrences: 1,C:\Users\admin01\Desktop\EICAR-Test01.txt,,Actual action: Quarantined,Requested action: Quarantined,Secondary action: Deleted,Event time: 2019-03-11 00:17:06,Inserted: 2019-03-11 00:18:38,End: 2019-03-11 00:17:06,Last update time: 2019-03-11 00:18:38,Domain: Default,Group: Company\Servers001,Server: EVENTSERVER01,User: admin01,Source computer: ,Source IP: ,Disposition: Good,Download site: ,Web domain: ,Downloaded by: ,Prevalence: This file has been seen by millions of Symantec users.,Confidence: This file is trustworthy.,URL Tracking Status: Off,,First Seen: Reputation was not used in this detection.,Sensitivity: ,Not on the permitted application list,Application hash: 275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F,Hash type: SHA2,Company name: ,Application name: [eicars.test.com](http://eicars.test.com),Application version: ,Application type: 127,File size (bytes): 68,Category set: Malware,Category type: Virus,Location: Default", "@version": "1" }, "fields": { "@timestamp": [ "2019-03-11T05:19:28.236Z" ] } } **We need this log to be converted to the below JSON format with only required fields as shown below. Can somebody help with a sample filter which can be used here?** "@timestamp": "2019-03-11T05:19:28.236Z" "IP Address": " 10.1.1.1" "Computer name": " client01" "Intensive Protection Level": " 0" "Certificate issuer": " " "Certificate signer": " " "Certificate thumbprint": " " "Signing timestamp": " 0" "Certificate serial number": " " "Source": " Vius scan" "Risk name": " EICAR Test String" "Occurrences": " 1" "FilePath": " C": "\Users\admin01\Desktop\EICAR-Test0065.txt" "Actual action": " Quarantined" "Requested action": " Quarantined" "Secondary action": " Deleted" "Event time": " 2019-03-11 00": "17": "06" "Inserted": " 2019-03-11 00": "18": "38" "End": " 2019-03-11 00": "17": "06" "Last update time": " 2019-03-11 00": "18": "38" "Domain": " Default" "Group": " Company\Servers001" "Server": " EVENTSERVER01" "User": " admin01" "Source computer": " " "Source IP": " " "Disposition": " Good" "Download site": " " "Web domain": " " "Downloaded by": " " "Prevalence": " This file has been seen by millions of Symantec users." "Confidence": " This file is trustworthy." "URL Tracking Status": " Off" "First Seen": " Reputation was not used in this detection." "Sensitivity": " " "Not on the permitted application list" "Application hash": " 275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F" ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 26, 2019, 3:42pm UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959/2 "2019-03-26T15:42:13Z")
You can parse it using ``` dissect { mapping => { "message" => "%{[@metadata][ts]} %{} <%{}>%{} %{} %{} %{} %{[@metadata][restOfLine]}" } } kv { source => "[@metadata][restOfLine]" field_split => "," value_split => ":" } date { match => ["[@metadata][ts]", ISO8601 ] } ``` To limit the fields you could use either include\_keys or exclude\_keys on the kv filter. ---
**Author:** ![akj01](https://avatars.discourse-cdn.com/v4/letter/a/bcef8e/32.png) [@akj01](https://discuss.elastic.co/u/akj01)\ **Post date:** [March 27, 2019, 3:00am UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959/3 "2019-03-27T03:00:28Z")
its giving the "dissectfailure" error and the output is not getting parsed as required. "@timestamp":"2019-03-11T02:49:31.817Z","tags":**["\_dissectfailure"]**}{"message":"\<54\>Mar 11 21:47:43 server06 Action: Virus found,IP Address: 10.1.1.1,Computer name: client01,Intensive Protection Level: 0,Certificate issuer: ,........... ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 27, 2019, 10:35am UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959/4 "2019-03-27T10:35:45Z")
Your first post said the message format was ```auto "message": "2019-03-11T05:19:28.004Z 10.63.1.223 <54>Mar 11 00:18:38 server06 Action: Virus found,IP Address: 10.1.1.1 ``` The most recent post said it was ```auto <54>Mar 11 21:47:43 server06 Action: Virus found,IP Address: 10.1.1.1 ``` That is a different format and will require a different dissect. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 24, 2019, 10:35am UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959/5 "2019-04-24T10:35:46Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.