# Decoding base64 logs when input is in JSON format?

**URL:** <https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042>\
**Category:** Logstash\
**Created:** [November 2, 2020, 2:22pm UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042 "2020-11-02T14:22:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mudabbir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mudabbir/32/77965_2.png) [@Mudabbir](https://discuss.elastic.co/u/Mudabbir)\
**Post date:** [November 2, 2020, 2:22pm UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042/1 "2020-11-02T14:22:04Z")

</div>

Greetings !

I am unable to decode my base64 logs that are coming from nxlog server.  
Here is my logstash configuration.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/d/4d8502dbfaa695acccfb9e9fab72f5836ad05397.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/a/cafa4a9e4e4c928659a39cb88d0aae76c8f65e91.png)

The error i am getting is shown below

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/3/e3f3cb09afe22df6a648bfad9fa126db738cca07.png)

The log field is in base64  
I have tried cipher filters too and also tried it with ruby.  
Suggest me possble solutions please

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [November 4, 2020, 8:57am UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042/2 "2020-11-04T08:57:12Z")

</div>

Can you avoid sharing screen shots and share your :

- Config file
- Logs

---

<div class="post-metadata">

**Author:** ![Mudabbir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mudabbir/32/77965_2.png) [@Mudabbir](https://discuss.elastic.co/u/Mudabbir)\
**Post date:** [November 4, 2020, 11:28am UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042/3 "2020-11-04T11:28:51Z")

</div>

config file:

input {

tcp {

```
    port => 5142
    type => "ossim-events"
    codec => json {
            charset => "CP1252"
    }

```

}  
}

filter {

mutate {  
add\_field =\> {

```
  "Agent_IP" => "%{host}"
 }

```

}

######## ALIENVAULT OSSIM Logs ########################################

grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:log-level} [%{DATA:class}]:%{GREEDYDATA:message}" }  
}

ruby {

code =\> 'event.set("decoded", Base64.decode64(event.get("[message][embedded][field]["log"]")))'  
}

if [type] == "ossim-events" {  
kv {  
value\_split =\> "='"  
field\_split =\> "' "  
}  
}

}

output {

stdout { }

elasticsearch {

```
    hosts => ["http://localhost:9200"]
    template => ["/var/test/elasticsearch-template.json"]
    template_overwrite => true
    codec => rubydebug

```

}  
}

Logs in console

[2020-11-04T11:24:21,293][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,499 Output [INFO]: event type="detector" date="1604482578" device="192.168.50.90" interface="eth1" plugin\_id="7033" plugin\_sid="1005402" src\_ip="0.0.0.0" dst\_ip="0.0.0.0" username="YXZhcGk=" userdata1="YXZhcGk=" userdata2="L2hvbWUvYXZhcGk=" userdata3="L2Jpbi9zaCAtYyAvdXNyL2Jpbi9weXRob24gL2hvbWUvYXZhcGkvLmFuc2libGUvdG1wL2Fuc2libGUtMTYwNDQ4MjU3Ni4yNS0xNTM5MTc0OTAwOTU4My9hdl9wbHVnaW5zOyBybSAtcmYgL2hvbWUvYXZhcGkvLmFuc2libGUvdG1wL2Fuc2libGUtMTYwNDQ4MjU3Ni4yNS0xNTM5MTc0OTAwOTU4My8g"[truncated 780 chars]; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,499 Output [INFO]: event type="detector" date="1604482578" device="192.168.50.90" interface="eth1" plugin\_id="7033" plugin\_sid="1005402" src\_ip="0.0.0.0" dst\_ip="0.0.0.0" username="YXZhcGk=" userdata1="YXZhcGk=" userdata2="L2hvbWUvYXZhcGk=" userdata3="L2Jpbi9zaCAtYyAvdXNyL2Jpbi9weXRob24gL2hvbWUvYXZhcGkvLmFuc2libGUvdG1wL2Fuc2libGUtMTYwNDQ4MjU3Ni4yNS0xNTM5MTc0OTAwOTU4My9hdl9wbHVnaW5zOyBybSAtcmYgL2hvbWUvYXZhcGkvLmFuc2libGUvdG1wL2Fuc2libGUtMTYwNDQ4MjU3Ni4yNS0xNTM5MTc0OTAwOTU4My8gPi9kZXYvbnVsbCAyPiYx" log="QVYgLSBBbGVydCAtICIxNjA0NDgyNTc4IiAtLT4gUklEOiAiNTQwMiI7IFJMOiAiMyI7IFJHOiAic3lzbG9nLHN1ZG8iOyBSQzogIlN1Y2Nlc3NmdWwgc3VkbyB0byBST09UIGV4ZWN1dGVkIjsgVVNFUjogImF2YXBpIjsgU1JDSVA6ICJOb25lIjsgSE9TVE5BTUU6ICJhbGllbnZhdWx0IjsgTE9DQVRJT046ICIvdmFyL2xvZy9hdXRoLmxvZyI7IEVWRU5UOiAiW0lOSVRdTm92ICA0IDE0OjM2OjE2IGFsaWVudmF1bHQgc3VkbzogICAgYXZhcGkgOiBUVFk9cHRzLzcgOyBQV0Q9L2hvbWUvYXZhcGkgOyBVU0VSPXJvb3QgOyBDT01NQU5EPS9iaW4vc2ggLWMgL3Vzci9iaW4vcHl0aG9uIC9ob21lL2F2YXBpLy5hbnNpYmxlL3RtcC9hbnNpYmxlLTE2MDQ0ODI1NzYuMjUtMTUzOTE3NDkwMDk1ODMvYXZfcGx1Z2luczsgcm0gLXJmIC9ob21lL2F2YXBpLy5hbnNpYmxlL3RtcC9hbnNpYmxlLTE2MDQ0ODI1NzYuMjUtMTUzOTE3NDkwMDk1ODMvID4vZGV2L251bGwgMj4mMVtFTkRdIjsg" fdate="2020-11-04 09:36:18" tzone="5.0" event\_id="1e8111eb-b1e8-000c-293a-51ec30cbabd5""}  
[2020-11-04T11:24:21,294][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,504 Detector [WARNING]: sudo[4005] Event's field src\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,504 Detector [WARNING]: sudo[4005] Event's field src\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"}  
[2020-11-04T11:24:21,295][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,503 Output [INFO]: idm-event username="YXZhcGk=" hostname="alienvault.alienvault" ip="192.168.50.90" inventory\_source="18" rule="IDM\_09""; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,503 Output [INFO]: idm-event username="YXZhcGk=" hostname="alienvault.alienvault" ip="192.168.50.90" inventory\_source="18" rule="IDM\_09""}  
[2020-11-04T11:24:21,300][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,507 Detector [WARNING]: ossec-single-line[7007] Event's field src\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,507 Detector [WARNING]: ossec-single-line[7007] Event's field src\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"}  
[2020-11-04T11:24:21,301][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,506 Detector [WARNING]: sudo[4005] Event's field dst\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,506 Detector [WARNING]: sudo[4005] Event's field dst\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"}  
[2020-11-04T11:24:21,301][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,508 Detector [WARNING]: ossec-single-line[7007] Event's field dst\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,508 Detector [WARNING]: ossec-single-line[7007] Event's field dst\_ip alienvault is not a valid IP.v4/IP.v6 address, falling back to default: 0.0.0.0"}  
[2020-11-04T11:24:21,302][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,508 Detector [WARNING]: sudo[4005] Event's device field alienvault is not a valid IP.v4/IP.v6 address, falling back to default local."; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,508 Detector [WARNING]: sudo[4005] Event's device field alienvault is not a valid IP.v4/IP.v6 address, falling back to default local."}  
[2020-11-04T11:24:21,302][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,511 Output [INFO]: idm-event username="YXZhcGl8JGRvbWFpbg==" hostname="alienvault.alienvault" ip="192.168.50.90" inventory\_source="18""; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,511 Output [INFO]: idm-event username="YXZhcGl8JGRvbWFpbg==" hostname="alienvault.alienvault" ip="192.168.50.90" inventory\_source="18""}  
[2020-11-04T11:24:21,305][WARN][logstash.codecs.jsonlines][main][9ca85e29afa4d481f486364c739c5065899b0274ee9bc49283e78eb315e8ca5a] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values  
at [Source: (String)"2020-11-04 14:36:18,510 Output [INFO]: event type="detector" date="1604482578" device="192.168.50.90" interface="eth1" plugin\_id="4005" plugin\_sid="6" src\_ip="0.0.0.0" dst\_ip="0.0.0.0" log="Tm92ICA0IDE0OjM2OjE4IGFsaWVudmF1bHQgc3VkbzogcGFtX3VuaXgoc3VkbzpzZXNzaW9uKTogc2Vzc2lvbiBjbG9zZWQgZm9yIHVzZXIgcm9vdCA=" fdate="2020-11-04 09:36:18" tzone="5.0" event\_id="1e8111eb-b1e8-000c-293a-51ec3005cbab""; line: 1, column: 6]\>, :data=\>"2020-11-04 14:36:18,510 Output [INFO]: event type="detector" date="1604482578" device="192.168.50.90" interface="eth1" plugin\_id="4005" plugin\_sid="6" src\_ip="0.0.0.0" dst\_ip="0.0.0.0" log="Tm92ICA0IDE0OjM2OjE4IGFsaWVudmF1bHQgc3VkbzogcGFtX3VuaXgoc3VkbzpzZXNzaW9uKTogc2Vzc2lvbiBjbG9zZWQgZm9yIHVzZXIgcm9vdCA=" fdate="2020-11-04 09:36:18" tzone="5.0" event\_id="1e8111eb-b1e8-000c-293a-51ec3005cbab""}

Image from kibana

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/f/9fb047cec8ec4f4daefb24d135a2b8ec2043ba82.png)

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [November 4, 2020, 11:52am UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042/4 "2020-11-04T11:52:38Z")

</div>

You are using the json codec to read events but your events seems not to be a valid json format. Try a simple config to get the format of your events

```
input {

tcp {
    port => 5142
    type => "ossim-events"
}
}

filter {}

output {
	stdout { codec => rubydebug	}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2020, 11:52am UTC](https://discuss.elastic.co/t/decoding-base64-logs-when-input-is-in-json-format/254042/5 "2020-12-02T11:52:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.