# Var/audit logs messages include wierd characters

**URL:** <https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 23, 2019, 3:54pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443 "2019-01-23T15:54:14Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![qnator](https://avatars.discourse-cdn.com/v4/letter/q/7bcc69/32.png) [@qnator](https://discuss.elastic.co/u/qnator)\
**Post date:** [January 23, 2019, 3:54pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/1 "2019-01-23T15:54:14Z")

</div>

Hello,

My current setup on my machine is:

Elasticsearch 6.5.4  
Kibana 6.5.4  
logstash 6.5.4

On a remote machine:  
filebeat 6.5.4

The output for filebeat is logstash and its configured to pull for the var/audit folder. I'm able to see the var/audit/\*.not\_terminated logs but the message body contains weird characters.

How do I filter this out?

See below for an example of the garbled test:

f��@�J�(launchd::Audit recovery#)/var/audit/20190117194635.crash\_recovery'�f9��@�J�(launchd::Audit startup'�9}�e@�Jyqasflags-am\_success-am\_failure$������'�}X��@�K@$��������(begin evaluation'�Xv��@�KC$��������(config.modify.com.apple.wifi(config.modify.'�v���@�KC$��������(config.modify.com.apple.wifi(client /usr/libexec/airportd(creator /usr/libexec/airportd'��V��@�KC$��������(end evaluation'�V}�e@�M�q0asflags-am\_success-am\_failure$������'�}}�e@�M�qasflags-am\_success-am\_failure$������'�}}�g@�M�qasflags-am\_success-am\_failure$������'�}X��@�N�$����a��b(begin evaluation'�Xt��@�N�$����a��b(system.login.console(system.login.console'�t��@�NH$����a��b(system.login.console( mechanism builtin:policy-banner'����@�O�$����\\��(EVerify password for record type Users 'admin' node '/Local/Default''�������@�X$����\\��

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 24, 2019, 1:38pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/2 "2019-01-24T13:38:45Z")

</div>

Are you using the `beats` input in logstash?

---

<div class="post-metadata">

**Author:** ![qnator](https://avatars.discourse-cdn.com/v4/letter/q/7bcc69/32.png) [@qnator](https://discuss.elastic.co/u/qnator)\
**Post date:** [January 24, 2019, 9:00pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/3 "2019-01-24T21:00:01Z")

</div>

Yes I am, see below for my 02-beats-input.conf file:

input {  
beats {  
port =\> 5044  
ssl =\> false  
}  
}

# The filter part of this file is commented out to indicate that it

# is optional.

filter {  
if [fileset][module] == "system" {  
if [fileset][name] == "auth" {  
grok {  
match =\> { "message" =\> ["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: %{DATA:[system][auth][ssh][event]} %{DATA:[system][auth][ssh][method]} for (invalid user )?%{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]} port %{NUMBER:[system][auth][ssh][port]} ssh2(: %{GREEDYDATA:[system][auth][ssh][signature]})?",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: %{DATA:[system][auth][ssh][event]} user %{DATA:[system][auth][user]} from %{IPORHOST:[system][auth][ssh][ip]}",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sshd(?:[%{POSINT:[system][auth][pid]}])?: Did not receive identification string from %{IPORHOST:[system][auth][ssh][dropped\_ip]}",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} sudo(?:[%{POSINT:[system][auth][pid]}])?: \s\*%{DATA:[system][auth][user]} ☹ %{DATA:[system][auth][sudo][error]} ;)? TTY=%{DATA:[system][auth][sudo][tty]} ; PWD=%{DATA:[system][auth][sudo][pwd]} ; USER=%{DATA:[system][auth][sudo][user]} ; COMMAND=%{GREEDYDATA:[system][auth][sudo][command]}",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} groupadd(?:[%{POSINT:[system][auth][pid]}])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} useradd(?:[%{POSINT:[system][auth][pid]}])?: new user: name=%{DATA:[system][auth][user][add][name]}, UID=%{NUMBER:[system][auth][user][add][uid]}, GID=%{NUMBER:[system][auth][user][add][gid]}, home=%{DATA:[system][auth][user][add][home]}, shell=%{DATA:[system][auth][user][add][shell]}$",  
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostname]} %{DATA:[system][auth][program]}(?:[%{POSINT:[system][auth][pid]}])?: %{GREEDYMULTILINE:[system][auth][message]}"] }  
pattern\_definitions =\> {  
"GREEDYMULTILINE"=\> "(.|\n)_"  
}  
remove\_field =\> "message"  
}  
date {  
match =\> ["[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]  
}  
geoip {  
source =\> "[system][auth][ssh][ip]"  
target =\> "[system][auth][ssh][geoip]"  
}  
}  
else if [fileset][name] == "syslog" {  
grok {  
match =\> { "message" =\> ["%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[system][syslog][hostname]} %{DATA:[system][syslog][program]}(?:[%{POSINT:[system][syslog][pid]}])?: %{GREEDYMULTILINE:[system][syslog][message]}"] }  
pattern\_definitions =\> { "GREEDYMULTILINE" =\> "(.|\n)_" }  
remove\_field =\> "message"  
}  
date {  
match =\> ["[system][syslog][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]  
}  
}  
}  
}

output {  
elasticsearch {  
hosts =\> "X.X.X.X:9200"  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"  
}  
}

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 25, 2019, 12:26pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/4 "2019-01-25T12:26:41Z")

</div>

Have you tried to open the file locally? Is the file pure text? Which encoding does the original file use?

From system/auth manifest file:

```auto
    os.darwin:
      # this works in OS X < 10.8. Newer darwin versions don't write
      # ssh logs to files
      - /var/log/secure.log*

```

---

<div class="post-metadata">

**Author:** ![qnator](https://avatars.discourse-cdn.com/v4/letter/q/7bcc69/32.png) [@qnator](https://discuss.elastic.co/u/qnator)\
**Post date:** [January 28, 2019, 4:19pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/5 "2019-01-28T16:19:43Z")

</div>

I'm working on macOS High Sierra. I don't see a secure.log in the /var/log/ directory.

when I cat the var/audit/\*.not\_terminated log, I get the following below. How can I check the encoding of this file?

attribute = 'dsAttrTypeStandard:AuthenticationAuthority', value(s) = ';ShadowHash;HASHLIST:\<SALTED-SHA512-PBKDF2,SRP-RFC5054-4096-SHA512-PBKDF2\>,;Kerberosv5;;oadmin@LKDC:SHA1.2346091D099B7705A2515E4DE344ED3444AA1F4B;LKDC:SHA1.2346091D099B7705A2515E4DE344ED3444AA1F4B;,;SecureToken;''???  
??\O  
????=??=(EVerify password for record type Users 'oadmin' node '/Local/Default''??X ??\O ?L????4??5(begin evaluation'?Xx  
??\O  
?L$????4??5(system.privilege.admin(system.privilege.admin'?x?  
??\O  
?L$????4??5(system.privilege.admin(3client /usr/local/McAfee/AntiMalware/VShieldUpdate(4creator /usr/local/McAfee/AntiMalware/VShieldUpdate'??V  
??\O  
?L$????4??5(end evaluation'?VX  
??\O  
?]????4??5(begin evaluation'?Xx ??\O ?^????4??5(system.privilege.admin(system.privilege.admin'?x?  
??\O  
?^????4??5(system.privilege.admin(,client /usr/libexec/security\_authtrampoline(4creator /usr/local/McAfee/AntiMalware/VShieldUpdate'??V ??\O ?^????4??5(end evaluation'?VX  
??\O  
??????4??5(begin evaluation'?Xx ??\O ??????4??5(system.privilege.admin(system.privilege.admin'?x?

---

<div class="post-metadata">

**Author:** ![qnator](https://avatars.discourse-cdn.com/v4/letter/q/7bcc69/32.png) [@qnator](https://discuss.elastic.co/u/qnator)\
**Post date:** [January 28, 2019, 5:00pm UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/6 "2019-01-28T17:00:13Z")

</div>

the command praudit cleans up the file:

header,86,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:29 2019, + 94 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,end evaluation

return,success,0

trailer,86

header,88,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 398 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,begin evaluation

return,success,0

trailer,88

header,120,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 399 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,system.privilege.admin

text,system.privilege.admin

return,success,0

trailer,120

header,203,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 399 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,system.privilege.admin

text,client /usr/local/McAfee/AntiMalware/VShieldUpdate

text,creator /usr/local/McAfee/AntiMalware/VShieldUpdate

return,success,0

trailer,203

header,86,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 399 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,end evaluation

return,success,0

trailer,86

header,88,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 418 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,begin evaluation

return,success,0

trailer,88

header,120,11,SecSrvr AuthEngine,0,Mon Jan 28 09:04:33 2019, + 419 msec

subject,-1,root,wheel,root,wheel,52,100000,53,0.0.0.0

text,system.privilege.admin

text,system.privilege.admin

return,success,0

trailer,120

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [January 29, 2019, 6:34am UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/7 "2019-01-29T06:34:36Z")

</div>

I'd say this confirms the file format actually being binary, not pure text.  
Feel free to open an enhancement request for macOS audit logs: [https://github.com/elastic/beats/issues/new/choose](https://github.com/elastic/beats/issues/new/choose)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 26, 2019, 6:34am UTC](https://discuss.elastic.co/t/var-audit-logs-messages-include-wierd-characters/165443/8 "2019-02-26T06:34:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.