Send json log file to logstash with logstash forwarder but see wrong log in kibana


(Soniaeli) #1

I have an app that writes logs to a json file that named allapp.json, Each log entry is a JSON object. An example of it looks like the following:

"{\"timestamp\":\"2015-08-30 19:42:26.724\",\"MAC_Address\":\"A8:7C:01:CB:2D:09\",\"DeviceID\":\"96f389972de989d1\",\"RunningApp\":\"null{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}\",\"PhoneName\":\"samsung\",\"IP\":\"192.168.1.101\"}"

I want send my log to logstash, so logstash-forwarder.conf configured to:

"network": {  "servers": [ "192.168.1.102:5000" ],  "timeout": 15,  "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"  "files": [  {  "paths":  ["/home/fr/allapp.json"],  "fields": { "type": "json" }  }  ]

and in /etc/logstash/conf.d/ i have two conf file: 01-lumberjack-input.conf:

 input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
} }

30-lumberjack-output.conf :

output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
} 

when i run /opt/logstash-forwarder/bin/logstash-forwarder -config /etc/logstash-forwarder.conf i see:

2015/09/01 01:01:52.194573 Waiting for 1 prospectors to initialise
2015/09/01 01:01:52.194867 Launching harvester on new file: /home/fr/allapp.json
2015/09/01 01:01:52.195326 harvest: "/home/fr/allapp.json" (offset snapshot:0)
2015/09/01 01:01:52.217068 All prospectors initialised with 0 states to persist
2015/09/01 01:01:52.217317 Setting trusted CA from file: /etc/pki/tls/certs/logstash-forwarder.crt
2015/09/01 01:01:52.256222 Connecting to [192.168.1.102]:5000 (192.168.1.102) 
2015/09/01 01:01:52.502025 Connected to 192.168.1.102      `

At the end in kibana i just see logs like this in far date:

  `        Time     _source     August 30th 2015, 10:21:46.000     message:
    Aug 29 22:51:46 ubuntu sudo: pam_unix(sudo:session): session closed for user root @version:
    1
@timestamp:
    August 30th 2015, 10:21:46.000
type:
    syslog
file:
    /var/log/auth.log
host:
    ubuntu
offset:
    16355
syslog_timestamp:
    Aug 29 22:51:46
syslog_hostname:
    ubuntu
syslog_program:
    sudo
syslog_message:
    pam_unix(sudo:session): session closed for user root
received_at:
    August 30th 2015, 10:21:47.899
received_from:
    ubuntu
syslog_severity_code:
    5
syslog_facility_code:
    1
syslog_facility:
    user-level
syslog_severity:
    notice
_source:
    {"message":"Aug 29 22:51:46 ubuntu sudo: pam_unix(sudo:session): session closed for user root","@version":"1","@timestamp":"2015-08-30T05:51:46.000Z","type":"syslog","file":"/var/log/auth.log","host":"ubuntu","offset":"16355","syslog_timestamp":"Aug 29 22:51:46","syslog_hostname":"ubuntu","syslog_program":"sudo","syslog_message":"pam_unix(sudo:session): session closed for user root","received_at":"2015-08-30T05:51:47.899Z","received_from":"ubuntu","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice"}
_id:
    AU99KWDPEqXW0EMoRjPr
_type:
    syslog
_index:
    logstash-2015.08.30

as you see at file: /var/log/auth.log above, this is not my logs from allapp.json file, it look like all Elasticsearch, Logstash, ngix and Kibana work without error but the answer i get is wrong and i am not sure where i wrong.


(Mark Walkom) #2

Looks like there is other config files being processed, can you see other config files for that file?


(Soniaeli) #3

No,there isn't config file in /etc/logstash/ or /etc/logstash-forwarde/ . in fact at the first when i installed logstash forwarder based a tutorial at url:https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04
logstash-forwarder.conf was contain syslog file path but I changed it
afterward and restarted service logstash-forwarder and others but result in kibana
didn't change I checked all of config files many times but config files
are like i wrote in this question,


(system) #4