I have an app that writes logs to a json file that named allapp.json, Each log entry is a JSON object. An example of it looks like the following:
"{\"timestamp\":\"2015-08-30 19:42:26.724\",\"MAC_Address\":\"A8:7C:01:CB:2D:09\",\"DeviceID\":\"96f389972de989d1\",\"RunningApp\":\"null{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}{com.tools.app_logs\\/com.tools.app_logs.Main}{com.gtp.nextlauncher\\/com.gtp.nextlauncher.LauncherActivity}{com.android.settings\\/com.android.settings.Settings$WifiSettingsActivity}{com.android.incallui\\/com.android.incallui.InCallActivity}\",\"PhoneName\":\"samsung\",\"IP\":\"192.168.1.101\"}"
I want send my log to logstash, so logstash-forwarder.conf configured to:
"network": { "servers": [ "192.168.1.102:5000" ], "timeout": 15, "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt" "files": [ { "paths": ["/home/fr/allapp.json"], "fields": { "type": "json" } } ]
and in /etc/logstash/conf.d/ i have two conf file: 01-lumberjack-input.conf:
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
} }
30-lumberjack-output.conf :
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}
when i run /opt/logstash-forwarder/bin/logstash-forwarder -config /etc/logstash-forwarder.conf i see:
2015/09/01 01:01:52.194573 Waiting for 1 prospectors to initialise
2015/09/01 01:01:52.194867 Launching harvester on new file: /home/fr/allapp.json
2015/09/01 01:01:52.195326 harvest: "/home/fr/allapp.json" (offset snapshot:0)
2015/09/01 01:01:52.217068 All prospectors initialised with 0 states to persist
2015/09/01 01:01:52.217317 Setting trusted CA from file: /etc/pki/tls/certs/logstash-forwarder.crt
2015/09/01 01:01:52.256222 Connecting to [192.168.1.102]:5000 (192.168.1.102)
2015/09/01 01:01:52.502025 Connected to 192.168.1.102 `
At the end in kibana i just see logs like this in far date:
` Time _source August 30th 2015, 10:21:46.000 message:
Aug 29 22:51:46 ubuntu sudo: pam_unix(sudo:session): session closed for user root @version:
1
@timestamp:
August 30th 2015, 10:21:46.000
type:
syslog
file:
/var/log/auth.log
host:
ubuntu
offset:
16355
syslog_timestamp:
Aug 29 22:51:46
syslog_hostname:
ubuntu
syslog_program:
sudo
syslog_message:
pam_unix(sudo:session): session closed for user root
received_at:
August 30th 2015, 10:21:47.899
received_from:
ubuntu
syslog_severity_code:
5
syslog_facility_code:
1
syslog_facility:
user-level
syslog_severity:
notice
_source:
{"message":"Aug 29 22:51:46 ubuntu sudo: pam_unix(sudo:session): session closed for user root","@version":"1","@timestamp":"2015-08-30T05:51:46.000Z","type":"syslog","file":"/var/log/auth.log","host":"ubuntu","offset":"16355","syslog_timestamp":"Aug 29 22:51:46","syslog_hostname":"ubuntu","syslog_program":"sudo","syslog_message":"pam_unix(sudo:session): session closed for user root","received_at":"2015-08-30T05:51:47.899Z","received_from":"ubuntu","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice"}
_id:
AU99KWDPEqXW0EMoRjPr
_type:
syslog
_index:
logstash-2015.08.30
as you see at file: /var/log/auth.log above, this is not my logs from allapp.json file, it look like all Elasticsearch, Logstash, ngix and Kibana work without error but the answer i get is wrong and i am not sure where i wrong.