File content is:
{"#account_id":"43DDBF5E46FBC68F2B9C395DA7C3443B","#distinct_id":"205aa1ec49b21576da4ee3beb994f132","#type":"track","#ip":"122.157.17 5.52","#time":"2020-01-06 19:44:54","#event_name":"TALENT_CAPTURE","properties":{"role_uid":1135771889734230,"current_server":1001,"c reate_server":1001,"channel":"32","role_name":"[æ¡<83>]鬼é<85>","role_create_time":"2019-11-02 12:15:21","total_rmb":0,"role_level" :100,"unionid_uid":2254052399020837219,"unionid_name":"å<90><8d>æ<89>¬å<9b><9b>æµ·","ad":"","device":"205aa1ec49b21576da4ee3beb994f13 2","os":"1","os_version":"Android OS 5.1 / API-22 (LMY47O/J5008ZMS1ARG1)","role_type":0,"fighting":256025,"pay":false,"exp":127210,"l evel":100,"money":0,"gift_money":0,"gold":17627,"daohang":18683,"contribution":8394,"achieve":1240,"vigour":16,"country":"0","dungeon _ticket":0,"friend_point":10,"account_time":"2019-11-02 12:13:54","idfa":"","TALENT_CAPTURE_talent_tree":"é<98>´é<98>³","TALENT_CAPTU RE_talent_list":"»ù´¡Ì츳£¨ÒõÑô£©","TALENT_CAPTURE_talent_name":"»ù´¡·ÀÓù","TALENT_CAPTURE_talent_level":5,"TALENT_CAPTURE_tree_level ":3}}
{"#account_id":"84576032","#distinct_id":"95099e577eff0ae2d7d6b8db6eb664d2","#type":"track","#ip":"103.228.209.175","#time":"2020-01-06 18:49:32","#event_name":"TALENT_CAPTURE","properties":{"role_uid":1135770815430664,"current_server":1001,"create_server":1001,"channel":"134","role_name":"é<87><8e>æ<8d><8f>å§<91>计贼","role_create_time":"2019-09-12 10:25:25","total_rmb":5771,"role_level":100,"unionid_uid":2254052399020837219,"unionid_name":"å<90><8d>æ<89>¬å<9b><9b>æµ·","ad":"","device":"95099e577eff0ae2d7d6b8db6eb664d2","os":"1","os_version":"Android OS 10 / API-29 (HUAWEIPCT-AL10/10.0.0.187C00)","role_type":0,"fighting":578437,"pay":true,"exp":127210,"level":100,"money":0,"gift_money":13,"gold":59812,"daohang":18803,"contribution":349,"achieve":1405,"vigour":33,"country":"0","dungeon_ticket":0,"friend_point":4860,"account_time":"2019-09-12 10:16:43","idfa":"","TALENT_CAPTURE_talent_tree":"å<91>½","TALENT_CAPTURE_talent_list":"»ù´¡Ì츳£¨Ãü£©","TALENT_CAPTURE_talent_name":"¸ß¼¶ÉúÃü","TALENT_CAPTURE_talent_level":5,"TALENT_CAPTURE_tree_level":4}}
{"#account_id":"84576032","#distinct_id":"95099e577eff0ae2d7d6b8db6eb664d2","#type":"track","#ip":"103.228.209.175","#time":"2020-01-06 18:49:32","#event_name":"TALENT_CAPTURE","properties":{"role_uid":1135770815430664,"current_server":1001,"create_server":1001,"channel":"134","role_name":"é<87><8e>æ<8d><8f>å§<91>计贼","role_create_time":"2019-09-12 10:25:25","total_rmb":5771,"role_level":100,"unionid_uid":2254052399020837219,"unionid_name":"å<90><8d>æ<89>¬å<9b><9b>æµ·","ad":"","device":"95099e577eff0ae2d7d6b8db6eb664d2","os":"1","os_version":"Android OS 10 / API-29 (HUAWEIPCT-AL10/10.0.0.187C00)","role_type":0,"fighting":578437,"pay":true,"exp":127210,"level":100,"money":0,"gift_money":13,"gold":59812,"daohang":18803,"contribution":349,"achieve":1405,"vigour":33,"country":"0","dungeon_ticket":0,"friend_point":4860,"account_time":"2019-09-12 10:16:43","idfa":"","TALENT_CAPTURE_talent_tree":"å<91>½","TALENT_CAPTURE_talent_list":"»ù´¡Ì츳£¨Ãü£©","TALENT_CAPTURE_talent_name":"ðÏÕѵÁ·","TALENT_CAPTURE_talent_level":4,"TALENT_CAPTURE_tree_level":4}}
logstash version is: 7.6.0
my logstash configration file is:
input {
file {
path => "/data/datacenter/td_data/server_*/*"
codec => "json"
start_position => "beginning"
sincedb_path => "/root/.sincedb_7ffa64810fcb36c4ee4110e93c729542"
sincedb_write_interval => 10
codec => {
charset => "UTF-8"
}
}
}
filter {
json {
source => "message"
}
date {
match => ["#time", "yyyy-MM-dd HH:mm:ss"]
}
if [path] =~ /ONLINE_NUM_[0-9]/ {
mutate {
add_field => [ "[@metadata][index_name]" , "ONLINE_NUM" ]
}
} else if [path] =~ /TALENT_[0-9]/ {
mutate {
add_field => [ "[@metadata][index_name]" , "TALENT" ]
}
}
else {
mutate {
add_field => [ "[@metadata][index_name]" , "TALENTxxxx" ]
}
}
}
output {
mongodb {
# codec => line {format => "%{message}"}
uri => "mongodb://172.19.209.83:28017"
database => "doujiang"
collection => "%{[@metadata][index_name]}"
}
stdout { codec => rubydebug }
}
when i started logstash, some error happend, but when i use input plugin, enter single whole line each time, it works successfully,
what's wrong with my configration files?
the error infomation is:
[2021-01-08T15:00:02,241][WARN ][logstash.filters.json ][main] Error parsing json {:source=>"message", :raw=>"{\"#account_id\":\"171355430\",\"#distinct_id\":\"38549fe83fc6c431cde37148c49beb2d\",\"#type\":\"track\",\"#ip\":\"223.104.63.4\",\"#time\":\"2020-01-06 09:21:20\",\"#event_name\":\"TALENT_CAPTURE\",\"properties\":{\"role_uid\":1135776184197152,\"current_server\":1003,\"create_server\":1003,\"channel\":\"15\",\"role_name\":\"长安危玄静\",\"role_create_time\":\"2019-09-21 09:36:08\",\"total_rmb\":5980,\"role_level\":100,\"unionid_uid\":2278822232313014479,\"unionid_name\":\"诗情画意\",\"ad\":\"\",\"device\":\"38549fe83fc6c431cde37148c49beb2d\",\"os\":\"1\",\"os_version\":\"Android OS 6.0 / API-23 (MRA58K/V10.2.2.0.MBFCNXM)\",\"role_type\":0,\"fighting\":464710,\"pay\":true,\"exp\":127210,\"level\":100,\"money\":0,\"gift_money\":805,\"gold\":13275,\"daohang\":18803,\"contribution\":385,\"achieve\":1370,\"vigour\":50,\"country\":\"0\",\"dungeon_ticket\":3,\"friend_point\":450,\"account_time\":\"2019-09-21 09:34:02\",\"idfa\":\"\",\"TALENT_CAPTURE_talent_tree\":\"阴阳\",\"TALENT_CAPTURE_talent_list\":\"\xBD\xF8\xBD\xD7\xCC츳\xA3\xA8\xD2\xF5\xD1\xF4\xA3\xA9\",\"TALENT_CAPTURE_talent_name\":\"\xBB\xF9\xB4\xA1\xB7\xC0\xD3\xF9\",\"TALENT_CAPTURE_talent_level\":10,\"TALENT_CAPTURE_tree_level\":10}}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('' (code 92)): was expecting double-quote to start field name
at [Source: (byte)"{"#account_id":"171355430","#distinct_id":"38549fe83fc6c431cde37148c49beb2d","#type":"track","#ip":"223.104.63.4","#time":"2020-01-06 09:21:20","#event_name":"TALENT_CAPTURE","properties":{"role_uid":1135776184197152,"current_server":1003,"create_server":1003,"channel":"15","role_name":"长安危玄静","role_create_time":"2019-09-21 09:36:08","total_rmb":5980,"role_level":100,"unionid_uid":2278822232313014479,"unionid_name":"诗情画意",""[truncated 720 bytes]; line: 1, column: 3]>}
Blockquote
please help me, thank you