Logstash how to parse different format message in one logfile

Elastic Stack Logstash
1

The original log
07/20/20 09:23:14 INFO login 2493 us.(channel/Channel.cpp:575) 07/20/20 09:23:14 INFO deletecity:cn(city/CityDeleteBusiness.cpp:55) 07/20/20 09:23:14 INFO ---p_dbSession->deleteCity() 14066 useconds.(city/CityDeleteBusiness.cpp:137) 07/20/20 09:23:14 INFO delete city 23792 us.(channel/Channel.cpp:575) 07/20/20 09:23:14 INFO logout 506 us.(channel/Channel.cpp:575) 07/20/20 09:25:55 INFO login 2441 us.(channel/Channel.cpp:575) 07/20/20 09:25:55 INFO deletecity:cn(city/CityDeleteBusiness.cpp:55) 07/20/20 09:25:55 INFO ---p_dbSession->deleteCity() 13846 useconds.(city/CityDeleteBusiness.cpp:137) 07/20/20 09:25:55 INFO delete city 28047 us.(channel/Channel.cpp:575) 07/20/20 09:25:55 INFO logout 524 us.(channel/Channel.cpp:575) 07/20/20 09:28:57 INFO login 2851 us.(channel/Channel.cpp:575) 07/20/20 09:28:57 INFO deletecity:cn(city/CityDeleteBusiness.cpp:55) 07/20/20 09:28:57 INFO ---p_dbSession->deleteCity() 13335 useconds.(city/CityDeleteBusiness.cpp:137) 07/20/20 09:28:57 INFO delete city 39199 us.(channel/Channel.cpp:575) 07/20/20 09:28:57 INFO createcity:中国(city/CityDeleteBusiness.cpp:55) 07/20/20 09:28:57 INFO logout 1005 us.(channel/Channel.cpp:575) 07/20/20 16:18:30 INFO updatecity:cn(city/CityUpdateBusiness.cpp:106) 07/20/20 16:18:30 INFO updatecity:cn(city/CityUpdateBusiness.cpp:106) 07/20/20 16:18:30 INFO infocity:cn(city/CityInfoBusiness.cpp:134) 07/20/20 16:18:30 INFO infocity:cn(city/CityInfoBusiness.cpp:134) 07/20/20 16:18:31 INFO updatecity:cn(city/CityUpdateBusiness.cpp:106)

  • Log formats in this file are not counted,The final desired format

split xxx us and (xxxxxxxxxxxx.cpp:xxx)
07/20/20 09:23:14 INFO login 07/20/20 09:23:14 INFO deletecity:cn 07/20/20 09:23:14 INFO logout 07/20/20 09:25:55 INFO login 07/20/20 09:25:55 INFO deletecity:cn 07/20/20 09:25:55 INFO logout 07/20/20 09:28:57 INFO login 07/20/20 09:28:57 INFO deletecity:cn 07/20/20 09:28:57 INFO createcity:中国 07/20/20 09:28:57 INFO logout 07/20/20 16:18:30 INFO updatecity:cn

2

Hi there,

I'm afraid I didn't really get what you'd like to achieve here. Can you explain it a bit further? Thanks

3

logfile has three format ,hwo to parse

07/20/20 00:48:00 INFO poll 512370 us.(channel/Channel.cpp:575)
07/20/20 00:48:00 INFO poll
07/20/20 00:48:28 INFO transfercity:bj(city/cityTransferQueryBusiness.cpp:72)
07/20/20 00:48:28 INFO transfercity:bj
07/20/20 00:48:29 INFO transferQuery city 965380 us.(channel/Channel.cpp:575)
07/20/20 00:48:29 INFO transferQuery city

The bold part is the final one you want, and the one after INFO is split into two fields if there is a colon, otherwise it will be a whole field whether there is space or not

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.