Duplicate collection information


(Little Five) #1

Log background description:
The log is a log of the java program, which is automatically divided into a single log file every day by day.

After collecting the log information from the filebeat to the elasticsearch, the number of log messages collected and the number of logs in the log source file are inconsistent (repeated).
Check the filebeat collection log, found that only collected once, not repeated collection, but do not know why the elasticsearch inside, the log information is repeated?

Kibana info
Time                              message
July 16th 2018, 18:06:20.081WARN  [2018-07-16 18:06:20,081]-[localhost-startStop-1] org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor:394 - Autowired annotation is not supported on static fields: private static org.slf4j.Logger com.iqb.onigiri.investor.login.service.impl.UserLRService.log 
July 16th 2018, 18:06:20.081WARN  [2018-07-16 18:06:20,081]-[localhost-startStop-1] org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor:394 - Autowired annotation is not supported on static fields: private static org.slf4j.Logger com.iqb.onigiri.investor.login.service.impl.UserLRService.log 
July 16th 2018, 17:36:41.246WARN  [2018-07-16 17:36:41,246]-[localhost-startStop-1] org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor:394 - Autowired annotation is not supported on static fields: private static org.slf4j.Logger com.iqb.onigiri.investor.login.service.impl.UserLRService.log 
July 16th 2018, 17:36:41.246WARN  [2018-07-16 17:36:41,246]-[localhost-startStop-1] org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor:394 - Autowired annotation is not supported on static fields: private static org.slf4j.Logger com.iqb.onigiri.investor.login.service.impl.UserLRService.log 

Number of lines after linux command filtering
[elk@FTJF-Test-Java web.onigiri]$ egrep 'WARN' *2018-07-16* |wc -l
 2

3„ÄĀFilebeat collection log
[elk@FTJF-Test-Java filebeat-6.2.4-linux-x86_64]$ cat nohup.out | grep 'web.onigiri' | grep '2018-07-16'
2018-07-17T18:18:30.620+0800	INFO	log/harvester.go:216	Harvester started for file: /var/log/onigiri/web.onigiri/log-info-2018-07-16.0.log
2018-07-17T18:18:37.988+0800	INFO	log/harvester.go:216	Harvester started for file: /var/log/onigiri/web.onigiri/log-warn-2018-07-16.0.log
2018-07-17T18:23:42.254+0800	INFO	log/harvester.go:241	File is inactive: /var/log/onigiri/web.onigiri/log-info-2018-07-16.0.log. Closing because close_inactive of 5m0s reached.
2018-07-17T18:23:43.069+0800	INFO	log/harvester.go:241	File is inactive: /var/log/onigiri/web.onigiri/log-warn-2018-07-16.0.log. Closing because close_inactive of 5m0s reached.

(Mark Walkom) #2

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them. It's much better if you copy and paste the text, and format it using the </> button.


(Little Five) #3

Ok, I have modified it according to your requirements. Please help me to see my problem.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.