Hello,
I use a filebeat instance to read some log files and send them to a logstash instance on another server to store them as a file output.
On one of the logs, line order is not critical, so I haven't encountered this issue before, but on the new log type I send, the line order is very important and the timestamp permits an accuracy to the second only (and many events occurs in one second).
Here is an example of the initial log read by filebeat :
[root@opaxvpol1 log]# cat log_prd_2018_10_30.log | grep "QyjYW/COKwE" opaxvgw4, QyjYW/COKwE 20181030 104539 102 NET I CONN_IND (477937) incoming connection indication [src_add="xxx/63227",dest_add="xxx/6321"] opaxvgw4, QyjYW/COKwE 20181030 104539 106 SECS I SES_INIT Server(1843946) Net profile GW_SFTP selected [src_add="xxx/63227"] [dest_add="xxx/6321"] opaxvgw4, QyjYW/COKwE 20181030 104539 111 SECS I SES_INIT Server(1843946) SSH profile GATEWAY_SSH_SRV selected opaxvgw4, QyjYW/COKwE 20181030 104539 103 NET I CONN_RESP (477937) incoming connection response [resp_add=""] opaxvgw4, QyjYW/COKwE 20181030 104539 114 SECS I SIGN_OK Server(1843946) DSS signature achieved with private key GATEWAY_PRIV opaxvgw4, QyjYW/COKwE 20181030 104539 003 PCNX I CONN (27009770) SFTP Connection Request Received opaxvgw4, QyjYW/COKwE 20181030 104539 004 PCNX I CONN (27009770) calling addr="xxx/63227", called_addr="xxx/6321" opaxvgw4, QyjYW/COKwE 20181030 104539 004 PCNX I CONN (27009770) login="MLC01", pwd="" opaxvgw4, QyjYW/COKwE 20181030 104539 001 PCNX I SEL (27009770) CGate xxx selected opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) template_site = "TSFTP" [GSFTP] opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) root_directory = "/MLC01" [MLC01], home_directory = "/MLC01" [MLC01] opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) route_local_agent = [], route_remote_agent = [] opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) route_originator_ident = [], route_destination_ident = [] opaxvgw4, QyjYW/COKwE 20181030 104539 118 SECS I SIGN_OK Server(1843946) DSS signature verification achieved with public key CLT_MLC01 opaxvgw4, QyjYW/COKwE 20181030 104539 102 SECS S SES_SUC Server(1843946) Session established for user MLC01, key exchange algo: dh-group1, public key algo: ssh-dss, cipher algo: aes128-cbc, mac algo: md5, no compression opaxvgw4, QyjYW/COKwE 20181030 104539 067 NET I SSHINFO (477937) SSH incoming connection from Client SSH-VERSION-STRING:SSH-2.0-JSCH-0.1.54 Local Server SSH-VERSION-STRING:SSH-2.0-XFB.Gateway Unix opaxvgw4, QyjYW/COKwE 20181030 104539 019 SFTP I XFERSND2 GREENTRF(175857) [0] begin sending from , LIST: opaxvgw4, QyjYW/COKwE 20181030 104539 023 SFTP I XENDSND2 GREENTRF(175857) [0] end sending from , LIST: opaxvgw4, QyjYW/COKwE 20181030 104539 106 NET I DISC_IND (477937) disconnection indication [reason="Success (0x0)"] [origin="0"] opaxvgw4, QyjYW/COKwE 20181030 104539 104 SECS I SES_END Server(1843946) Session ended for user MLC01
And here is the file written on the logstash side :
[root@opsgyst1 ~]# cat /opt/application/splunk/logs_data/axv/log_prd_2018_10_30.log | grep "QyjYW/COKwE" opaxvgw4, QyjYW/COKwE 20181030 104539 106 SECS I SES_INIT Server(1843946) Net profile GW_SFTP selected [src_add="xxx/63227"] [dest_add="xxx/6321"] opaxvgw4, QyjYW/COKwE 20181030 104539 001 PCNX I SEL (27009770) CGate xxx selected opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) route_originator_ident = [], route_destination_ident = [] opaxvgw4, QyjYW/COKwE 20181030 104539 019 SFTP I XFERSND2 GREENTRF(175857) [0] begin sending from , LIST: opaxvgw4, QyjYW/COKwE 20181030 104539 103 NET I CONN_RESP (477937) incoming connection response [resp_add=""] opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) root_directory = "/MLC01" [MLC01], home_directory = "/MLC01" [MLC01] opaxvgw4, QyjYW/COKwE 20181030 104539 102 SECS S SES_SUC Server(1843946) Session established for user MLC01, key exchange algo: dh-group1, public key algo: ssh-dss, cipher algo: aes128-cbc, mac algo: md5, no compression opaxvgw4, QyjYW/COKwE 20181030 104539 106 NET I DISC_IND (477937) disconnection indication [reason="Success (0x0)"] [origin="0"] opaxvgw4, QyjYW/COKwE 20181030 104539 111 SECS I SES_INIT Server(1843946) SSH profile GATEWAY_SSH_SRV selected opaxvgw4, QyjYW/COKwE 20181030 104539 003 PCNX I CONN (27009770) SFTP Connection Request Received opaxvgw4, QyjYW/COKwE 20181030 104539 004 PCNX I CONN (27009770) calling addr="xxx/63227", called_addr="xxx/6321" opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) template_site = "TSFTP" [GSFTP] opaxvgw4, QyjYW/COKwE 20181030 104539 118 SECS I SIGN_OK Server(1843946) DSS signature verification achieved with public key CLT_MLC01 opaxvgw4, QyjYW/COKwE 20181030 104539 023 SFTP I XENDSND2 GREENTRF(175857) [0] end sending from , LIST: opaxvgw4, QyjYW/COKwE 20181030 104539 102 NET I CONN_IND (477937) incoming connection indication [src_add="10.228.175.75/63227",dest_add="10.117.40.6/6321"] opaxvgw4, QyjYW/COKwE 20181030 104539 114 SECS I SIGN_OK Server(1843946) DSS signature achieved with private key GATEWAY_PRIV opaxvgw4, QyjYW/COKwE 20181030 104539 004 PCNX I CONN (27009770) login="MLC01", pwd="" opaxvgw4, QyjYW/COKwE 20181030 104539 005 PCNX I CONN (27009770) route_local_agent = [], route_remote_agent = [] opaxvgw4, QyjYW/COKwE 20181030 104539 067 NET I SSHINFO (477937) SSH incoming connection from Client SSH-VERSION-STRING:SSH-2.0-JSCH-0.1.54 Local Server SSH-VERSION-STRING:SSH-2.0-XFB.Gateway Unix opaxvgw4, QyjYW/COKwE 20181030 104539 104 SECS I SES_END Server(1843946) Session ended for user MLC01
Is there an option on either side (logsatsh or filebeat) to ensure that lines ordering is kept ?
Thank you