Hi!, I'm using packetbeat to collect data about our billing, monitoring the numbers of inserts in a mysql database table, whatever sometimes it have differences between the numbers returned from elasticsearch and the table inserted lines. I've checked out the cpu, memory, network and logs but it is all ok on the period that it lose the packages. Ex: "February 14th 2016" 6pm to 7pm are 7395 elasticsearch lines versus 11051 table lines.
I'm sure that the only entry of lines on the table came from this channel, and don't have another source inserting on table.
It happens sometimes because I've compared other values of elastic and mysql, and it was correct.
Ex: Another hours
February 15th 2016, 6pm to 7pm are 4225 in elastic versus 4225 table lines.
February 15th 2016, 13pm to 14pm are 13219 in elastic versus 13219 table lines.
I just verified using tcpdump, between the host and the elasticsearch there are no packet lost.
interfaces: device: eth0 type: af_packet snaplen: 1500 buffer_size_mb: 500 protocols: http: ports: [80,443] send_response: true send_request: true send_headers: ["User-Agent"] real_ip_header: "X-Forwarded-For" include_body_for: ["text/html", "application/json" ,"application/xhtml+xml", "application/xml"] mysql: ports:  max_rows: 100 max_row_length: 5000 output: elasticsearch: enabled: true worker: 2 hosts: ["node1","node2","node3"] max_retries: 15 bulk_max_size: 1500 flush_interval: 20 shipper: name: sender1 tags: ["sender"] logging: to_syslog: false to_files: true files: path: /var/log/ name: packetbeat.log rotateeverybytes: 10485760 # = 10MB keepfiles: 7 selectors: ["*"] level: error
packetbeat version 1.1.1 (amd64)
System: Linux ip-10-1-0-174 4.1.13-19.30.amzn1.x86_64 #1 SMP Fri Dec 11 03:42:10 UTC 2015 x86_64 GNU/Linux