In an app with 2 nodejs services that connect to a 3-node mongodb replicaset, where is the best place to install packetbeat? On the nodejs boxes of on the mongodb boxes? Are there any pros/cons?
Thank you very much.
It needs to be on the mongo boxes.
there are different deployment strategies for packetbeat. If possible I'd prefer port mirroring over installing packetbeat on any of the servers.
Can you give some more details about your setup? You have physical boxes + switch? Running in Cloud?
Thanks.
The two node apps are running in different virtual machines in an openstack public cloud environment, so I don't think we can do port mirroring.
We use two apps with a load balancer (nginx) to have some HA. They connect to the mongodb replica set, that is running in three different boxes in different availability zones. All the boxes are running Linux (currently RedHat and CentOS).
As you're interested in MongoDB only, I'd put packetbeat on MongoDB VMs. Alternatively you can run packetbeat on you app servers (getting HTTP+MongoDB traffic), but depending on amount of traffic packetbeat might require s it's share of CPU usage.
Thanks.
One last question, do you think that using iptables to do port mirroring would be a good idea?
Never tried mirroring packets via iptables myself. In your case I'd test this in some test-env first before breaking my actual installation.
Clear disadvantes of forwarding packets:
- increased bandwidth (limitations, e.g. rate limiting by cloud provider?)
- duplicate packets if two hosts forward packets for same TCP connect ( + bandwidth is 3 times higher). Duplicate packets are no big problem for TCP module and packet analyzer, but if the 2 servers 'race' with forwarding packets there is a chance of false positive on packet loss logic (need to drop TCP stream state and resync).
Advantage is, actual monitoring is decoupled from application server.