I am trying to figure out whether replicas take up as much memory as primary shards, or do they just require a little bit of overhead?
As much as primary shards. Same amount of space too.
So it has to put into memory all of the parent/child and index information into memory for each primary and replica shard?
Yes. A replica is exactly the same and acts the same as a primary.