Do Replicas take up memory?

I am trying to figure out whether replicas take up as much memory as primary shards, or do they just require a little bit of overhead?

As much as primary shards. Same amount of space too.

So it has to put into memory all of the parent/child and index information into memory for each primary and replica shard?

Yes. A replica is exactly the same and acts the same as a primary.