If there are two pairs of twins, A and a and B and b, and A marries B and a marries b...
A and a are genetically as close to each other as they are to themselves (that is, not all of anyone's cells are exactly the same because of copying errors.. but identical twins result from division of one embryo at an early stage).
Likewise B and b.
So when A and B have babies, those babies have 23 chromosomes, one of each from each of their parents. Each chromosome inherited from A is a shuffled-up mixture of the two chromosomes A inherited from her parents (well, except the sex chromosomes, but we'll ignore them..). Likewise, each baby will inherit a copy of each chromosome from B.
None of A and B's children will have chromosomes that are identical (unless A and B also produce identical twins), but they will each have 50 % of their genetic material from each of A and B.
Because a and b are genetically identical to A and B respectively, their children will be genetically as similar to their cousins as to their brothers and sisters.
Note that although the genetics are identical that doesn't necessarily mean that there wouldn't be other differences. Even identical twins have different experiences starting as soon as the embryo divides. One twin may get a stronger blood supply and send up bigger, or have an easier or harder birth, or catch a tummy bug or an ear infection that affects their developement over a few days, and these could all have small effects that might cancel out or act cumulatively to affect their physical health or behaviour, and these might lead to a different environment for the production of sperm (in the bloke) at the time of having kids, or might affect the environment in the womb (for the woman), or might affect the children's home lives and whether they look more or less similar.