Indeed the 'cousins' will genetically resemble full-sibs, bar the few mutations that have taken place in the germline of the 4 parents.
Phenotypically they are likely to differ more than 'regular' full sibs because they will have a different maternal environment, both pre-natal and post-natal.
However, the actual full-sibs, i.e. kids born from the same couple will have the same genetic similarity as normal full-sibs: The fact that both their parents are half of a pair of monozygotic twins only affects their genetic similarity with cousins that are chicldren of the other half of the twins.
Of course any of the couples can have a pair of monozygotic ('identical') twins themselves and it is an interesting question whether there is a genetic dsiposition for having monozygotic twins. While both monozygotic and di-zygotic twins are used extensively in human research I have no quick answer whether there is genetic disposition to monozygotic twinning. There is some evidence for a genetic disposition for dizygotic twinning. The latter is more plausible because it is related to ovulation rate while the former is more a process related to embryo development.