It's mostly a cultural thing. It is fairly common in tribes living in warm temperatures to not wear clothes, or only wear clothes that cover the genitals.
One thing is common though: women genitals are covered in almost all of the known cultures, and if they are not covered by clothing, there are strict rules how to cover them with movement and posture.
This is thought to be because however common the sight may be, men will always be distracted by the vagina. It's hardwired into us.
This is not really true. In fact most "stone age" cultures in hot climates go naked or nearly naked. However most people live in climates that are too cold to be naked for some part of the year. It's these cultures that develop "modesty".
In the Abrhamic Religions (Judaism, Christianity, and Islam) embarrassment of nudity came with the fall of man and so is tied deeply with religion. To be nude is a sin!
In cultures that see humans as different than animals often wear some kind of body decoration that animals do not use to separate themselves from animals.
The origin of clothing is not known but probably predates the arrival of truly modern humans. Most lice species are specific to one species of host. Humans have very little fur and the body lice that infect humans needs to have clothing to live in (head lice is a different species) so clothes had to have been around long enough for the bugs to make this evolutionary change.
There is an indigenous tribe in South America who wear nothing but a thin cord around the middle. The cord hides 0% of their bodies but for them it is unthinkable to go in public without the cord.
People who grow up as naturists are not embarrassed (or titillated) by nudity at all. It's simply what they wear (nothing) to swim or relax or sleep in warm weather. If you think about it wearing clothes while swimming is really stupid. It's uncomfortable to wear clothes. Swim suits slow you down and make it harder to swim. The Swedish Olympic swim team did time trials nude in the 80's and found they had faster times. Unfortunately the Olympic committees (not to mention American Network Television) frowned on nude competitors. Only in the last few years have fabrics been developed that are faster than bare skin.
One of my favorite quotes is "If God had wanted us covered He'd have given us fur".