A fever is part of your body's immune response to an infection. Perhaps combating the infection with an unfavorable body temperature for the microbe, perhaps part of increased activity in the body. Essentially it is a change in your body's temperature set-point.
Generally it is not recommended to physically cool the body unless you have an extreme temperature, perhaps greater than about 105°F (40.5°C) that isn't responsive to ordinary antipyretic drugs, in which case you certainly should be consulting a physician.
For "ordinary" fevers, you should try to get comfortable. If you feel cold, put on extra blankets. If you feel hot, take off some blankets.
The shivering is a sign that your body thinks it should be warmer by initiating muscle movements to generate heat.
Sweating would be a sign that your body thinks it should be cooler.