This myth is thought to have stemmed from a test the U.S. military performed in which subjects wore Arctic survival suits in winter weather. Because their heads were the only parts of their bodies left uncovered, that's where they lost most of their heat. Sounds like a good way to conclude that you lose most of your heat through your head, but not so fast: if the subjects had worn hats and left an equivalent area of their arms or legs uncovered, the same amount of heat would have escaped through those areas.
The reason it feels like more heat escapes through some places than others is that your head, face, neck, and chest are up to five times more sensitive to temperature changes than the rest of your body. By bundling up with a hat and scarf in winter or taking off a ball cap in summer, you feel like you're doing more to change your body temperature than if you were to change other clothing. In reality, roughly 10 percent of your body heat escapes through your head, which just so happens to be the percentage of your body's surface area that your head makes up. Whether you wear a hat or not, the other clothes you wear have just as much of a role in how hot or cold you feel.