We employed a whole body magnetic resonance imaging protocol to examine the influence of age, gender, body weight, and height on skeletal muscle (SM) mass and distribution in a large and heterogeneous sample of 468 men and women. Men had significantly (P < 0.001) more SM in comparison to women in both absolute terms (33.0 vs. 21.0 kg) and relative to body mass (38.4 vs. 30.6%). The gender differences were greater in the upper (40%) than lower (33%) body (P < 0.01). We observed a reduction in relative SM mass starting in the third decade; however, a noticeable decrease in absolute SM mass was not observed until the end of the fifth decade. This decrease was primarily attributed to a decrease in lower body SM. Weight and height explained approximately 50% of the variance in SM mass in men and women. Although a linear relationship existed between SM and height, the relationship between SM and body weight was curvilinear because the contribution of SM to weight gain decreased with increasing body weight. These findings indicate that men have more SM than women and that these gender differences are greater in the upper body. Independent of gender, aging is associated with a decrease in SM mass that is explained, in large measure, by a decrease in lower body SM occurring after the fifth decade.