There are few longitudinal data on change in bone structure and muscle mass, strength and function in later life. We report these, and consider bone–muscle interrelationships in older men and women. We studied 188 men and 166 women from the Hertfordshire Cohort Study, who underwent peripheral quantitative computed tomography (pQCT) of the radius and tibia in 2004–2005 and then again in 2011–2012. Grip strength and gait speed were also assessed at both timepoints. Percentage change per year was calculated for grip strength, gait speed, muscle cross-sectional area (mCSA), fat cross-sectional area (fCSA) and diaphyseal bone parameters [total area (Tt.Ar), cortical area (Ct.Ar), cortical density (cBMD) and trabecular density (tBMD)]. The mean (SD) age of men and women at baseline was 68.9 (2.5) and 69.2 (2.6) years, respectively. Rates of muscle area and strength loss did not differ by sex. Tt.Ar increased with age and faster in men [mean (SD) 1.78 (1.64) %/year] than women [mean (SD) 1.03 (1.69) %/year] in the radius (p < 0.001). In both the radius (p = 0.006) and tibia (p < 0.001), Ct.Ar reduced more rapidly in women than men. Change in Ct.Ar was associated with change in muscle area in the corresponding limb (radius; men: regression coefficient 0.36, 95% CI 0.20–0.52, p < 0.001; tibia; men: regression coefficient 0.14, 95% CI 0.00–0.27, p = 0.043, women: regression coefficient 0.16, 95% CI 0.01–0.30, p = 0.032). We have demonstrated that muscle strength and function decrease faster than muscle mass and have provided further evidence that changes in bone structure with age differ by sex.