Purpose: Exploring the domains and degrees of health-related quality of life (HRQOL) that are affected by the frailty of elders will help clinicians understand the impact of frailty. This association has not been investigated in community-dwelling elders. Therefore, we examined the domains and degree of HRQOL of elders with frailty in the community in Taiwan. Methods: A total of 933 subjects aged 65 years and over were recruited in 2009 from a metropolitan city in Taiwan. Using an adoption of the Fried criteria, frailty was defined by five components: shrinking, weakness, poor endurance and energy, slowness, and low physical activity level. HRQOL was assessed by the short form 36 (SF-36). The multiple linear regression model was used to test the independent effects of frailty on HRQOL. Results: After multivariate adjustment, elders without frailty reported significantly better health than did the pre-frail and frail elders on all scales, and the pre-frail elders reported better health than did the frail elders for all scales except the scales of role limitation due to physical and emotional problems and the Mental Component Summary (MCS). The significantly negative differences between frail and robust elders ranged from 3.58 points for the MCS to 22.92 points for the physical functioning scale. The magnitude of the effects of frail components was largest for poor endurance and energy, and next was for slowness. The percentages of the variations of these 10 scales explained by all factors in the models ranged from 11.1% (scale of role limitation due to emotional problems) to 49.1% (scale of bodily pain). Conclusions: Our study demonstrates that the disabilities in physical health inherent in frailty are linked to a reduction in HRQOL. Such an association between clinical measures and a generic measure of the HRQOL may offer clinicians new information to understand frailty and to conceptualize it within the broader context of disability.