This study aimed to examine the moderating factors of the association between perceived sexual stigma from family and peers and internalized homonegativity, as well as to compare the effects of perceived sexual stigma from family and peers and internalized homonegativity on loneliness, depression, and anxiety in gay and bisexual men. In total, 400 gay and bisexual men participated in this study. The experiences of perceived sexual stigma from family and peers on the Homosexuality subscale of the HIV and Homosexuality Related Stigma Scale, internalized homonegativity on the Measure of Internalized Sexual Stigma for Lesbians and Gay Men, loneliness on the UCLA Loneliness Scale (Version 3), depression on the Center for Epidemiological Studies-Depression Scale, and anxiety on the State subscale of the State-Trait Anxiety Inventory were collected. The results indicated that perceived sexual stigma from family and peers was significantly associated with internalized homonegativity in both gay and bisexual men, and that sexual orientation moderated the association. Moreover, the association between internalized homonegativity and loneliness was significantly greater than that between perceived sexual stigma from family and peers and loneliness, although no significant differences were observed in their associations with depression and anxiety. Intervention programs that promote changes in the attitudes toward gay and bisexual men among the general population are needed to help prevent the development of internalized homonegativity and further loneliness, depression and anxiety.