Plants produce an array of chemical and mechanical defenses that are thought to serve a protective function against herbivores and pathogens, but these putative defenses frequently also occur in floral rewards. For example, the pollen of numerous plant taxa contains high concentrations of toxic compounds or exhibits conspicuous spines. However, while plant defenses strongly restrict host-plant use by insect herbivores, little is known about whether the ability to tolerate such defenses similarly restricts host-plant use in pollinators. In this study, we asked whether bee host-plant use is restricted by plant defenses—that is, do bees parallel insect herbivores in being tolerant of the defenses of their hosts while being harmed by non-host defenses? To test this, we compared the ability of nine species of the bee tribe Osmiini (Megachilidae)to develop on diets containing pollen defenses from their hosts vs. those from co-occurring non-host plants exploited by other Osmiini species. Specifically, we tested the effect of a putative mechanical defense from Asteraceae pollen and a chemical defense from Lupinus pollen. We found that pollen defenses dramatically increase larval mortality of some, but not all, investigated Osmiini species. However, the effects of pollen defenses varied in relation to bee host-plant use, with Osmiinilarvae being able to cope with the defenses of their species’ pollen-hosts. This pattern suggests that bees need physiological or morphological adaptations to cope with the pollen defenses of their hosts and that host-plant use among different bee species is constrained by their ability to tolerate pollen defenses.