This maybe a little out there, but I've never heard anything say that it couldn't happen. (Swedish15 made me think of this), what if there were a slayer that had an affinity for magic as well. I don't think they necessarily cancel each other out, but it seems like the focus on physical fighting would distract any slayer from really looking into the mystical.
Wouldn't really be such an issue after season 7 since there are a few hundred slayers (at least) running around leaving some open to more "imaginative" endeavors.