I worry over this a lot too, but there's really no reason for it. Most people I know don't care, and if they do, they're pretty up front about it. Also, if you're comfortable with it, you can always ask him to do the trimming/shaving for you.