Having a dick doesn't make you responsible for others with dicks, but that's the thing about sexist men with dicks--they don't listen to women. That's their whole deal. They only change their behavior when challenged by another person with a dick. So no, you're not responsible for other men, but you could help a gal out by speaking up. Your choice of course.