The answer depends on who’s doing the injuring and who’s getting injured while practicing self defense…
Bigger, faster, and stronger are great attributes to have, if you know what to do with them:
If you have more mass, you can hit harder. If you have more strength, you can hang on nice and tight for joint breaks, and send people flying with throws. If you’re quick, you can get in and get it done before he even knows what’s happening.
So yeah, bigger, faster, and stronger can make a difference in practicing self defense for you if you have them.
If you don’t have those attributes, if you’re not bigger or stronger or faster than most people (and let’s be honest, most of us aren’t), then that’s where proper training makes a huge difference. You need training to know how to throw your 140 pounds through someone to get same results that a 240-pound guy gets accidentally. You need training in leverage and timing to break joints and send people flying as if you were amazingly strong. You need to be practicing self defense that gets you to act earlier in the sequence of events rather than later if you’re slow.
Because being bigger, stronger and faster do nothing to make you (or him) immune to violence.
Size, strength, speed and other assets of physical conditioning can help you absorb non-specific trauma—in other words, it can help you “take a punch.” This fact can be seen in MMA or combat sports competitions. But no one, no matter how big, strong, fast and tough they are, can take injury as we define it (or as the criminal sociopath defines it). No one can take a gouged eye, a crushed throat, or a broken knee.
The best example of this can be seen in American football. When a player gets his leg bent backwards until the knee snaps, what we have is a highly trained and highly developed athlete taking a crippling, game-ending (for him) injury. If bigger-stronger-faster conferred immunity to physical harm, these guys would have it. But they don’t. They break just like the rest of us.
In the end, bigger, stronger, and faster are positives for causing injury, but do nothing to protect you from it. As long as you take this idea the right way, it’s actually a lot of good news. If you have those attributes, you just need to be practicing self defense that puts them to good use. If you don’t have those attributes, you just need a little training to learn how to make up for them.
If the other guy has those attributes, it does nothing to protect him from you. No matter how big, strong, or fast he looks, he breaks just like everybody else. He might be able to “take a punch” (and probably more than just one) but you’re not going to punch him. You’re going to gouge out his eye, crush his throat, and tear out his knee.
You’re going to do the things to him that nobody can take, the things that work regardless of size, strength or speed—no matter who has them or who doesn’t.