Hey man, I'm a taekwondo student and I've gotta say getting bigger is going to improve sparring (in my experience) if you're concerned about flexibility and speed then just train for strength, lower reps and higher weights!
just out of out curiosity what belt grade are you mate?
Actually just got my black belt like a month ago, so I am really trying not to lose too much flexibility now. I think I'll give some lifting with my legs a try tomorrow, something pretty easy just to start off with and get my legs use to lifting again. Low reps and high weights of course.