If you think the size of those frameworks is too large to build a fast site, then don't use those frameworks, though I'm guessing the frameworks aren't the problem as the companies who make and use...