Absolutely. Body size is quite malleable.
Increasing brain size allows for increased brain complexity, which is the real object. Much neural development involves culling of neurons. It is the complexity that allows for the distinctive human capabilities. The increase in human brain size is not proportional by region and structure, developing the upper and frontal lobes without much change in the regulatory stuff.