Why didn't they just destroy the stone around the sword?

Because that would have just been dumb wouldn't it have? Besides, its whoever pulls the sword from the stone, not, break the sword out of the stone