Thank you to everyone who provided answers here, it's greatly appreciated!
As many of you have noted, The Greybeards were formed by Jurgen Windcaller (Later called Jurgen the Calm) after he spent years pondering the reasons why his army was destroyed during the War of Succession.
Like Stygies_VIII mentioned, (Thanks for the link too!) Jurgen came to realise that misuse of the Thu'um (weaponizing it) was what made the Nord gods turn away from them, and so he formed the Greybeards out of a pacifist idea, in which the Thu'um would only be used to pray and worship their gods.
Super interesting stuff, we could honestly read about the lore for days on end!