I know its "possible" to completely ignore magic in the game due to the fighters guild abilities. But it does not really seem viable to use stamina for every attack you do and also sprint dodge roll and sneak when needed. I just find it funny because in the very last TES game magic was frowned by the native race of skyrim for the most part and it was quite viable to use only stamina based attacks.