For an example I am going to use the Nightblade's Shadow Ultimate.
If the ONLY Shadow ability I have is the Ultimate skill (none in slots 1-5) will the Shadow skill line still increase without actively using the Ultimate skill?
Yes, it doesn't matter if the skill is a normal skill or the ultimate, if it's on the active bar when you get xp then that skill line will get xp regardless if you use the skill or not.