Special Sets are where the power is in this game. Overall you have Magic or Stamina or Health sets. Within those, you have sets that up your damage or resources or regen or some combo.

In a nutshell, you should always wear set gear, or try to. Now, do you always need 5 pieces? Not always, 2 or 3 pieces can be enough in some cases. Once you reach CP160, then you can focus on making 5 piece sets. Even then, I have one CP160 character that has one 5 piece set and 3 two piece sets because I can stack more damage that way.

Note, Dropped Sets have Jewelry for part of the sets so you can wear two 5 piece sets if you want. You get the Jewelry from the chests you get from beating a dolmen.

While leveling, your going to be out-leveling your gear pretty quick, so I would try to focus on 2 and 3 pieces of sets as you go along, by the time you get all 5 pieces, you might have out-leveled the first piece of the set. Note you can be 5 to 10 character levels above the gear before the gear becomes to weak to be of benefit.

Each zone has a Magic or a Stamina or a Health set. So, to complete a 5 piece, you "farm" that zone until you have all 5 pieces. Farming in this case means killing mobs and collecting the gear.

Here are all the sets in the game, both Crafted and Dropped and where to find them

http://elderscrollsonline.wiki.fextralife.com/Sets

And, here are more details about Sets in general and where to find certain pieces.
http://esoacademy.com/set-bonus-items/

There are also sets that are not strictly "stamina, magicka, health" but a mix of those. Shacklebreaker comes to mind, a new set in Vvardenfell, that increases magicka, stamina, weapon damage and spell damage.