No, the starter islands do not have any Guildhalls. You will need to head to the first main city (Daggerfall for you) to join the Fighters Guild and Mages Guild.
Each of the NPC guilds has a special skill line that can only be unlocked if you're a member. Each guild's questline is also pretty well written if you like the questing aspect of things. Beyond that, there's some gear that you get as quest rewards, and of course every quest will give you gold and XP.
(The term "guild" is also used to refer to player guilds, which are something else entirely.)