You don't have to do guild quests right away. In fact, it isn't recommended. You always get these quests long before you're ready to do them - the old recommended level was 15. Now it shows as 12 (your current level, I assume) because level scaling was implemented, but as I discovered during the final Mages quest, enemy levels don't really scale down (makes just perfect sense...) So you can try to do it, but it will be hard. Just leave it in your journal for now, do something else and pick it up again when you reach Wayrest.

As a side note, it's possible to reach Wayrest at level 12. I once went sightseeing around level 10 and walked all the way to Hallin's Stand in Bangkorai. I didn't even have to sneak a lot, but when it was necessary, I was quite happy my character was a nightblade - Dark Cloak is pretty useful.

You can still go almost anywhere if you stay on the road and carefully walk around dangerous spots. There are only a few places where sneaking (crouch, not Cloak) past is very difficult. I don't remember what's on the way to Wayrest though, so maybe it's one of these places.

Anyhow, you don't need to go there yet. There's a lot to do in western Stormhaven, preparing for the next guild quests.

I also got sent to grahwood for an enchanting writ ( i was in auridon) and on the way there i got killed several times. In the end I crouched and avoided even the random mobs because I wasn't ready for them. ( I did find a few interesting things while avoiding dangerous things!)