I must have taken Cadwell's portal to start the quests, but somehow started something else and now can't find my way back to get started again. I can't talk to Cadwell. What can I do? It's the only quest I have.
I think what's happening is that you're trying to use the quest markers to see Cadwell's Silver quest's next step.
This particular quest doesn't work that way - not that there's any indication of this!
Really all it means is that you go to the Cadwell's Silver tab in your Journal and see the long list of quests in each of the 5 zones of the Alliance (I can't say which, not knowing your own alliance) which you need to complete to complete the overarching super-quest of Cadwell's Silver, much like the Lights of Meridia in Coldharbour.
I hope I explained that well. Basically, it means Cadwell's Silver, if it's in your journal, is all you need. Just complete the quests on that list.