what I imagined was that there are ones there that guard this portal and its location - not allowing ones to leave Clockwork City. I think there are those in Clockwork City that might be aware of this portal and others who are not.

I don't think the portal is actually there. There is a portal of course but it's not right there in the middle for everyone to access. The location of Clockwork City is one of the most well-kept secrets in the entire Elder Scrolls universe and ESO has been extremely out of line by treating it like it's just any other place, with ordinary NPCs suddenly having access to Clockwork technology, like Skeevatons. Even the Daedric Princes didn't know its location (until the events of Morrowind and Clockwork City).
As WraithCaller88 already pointed out, Divayth Fyr too can leave the city at will and so can Barilzar. Of course Sotha Sil can leave it as well. This leads me to believe that the main reason for nobody being able to leave is that nobody is allowed to leave in order to maintain the secret of its location. Barilzar, Divayth Fyr and the player character are simply trusted individuals who would never expose this secret, even when faced with the threat of a Daedric Prince. That's my best guess.

Also if I remember my lore right Sotha Sil didn't build Clockwork City to be inhabited by the Apostles at all. They simply wished to follow him there and he allowed them to, not exactly caring for their well-being as it was their wish to join him no matter the hardships they'd face. They made it work, but we also saw that it's not exactly the best life. The people that accidentally found their way into the Clockwork City are simply out of luck as they aren't allowed to leave due to the risk of exposing its location.

Ratzkifal wrote: »

Sotha Sil didn't build Clockwork City to be inhabited by the Apostles at all

Wait, but what's on the City being a kind of copy of Nirn or something?.. I know lots of what apostles think of their god and his realm is nonsense but still I always thought Sil actually intended the place to replicate Nirn in some sense. Doesn't that mean that some intelligent inhabitants must be there too? Correct me if I'm wrong please.

m4lenov wrote: »

Ratzkifal wrote: »

Sotha Sil didn't build Clockwork City to be inhabited by the Apostles at all

Wait, but what's on the City being a kind of copy of Nirn or something?.. I know lots of what apostles think of their god and his realm is nonsense but still I always thought Sil actually intended the place to replicate Nirn in some sense. Doesn't that mean that some intelligent inhabitants must be there too? Correct me if I'm wrong please.

The city is an essential part of reaching Nirn-Ensuing but we don't know what part it plays, only that it is essential. We don't know what shape Nirn-Ensuing will take either, but it very much sounds like something that affects all of Nirn and not just the people in the Clockwork City.
From how I understand it, having Apostles live there neither helps nor hinders Sotha Sil in his plans but it does make his followers feel closer to him and they try to emulate his divine work by replicating his augmentations for themselves or doing maintenance on the machines that surround them. The Apostles serve their selfish need to follow in their god's footsteps and living in the Clockwork City and replacing their limbs and organs with machines are part of that.
After all, they do complain about not receiving any direct orders from Sotha Sil the way the followers of Almalexia and Vivec do, so I can only imagine that most of what the Apostles do hasn't been asked of them by Seht, which sounds to me like their existence inside the Clockwork City also wasn't one of his requests.

m4lenov wrote: »

Also if the main thing is to keep the location of the City in secret, Sil can just erase people's memory of it. But I guess he doesn't really care and the Apostles are too asses to do it...

As we've seen, Sotha Sil, is very busy most of the time. And he is immortal too, so I would imagine sometimes he might not leave his throne for more than a mortal lifetime. I doubt he would like having to interrupt his work to erase memories whenever a mortal chickens out and wants to go back. Machines have no use for compassion either so it's entirely possible he simply doesn't care.
I can also imagine that erasing a mind is like erasing data from a harddrive - it just designates the space as empty, it doesn't actually delete it. If that was the case then letting anyone leave could allow Daedra or whoever to capture the people who left and extract that information anyway.