If you leave an instanced dungeon, raid or arena it will usually take a few minutes before it resets. I think this behaviour was added, so you wouldn't loose all your progress when you have a short disconnect or have to leave the instance for some other reason.
You have to wait like 5 minutes before it resets. You used to be able to reset a dungeon solo by switching normal and vet around. They changed the ability to do that a while back, because of course they did.