I have this problem, especially in Delves. It can disturb to open a door, use an object or pick up whatever (search on a dead body...). For me, it's just one more of those QoL peanuts: it would be nice to have it improved some day, but it goes without too.