You are not doing anything wrong, you just missed the glowing bug which was leading you to the next quest objective. If I remember correctly, you are looking for two sculls in some burnt tree houses. When you come close to them, there will be a marker on your compass so you will not miss those. The burnt houses can be found to the north from the Driladan Pass on the map, about half way to the City of Ash and you see them as light brown circles on the map. I also remember that for me those houses were outside of the large white circle which marks the questing area on the map...

Sorry for the messy explanation, but I hope it will help