Governors of towns and castles will resolve all issues with towns and attached villages. In case you were not aware, unresolved quests in towns and villages will harm prosperity and loyalty of towns and castles. This makes it very difficult and grindy to p