On the second floor in the eastern "room" there is a really big chest.
You have to wonder why the goblins put a small key in such a big chest, but they did. You'll need this key to unlock the door to the exterior rampart of the Keep on the third floor, where you'll find an enemy guarding yet another key. The second key opens the doors to the north of the third floor. While you're on the third floor, collect the chests in each corner. Here is a map of the third floor:
