First of all, have you delivered the letter to the farmer?
When you talk to him, he'll tell you something important about the guards.
They like sandwiches!
Perhaps you can make a sandwich for them. You have a sandwich spell in your spell book.