It wouldn't hurt to talk to people.
Some will refer you to someone else. Once you know who else you should talk to and what to say, follow up on the lead.
Start with the book seller.
He'll send you to the ferry man.
Then you can go to the pawn shop owner to get a magic map.