The point is that whether this payment was mentioned in any document or paper that you've signed with the Bank A. If yes, you or someone else have to pay it, no choice. Tell your story to Bank B to see if they can do anything for you is a good and smart idea.