Pirates will hold to a protection agreement for six months without formally cancelling, however you may get ships continuing to raid independent colonies you invade and take over, which will auto-cancel the protection agreement as soon as they land on the planet. I don't know if that's working as intended, or is a bug related to orders not being updated when a colony changes. I sort of suspect the latter, since I've seen pirate constructors continue to attempt to retrofit at a planet of theirs that I'd taken over.
I think when you agree to the protection payments, the first payment is immediately made. What might make this less abusable is if not only did the amount of the payments continue to increase as the pirates hate you, but if you are prone to cancelling your agreement, they start demanding large up-front payments. Also, if when you cancel an agreement with them, they instantly re-evaluate ship orders, so if they had just left your system, they may decide to turn around and attack it again.