No. Once you make a purchase for an Audiobook, it will go into your My Library. Audiobooks in your My Library will always be available for you to download - even several years down the road. We do have a limit on how many times you can download the same Audiobook file (12 times) however just email Customer Service (firstname.lastname@example.org) if you have reached your download limit.
If you cancel your account, you will still have access to your My Library as well.