I found this a lot harder to do than I would have thought necessary, which leads me to believe that there is some other obvious way of dealing with this issue. Nonetheless, after several hours of stuffing about with the AzureRM Powershell Module, I finally figured out how to remove all of the expired web certificates from AzureRM.
It's not the prettiest thing in the world, but it works.
Don't worry, if you have a certificate that is expired and still tied to a site, it will just throw an error despite the -Force Flag.