Error: Downloading a certificate gives the error "The public key of the certificate does not match the value specified"

Symptom

When trying to download the certificate with private key in PKCS#12 format the error "The public key of the certificate does not match the value specified" is shown.

Cause

This error is thrown because the PFX cannot be created if the public key of the certificate and the private key do not match. This can occur if the wrong private key is uploaded or if the certificate renewal is incomplete (meaning that the new private key was generated but the certificate is still the old copy).

Resolution

Complete the certificate renewal or find the original private key for the certificate and upload it within the settings tab.