SCEP requests to Venafi are failing. The Default SQL Channel logs show following:
Network Device Enrollment - Failed to parse CSR
Failed to extract the subject from the received CSR
Network Device Enrollment - Failed to unwrap inner PKCS7
Failed to properly decode the inner PKCS#7 envelope in the received SCEP data
1. Typically this is caused by the payload sent to Venafi SCEP being encrypted to the wrong certificate as configured in the RA Certificate settings in Venafi.
2. This may also be caused by Key Usage being set to "Digital Signature"
Item 1 Resolution:
Ensure that the certificate settings on the system sending the SCEP request matches those configured in the RA settings within Venafi (see graphic below).
Item 2 Resolution:
Modify the Key Usage to: "Digital Signature, Key Encipherment" and try again. Make sure to reset the VEDSCEP App pool in IIS after making any changes.
Example on how to configure Venafi SCEP:
How to determine what cert is being used in the request:
(NOTE: This will only work if the payload plus IIS log data is less than 4096 bytes, which is the maximum length for a single IIS log entry. Otherwise, the payload will be truncated.)
Extract the payload from IIS logs
Use following Powershell script to decode the URL formatted payload
[Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null
$encode = '****place encoded payload data here***'
$b64 = [System.Web.HttpUtility]::UrlDecode($Encode)
$filename = 'C:\temp\filetolookat.p7b'
$bytes = [Convert]::FromBase64String($b64)
3. View the file with certutil command:
certutil.exe -dump $filename
Find the "Recipient Info" section:
Serial Number: 187fdfgg2
Issuer: CN=Venafi Root CA
Compare the Serial Number of the certificate used to encrypt the payload to what is configured in Venafi SCEP RA settings: