Upgrading to Venafi Trust Protection Platform 19.2.1
Venafi Trust Protection Platform 19.2.1 introduces new functionality and several changes to existing functionality. Click here for a complete list of new features.
IMPORTANT! Before upgrading, read through this article carefully. Depending upon the version from which you're upgrading, several enhancements made over the past two years to Trust Protection Platform require that you take important steps, either before or immediately after upgrading.
CRITICAL! You MUST re-run the sample-grants.sql script after upgrading the database using the 19.2.1 upgrade script. This impacts multiple features and functionality.
For detailed upgrade steps, refer to the Readme.rtf document that is packaged with Venafi Trust Protection Platform 19.2.1 installation files.
Visit Venafi Product Life Cycle for more information about ship dates and when support for each version ends.
Delete any instances (engines) of Trust Protection Platform from the Platform tree in WebAdmin for any decommissioned Trust Protection Platform servers
A new migration framework has been implemented that can perform data migrations while services are running. Before you upgrade, delete any TPP server objects from the Platform tree in WebAdmin that have been decommissioned (but weren’t removed from the Platforms tree).
Keep in mind that for large and under-resourced environments, you might see slower performance from anywhere from a few minutes to a few hours. However, this depends on the size and resources of the environment while migrations continue. To see if your migrations are done, you can execute this script:
select * from config_contains where attribute = 'Pending Migration Task'
If no results are returned, then all migrations are complete (assuming you have turned your Venafi services back on after you upgraded to 19.2).
Deprecation of Microsoft SQL 2008 R2
Before upgrading to Trust Protection Platform 17.4 or later, verify that the Microsoft SQL Server hosting the Trust Protection Platform database is running Microsoft SQL Server 2016 SP1 with the latest patches applied.
If your Venafi database has been upgraded from older versions of Microsoft SQL, also make sure that the database compatibility version is also updated. The minimum level required is 110, which is the level for SQL Server 2012 or higher.
See Also: ALTER DATABASE (Transact-SQL) Compatibility Level
Important Note for DB Version: Starting in 19.4, the minimum SQL version compatible with TPP will be Microsoft SQL 2016 SP1
For more information, see Error: "This Upgrade Is Not Allowed Due To An Incompatible Version Of SQL Server"
POST Log REST API endpoint was updated to only support logging for custom event
In previous versions of TPP before 19.2, any event, including those that are used by Venafi Product code, could be logged using the Rest API. If you have a REST API integration using the POST LOG endpoint, review it to ensure it is only submitting Event IDs that are within the decimal range of 16777216 - 43646975.
Reconfigure custom endpoints for CDP monitoring following upgrades
Because the CRL Distribution Point (CDP) Monitoring feature was completely rewritten for scalability, performance and usability, you’ll need to reconfigure any custom endpoints for CDP Monitoring following the upgrade to 19.2.
Enabling New Authentication Services allows any TPP user access to WebSDK
In order to take advantage of the new Token Authentication to WebSDK, you must enable the new Authentication Service. However, enabling the Authentication Service will enable any user who can authenticate to the Venafi Platform can get a token of any scope, even if they do not have the WebSDK role added to their account.
In future version of the platform, we plan to add entitlement controls over scopes so administrators can control which users can use which grouping of API endpoints.
New SSH policy violation feature requires post-install configuration
A new attribute in 19.2 has been added to device settings for detecting SSH policy access violations across environments. To start seeing policy violations, you must configure your environment either on the SSH Policy level or on each SSH Device.
Improved Authorized Users report data enhancements could mean a larger database
Because a new table has been to the database in 19.2, reports are calculated in the background, so you may not see all data immediately after upgrade. If there are a lot of SSH keys in the inventory, the report could potentially contain hundreds of millions of rows. Therefore, disc space should be considered for the potential increase in the size of the database.
Passing the WebSDK API key via URL query string
The Trust Protection Platform WebSDK has historically permitted the post-authentication API key to be passed to REST methods in two different ways: as a query string parameter (i.e. ?apikey=) or as an HTTP header (i.e. X-Venafi-Api-Key). Since its introduction in 14.1, the HTTP header approach has been recommended over the query string because the latter exposes the cleartext API key value in the IIS logs of the Venafi server. Since this poses a security risk the option will no longer be supported as of 19.1.
Missing issuers and chains following the upgrade
When upgrading to 19.2, you’ll see root certificates automatically added to the Roots tree. You won’t be able to delete roots that are the only valid issuer of a certificate in the roots tree or policy tree.
Certificate Revocation Monitor Service Module is disabled by default for revocation monitoring and CDP endpoint monitoring
When you upgrade, the Certificate Revocation Monitor service module is installed but not enabled. Certificate revocation monitoring and CDP endpoint monitoring will not work unless this service module is enabled on at least one engine. You can do this in Venafi Configuration Console (VCC):
- Open the Product node and look for Certificate Revocation Monitor in the Component list.
- In the Actions panel, click Enable.
If you are using an Answer XML file for automated deployments or to upgrade from 18.x to 19.1 or later, you need to update your Answer XML file to include enabling Certificate Revocation Monitor.
Trust Protection Platform 19.1 and later requires .NET Framework 4.7.2
Before upgrading to Trust Protection Platform 19.1 or later, make sure the .NET Framework is updated to 4.7.2. You can download the offline .NET 4.7.2 installer for Windows Server 2012 R2 and Windows Server 2016 here.
Venafi User Agent for Windows (AJ) version 19.1 and later requires .NET Framework 4.6.1 or higher
If you are using the Enterprise Mobility Protect User Agent for Windows (sometimes referred to as AJ), you should ensure that workstations have .NET Framework 4.6.1 or installed before deploying the 19.1 or later version of the User Agent. This change is required to take advantage of important security enhancements of the more updated .NET framework. This does not prevent you from upgrading servers to 19.1 or later.
Aperture Certificate Request Wizard no longer supports User Provided CSRs when Management Type is set to Provisioning
This applies to customers who have management type locked to Provisioning but also use user provided CSRs. As part of our efforts to improve automatic certificate installation (provisioning) we have made many significant changes that make the process easier. This has necessitated changes to the certificate flow so that users get a different experience when management type is set to Enrollment vs Provisioning. If you have management type locked to Provisioning your end users will no longer be able to upload User Provided CSRs (which can't be provisioned anyway) in 19.1 or later. If your users need to be able to enroll certificates with user provided CSRs you will want to either remove the policy lock for Provisioning Management Type or change the policy value to Enrollment.
Custom SSH email notifications might need to be modified on upgrade to 19.1 or later in order to function
Custom SSH email notification rules might need to be updated manually or their macros will not work when Trust Protection Platform is upgraded to 19.1 or later.
If you have copied and modified the SSH Email template channel called Email to Key Owner - SSH you will need to modify the macro of your custom email channel(s).
There are 3 locations on the plaintext version of the message where you need to replace $Event.Value1$ with $Event.Text1$
There are 3 additional locations on the HTML message version where you need to replace $Event.Value1$ with $Event.Text1$
If you are only using the read-only templates for SSH email notifications, then no action is needed.
To update the custom SSH email notifications:
- Launch the Windows Administration Console (WinAdmin).
- Navigate to the Logging Tree.
- Expand the Channels Container.
- Locate your custom SSH channel that was copied from Email to Key Owner - SSH.
- Update the necessary macro strings in the Plaintext Message and HTML message tabs, replacing $Event.Value1$with $Event.Text1$ (6 locations total unless there were further modifications).
Auto Layout Manager Service Module is disabled by default for placement jobs
When you upgrade, the Auto Layout Manager service module is installed but not enabled. Device and Certificate Placement jobs won't function unless Auto Layout Manager is enabled on at least one engine in the cluster. You can do this in WebAdmin or in Venafi Configuration Console (VCC):
- After installation, launch WebAdmin and navigate to the Platform tree.
- Expand the engines on which you want to enable AutoLayout Manager and click Auto Layout Manager.
- Uncheck the Disable this Module option and click Save.
- Open the Product node and look for Automatic Layout Manager in the Component list.
- In the Actions panel, click Enable.
If you are using an Answer XML file for automated deployments or to upgrade from 18.x to 19.1 or later, you need to update your Answer XML file to include enabling Automatic Layout Manager.
Permissions changed for placement rules for non-master admins
Any permissions for placement rules given to non-master admins prior to upgrading to version 19.1 or later must be given again following the upgrade.
Permissions for placement rules are now managed only in Aperture (in Configuration > Placement Rules).
Database administrators may need to re-index the database after upgrading to 19.1 or later
If you’re upgrading from 18.4 to 19.1 or later, there are several upgrade stages in the SQL script that delete attributes that are no longer needed by Venafi Platform. Deletion of rows can cause the fragmentation of database indexes. If you already have a regularly scheduled index defragmentation plan in place, then no action is needed. However, if you do you not regularly defragment Venafi Platform indexes and are noticing a performance decrease after upgrading to 19.1 or later, it is recommended that you work with your DBA to defragment your Venafi Platform database.
Enrollment of certificates with domain components is not allowed by default
Version 19.1 or later offers full-feature support for certificates with domain components. When you upgrade to 19.1 or later, policy for allowing domain components on certificates is disabled by default. Therefore, if your users are uploading user-provided CSRs with domain component attributes, you must update your policy settings to allow domain components for those folders in Aperture (only).
Last SSL/TLS validation results removed upon upgrade
Version 19.1 or later introduces new dedicated validation tables in the database for SSL/TLS validation results. Existing validation results are not migrated to the new validation tables. For certificates that have validation enabled, Trust Protection Platform will re-populate SSL/TLS validation results after the next daily validation.
SSL/TLS validation results can no longer be read with POST Config/Read via REST API
If you are using Config WebSDK to read validation results, you will need to update to new WebSDK calls in 19.1 2 to continue to read validation data. The new REST API endpoint is:
All Adaptable scripts will be blocked from working until they are re-saved in the web UI
To increase the security of the Venafi Platform, the system now securely stores the last known good hash of all the Adaptable PowerShell scripts. This prevents unauthorized edits of the adaptable scripts by Windows Server Administrators from being automatically executed by the Venafi Platform. Therefore, on upgrade, all adaptable scripts will stop processing.
NOTE If you are upgrading from version 18.4, this change only applies to Adaptable Workflow. This is because all other Adaptable solutions were updated in 18.4.
After upgrading to version 19.2, but before restarting the Venafi Platform Windows services, do the following:
- Confirm that the correct (and the exact same) version of the script is on all Venafi servers in the cluster.
- Using either WebAdmin or Aperture (depending on the feature), re-open the screen where you specify the Adaptable PowerShell script, and then re-save the page.
- Start the Venafi Platform Windows services.
New updates to Adaptable solutions need to be manually replicated on all servers in the cluster and will also require administrator acknowledgement whenever the associated PowerShell script changes.
For more information, see: Protecting against unapproved changes to Adaptable scripts.
Special 19.2 upgrade steps for Server Agent for Windows using Agent Upgrade work
NOTE These instructions also apply if you are upgrading to server agent 19.1 or later directly from a version of server agent 18.1 or older
NOTE If you upgrade your Venafi Server Agents by re-deploying the MSI then you can skip this section.
Microsoft Visual C++ 2013 Redistributable end-of-mainstream support occured on April 9, 2019. Venafi Server Agent for Windows versions 18.3 and newer already include the Microsoft Visual C++ 2017 runtime, but versions 18.2 and older do not.
To upgrade your Server Agents running on Windows from version 18.1 or older to version 18.2 or later, you must do one of the following:
- Upgrade your Venafi Platform Server to version 18.2, 18.3, or 18.4 first. Then upgrade your Venafi Server Agents (on Windows) using Agent Upgrade work in Aperture. Once the agents are upgraded to 18.2, 18.3, or 18.4 and have reported back to the Venafi Platform server as upgraded, you can upgrade your Venafi Platform servers to version 19.1 or later and proceed to upgrade the Server Agents (on Windows) to 19.1 or later (with the Server Agent version being less or equal to the Trust Protection Platform server version) using Agent Upgrade work.
NOTE The Agent Upgrade Configuration feature upgrades Server Agents to match the current version of your Trust Protection Platform server. If your Trust Protection Platform server is already at version 19.1 or later, then you must use the following method to upgrade your Windows-based Server Agents.
- After upgrading your Venafi Trust Protection Platform server to version 19.1 or later, use the Server Agent for Windows installation file to upgrade the agents to 19.1 or later. The agent installation file can be deployed manually or by using automated tools such as SCM, Puppet, etc.
Server Agent clears internal event log store (Events.sq3) when you upgrade to 19.1 or later
During the upgrade to Server Agent 19.1 or later, the agent's internal events log is cleared. This applies to Server Agent on all operating systems. The logs will begin to accumulate again after a successful upgrade to 19.1 or later. This does not affect events that are written to SysLog or Windows Events.
Potential longer upgrade time if upgrading from 18.3 or earlier
Due to the re-write of the work queue framework, existing queued actions against certificates and keys must be migrated to the new framework. If there is a significant amount of pending actions in the system at the time of upgrade, you might experience longer than normal upgrade times. For example, if you have 20,000 queued actions when you upgrade, the upgrade time will take approximately five minutes longer to make the migration.
If you are upgrading from version 18.2 or earlier, significant changes made in Trust Protection Platform affect how the database stores vaults and references to objects. These changes require a migration of how data is stored in the database. Depending on the size of your database, and the configuration of AlwaysOn, you can expect the SQL migration scripts to take from 10 to 25 minutes longer than usual for larger deployments.
Unix/Linux Server Agent now setting file permissions on installed certificate keystores
Previous versions of the Venafi Platform and Server Agent did not honor permission configuration of certificate keystores when installed by the Server Agent.
Starting with version 18.4, these permission configurations for certificate installation for Unix/Linux agents will be honored. Therefore, you should verify that the permissions that you have in place are correct.
Your users can now request additional SAN types in Aperture
In previous versions of Venafi Platform, Aperture only supported DNS SANs to be requested on a certificate. Beginning with 18.3, in addition to DNS SANs, Aperture now supports IP, URI, UPN, and email SAN types. If you don't want your users to be able to request these types of SANs, or if you are concerned that they might be confused by the new options, visit your Certificate Policy settings in Aperture to disable the use of specific SAN types.
SSH Self-Service Keys feature replaces External Key feature
Beginning with version 18.3, the new Self-Service Keys feature has replaced the External Keys feature. Therefore, after upgrading, you can no longer create external keys. Instead, create self-service keys. This is typically done when you're resolving orphaned keys or when you have a user with a device that is not in the Trust Protection Platform inventory. After upgrading, you'll have the option of migrating each external key to a self-service key (from the Keyset Details page in Aperture).
User Portal now shows the current version of certificates
Beginning in version 18.3, your users can now request and download certificates, including previous versions, directly from the portal. In addition, the user experience has been improved through significant visual enhancements.
NOTE Because of these enhancements, some of the product terminology has been updated; so if you have customized the terminology or styles in your implementation of the User Portal, you should revisit and update your customization to match these changes. For more information, visit https://support.venafi.com/hc/en-us/articles/228093008.
Amazon integrations updated to leverage new credential type
Prior to the 18.3 release, password credentials were used to configure different Amazon integrations, such as Certificate Authority, Provisioning Driver, and AWS EC2 Instance Monitoring. During the upgrade from 18.2 or earlier, these integrations are migrated automatically so that they leverage the new Amazon credential type. However, if you have any WebSDK integrations that automate creating or editing these types of objects in Trust Protection Platform, you'll need to update your WebSDK integration. This is because these integrations now require the use of the Amazon Credential type in order to support ADFS SAML authentication into Amazon Web Services.
Permissions updated for running Server Agent on Linux/Unix
The permissions that the Server Agent installs with have been updated. Specifically, user and group permissions have been updated so that they are set to 0 (where in some places they were previously set to 5). Therefore, if you'd had non-root users interacting with the Server Agent previous to upgrading from 18.2 or earlier, then you might need to revisit their sudo permissions after the latest version of the Server Agent is installed.
System must provide iconv functionality for Server Agents running on Linux
In previous versions, Server Agent implemented its own iconv functionality. Beginning with Venafi Platform version 18.3, Server Agent requires that your system libc provides the iconv functionality. You should make sure that iconv is available before upgrading either Trust Protection Platform or Server Agent. Current versions of glibc (The GNU C Library) provide the required functionality. The required files should be installed as part of the glibc package.
Workflow approval logs
Beginning with release 18.2, permissions were added and are now required in order to see the approval and rejection logs in WebAdmin. On upgrade, only Master Admins will have permission to see the log. If there are approvers or other users that should see these special log views, you will need to give those users and groups view/read permissions.
Workflow approver update interval has been modified
In versions 18.1 or earlier of the Venafi Platform, if the approvers for a workflow changed for an existing workflow ticket, the ticket would be deleted and recreated so that the updated approver could act on the workflow ticket. In order to improve the scalability and stability of the platform when there are many outstanding workflow tickets requiring approval, the interval for updating existing tickets has changed from 1 minute to 4 hours. Therefore, if the approver changes, it will take Venafi Platform approximately 4 hours to respond to updates regarding who the approvers are for existing workflow tickets.
Referrer and origin checking
To enhance the security of the web consoles, both Aperture and WebAdmin have been enhanced to check that the referrer or the origin in the HTTP headers is not null. In addition to not being null, if either has a non-null value, then the fully qualified domain name must match the fully qualified domain name of the Venafi Platform server hosting the web console. Some corporate proxies are configured to remove or modify the referrer and/or origin from the HTTP header. Some users may install browser plugin/extensions that also remove/modify the referrer/origin. After upgrade, users or organizations in these situations will experience issues with the web UI consoles.
If you are getting a 403 Forbidden error in either Aperture or WebAdmin, then your organization or specific user likely has a browser plugin or corporate proxy that is stripping out both referrer and origin information from HTTP headers. For information on how to troubleshoot this issue, visit:
WebSDK integrations for Disabled and In Error
In order to optimize performance for certain certificate queries, Disabled and In Error were promoted from being attributes within the config_contains tables to permanent columns within the config_objects table. Even though all parts of the product, including WebSDK, were updated to ensure that this was a safe change, it is highly recommended that customers spend more time testing their WebSDK integrations in lower environments before upgrading to production in 18.2.
Master admins permissions changed
Beginning with version 18.2, Master Admins can no longer have their permissions accidentally or intentionally removed at certain locations in the Policy tree. This change occurred because of the number of customers unintentionally making changes to the permissions of Master Admin accounts that resulted in a call to a Venafi Support Engineering in order to reverse the problem. Adding extra permissions to Master Admins causes considerable slowdowns for Aperture, WebAdmin, and Custom Reports created by that user. By completely removing the ability to change Master Admins permissions in subsets of the tree, all Master Admins will see an increased performance benefit. It is important to note that when managing permissions, it is still possible to add Master Admins to the permissions control for a specific object and its children, but those permissions assignments will be ignored.
AWS certificate installation/provisioning driver changes it default provisioning behavior
Beginning with version 18.2, the default behavior of the Amazon Web Services certificate installation and provisioning driver changed in the way that it provisions non-Amazon issued certificates. By default, certificates are now provisioned to the Amazon Certificate Manager (ACM) store instead of the IAM store. However, you can still provision certificates to the IAM store if you modify the Provision To setting.
Generally, Venafi does not change default behavior because it can create issues; however, in order to align Venafi Platform functionality with Amazon's own recommendations, an exception was made:
“ACM is the preferred tool to provision, manage, and deploy your server certificates. With ACM you can request a certificate or deploy an existing ACM or external certificate to AWS resources. Certificates provided by ACM are free and automatically renew. You can use ACM to manage server certificates from the console or programmatically. For more information about using ACM, see the AWS Certificate Manager User Guide.”
“Use IAM as a certificate manager only when you must support HTTPS connections in a region that is not supported by ACM. IAM securely encrypts your private keys and stores the encrypted version in IAM SSL certificate storage. IAM supports deploying server certificates in all regions, but you must obtain your certificate from an external provider for use with AWS. You cannot upload an ACM certificate to IAM. Additionally, you cannot manage your certificates from the IAM Console.”
Supported/compatible browsers changed
In order to bring greater parity between the web browsers supported by Venafi and the web browsers used by Venafi customers, we have updated support for Google’s Chrome browser from compatible to supported.
In addition, we have changed support of Mozilla Firefox from supported to compatible, and beginning with Venafi Platform version 18.2, we had updated the already compatible Firefox ESR version to Firefox ESR 60 and ESR 68.
Server Agent: Remote Mount Point Scanning
In previous versions of the Server Agent, NFS and CIFS mount points on Windows and *NIX operating systems were always scanned, even when not configured to do so. Beginning with Trust Protection Platform 18.2, Server Agent properly honors the agent certificate discovery work configuration as to whether to scan NFS and CIFS mount points or not. In addition, specific detection has been added for file systems mounted via NTFS junction points and their scanning is controlled via the same expanded option. If you use Server Agent to do certificate discovery, you should review your work definition with regards to the configuration of the scanned systems.
NOTE The *NIX portion of this fix was introduced in the 18.1 version of the Server Agent. The Windows portion of the fix was introduced in the 18.2 version of the Server Agent.
18.2 User Agent for macOS and Windows requires Venafi Platform 18.2 or later
While our general rule is that the Venafi Platform must be the same version or newer than the User Agent, we are specifically calling it out this release because of the overhaul done in this specific version. With the introduction of macOS and non-domain joined windows support, new APIs were introduced that require a Venafi Platform 18.2 or higher server for successful communication. Older versions of User Agent will still function with a Venafi Platform 18.2 server.
Deprecation of Windows Server 2008 R2
With the release of Trust Protection Platform 17.4, support for Windows Server 2008 R2 was removed. Trust Protection Platform 18.1 and 18.2 take advantage of new functionality available in Windows Server 2012 and 2016. Therefore, you can no longer install Trust Protection Platform on Windows Server 2008 R2. You must upgrade or replace your Windows systems with Windows 2012 R2 or Windows 2016 servers before attempting to upgrade to Trust Protection Platform 18.1 or 18.2.
Venafi Advanced Key Protect required for HSM Remote Key Generation
If you have been provisioning certificates with keys remotely generated in a Gemalto SafeNet HSM and want to continue doing so, you must enable Advanced Key Protect after upgrading to Trust Protection Platform 18.1 or later.
Upgrade process - Windows Services
Beginning with version 18.1 Venafi Platform includes a new and enhanced install and upgrade process. Following an upgrade, the new installer does not start Venafi Windows Services automatically. Instead, use the new Venafi Configuration Console on the Product node to start and stop services.
Upgrade process - Windows Authentication to SQL Server
In previous versions of Trust Protection Platform, when using Windows Authentication for communicating with the Venafi MSSQL Server, the Windows user who performed the Trust Protection Platform upgrade was also required to have appropriate permissions for database access. However, starting with Trust Protection Platform 18.1, this is no longer true. This is because the newly enhanced installation and upgrade process allows database communication to be proxied through the database credentials specified at installation.
Server Agent: RPM files are now signed
Starting with Server Agent 18.1, Server Agent RPM files are now signed with an RPM V4 signature. In order to properly validate the RPM files before installation or upgrade, import the Venafi RPM signing key before installation/upgrade. Validation of RPM V4 signatures might fail on versions of RHEL that are earlier than 6.0. In addition, if you customize the RPM, the signature will no longer be valid.
Oracle Access Manager/SSO Authentication Refactor
Where Trust Protection Platform is configured to authenticate via custom header attributes, Oracle Access Manager integration was refactored in 17.4 and renamed Pass-through Authentication. If you are integrating with Oracle Access Manager, Siteminder, or any other Single-Sign-On solution that integrates via custom HTTP headers, you will need to update your configuration before the integration will work on 17.4.
For more information, see Pass-Through Authentication For SSO Updated In 17.4.
DigiCert CA platform support
The DigiCert integration driver was updated in 17.3 to support CertCentral. It no longer supports the legacy Enterprise platform. If you are currently using the Enterprise platform, please coordinate with DigiCert to migrate to the new CertCentral platform before upgrading to 18.2 or later.
DataPower integration support
The DataPower integration driver was updated in 17.3 to support the new REST interface available on newer versions of DataPower. If you are using older versions that support the legacy SSH Command Line Interface (CLI), consider upgrading so that you can take advantage of new functionality in the integration. Legacy versions using SSH CLI are expected to continue to work but are no longer actively tested.
Click here for the latest KB listing deprecated functionality in the current and past versions of Trust Protection Platform:
Functionality scheduled for deprecation in future releases:
Click here for the latest KB article on features scheduled for deprecation in future releases of Trust Protection Platform: