Important Considerations Before Upgrading to Trust Protection Platform 19.1.0 – Venafi Customer Support

Applies to:

Upgrades to Venafi Trust Protection Platform 19.1.

Summary:

Venafi Trust Protection Platform 19.1 introduces new functionality and several changes to existing functionality. Click here for a complete list of new features.

IMPORTANT! Before upgrading, read through this article carefully. Depending upon the version from which you're upgrading, several enhancements made over the past two years to Trust Protection Platform require that you take important steps, either before or immediately after upgrading.

Pay special attention to the list of features that have been deprecated, and to the list of features scheduled for deprecation.

For detailed upgrade steps, refer to the Readme.rtf document that is packaged with Venafi Trust Protection Platform 19.1 installation files.

Visit Venafi Product Life Cycle for more information about ship dates and when support for each version ends.

Trust Protection Platform 19.1 requires .NET Framework 4.7.2

Before upgrading to Trust Protection Platform 19.1, make sure the .NET Framework is updated to 4.7.2. You can download the offline .NET 4.7.2 installer for Windows Server 2012 R2 and Windows Server 2016 here.

CRITICAL! You must re-run the sample-grants.sql script after upgrading the database to 19.1 upgrade script

If upgrading from version 18.3 or earlier, the new Work Queue framework introduces new database-stored procedures, which require execute permissions to be given to the Windows service account that Venafi uses to communicate with the database. If the sample-grants.sql script is not run, no work can be performed by the Venafi Platform.

19.1 Venafi User Agent for Windows (AJ) requires .NET Framework 4.6.1 or higher

If you are using the Enterprise Mobility Protect User Agent for Windows (sometimes referred to as AJ), you should ensure that workstations have .NET Framework 4.6.1 or installed before deploying the 19.1 version of the User Agent. This change is required to take advantage of important security enhancements of the more updated .NET framework. This does not prevent you from upgrading servers to 19.1.

Aperture Certificate Request Wizard no longer supports User Provided CSRs when Management Type is set to Provisioning

This applies to customers who have management type locked to Provisioning but also use user provided CSRs. As part of our efforts to improve automatic certificate installation (provisioning) we have made many significant changes that make the process easier. This has necessitated changes to the certificate flow so that users get a different experience when management type is set to Enrollment vs Provisioning. If you have management type locked to Provisioning your end users will no longer be able to to upload User Provided CSRs (which can't be provisioned anyway) in 19.1. If your users need to be able to enroll certificates with user provided CSRs you will want to either remove the policy lock for Provisioning Management Type or change the policy value to Enrollment.

Custom SSH email notifications might need to be modified on upgrade to 19.1 in order to function

Custom SSH email notification rules might need to be updated manually or their macros will not work when Trust Protection Platform is upgraded to 19.1.

If you have copied and modified the SSH Email template channel called Email to Key Owner - SSH you will need to modify the macro of your custom email channel(s).

There are 3 locations on the plaintext version of the message where you need to replace $Event.Value1$ with $Event.Text1$
There are 3 additional locations on the HTML message version where you need to replace $Event.Value1$ with $Event.Text1$

If you are only using the read-only templates for SSH email notifications, then no action is needed.

To update the custom SSH email notifications:

Launch the Windows Administration Console (WinAdmin).
Navigate to the Logging Tree.
Expand the Channels Container.
Locate your custom SSH channel that was copied from Email to Key Owner - SSH.
Update the necessary macro strings in the Plaintext Message and HTML message tabs, replacing $Event.Value1$ with $Event.Text1$ (6 locations total unless there were further modifications).

Auto Layout Manager Service Module is disabled by default for placement jobs

When you upgrade, the Auto Layout Manager service module is installed but not enabled. Device and Certificate Placement jobs won't function unless Auto Layout Manager is enabled on at least one engine in the cluster. You can do this in WebAdmin or in Venafi Configuration Console (VCC):

In WebAdmin:

After installation, launch WebAdmin and navigate to the Platform tree.
Expand the engines on which you want to enable AutoLayout Manager and click Auto Layout Manager.
Uncheck the Disable this Module option and click Save.

In VCC:

Open the Product node, and look for Automatic Layout Manager in the Component list.
In the Actions panel, click Enable.

If you are using an Answer XML file for automated deployments or to upgrade from 18.x to 19.1, you need to update your Answer XML file to include enabling Automatic Layout Manager.

Permissions changed for placement rules for non-master admins

Any permissions for placement rules given to non-master admins prior to upgrading to version 19.1 must be given again following the upgrade.

Permissions for placement rules are now managed only in Aperture (in Configuration > Placement Rules).

Database administrators may need to re-index the database after upgrading to 19.1

During the upgrade from 18.4 to 19.1, there are several upgrade stages in the SQL script that delete attributes that are no longer needed by Venafi Platform. Deletion of rows can cause the fragmentation of database indexes. If you already have a regularly scheduled index defragmentation plan in place, then no action is needed. However, if you do you not regularly defragment Venafi Platform indexes and are noticing a performance decrease after upgrading to 19.1, it is recommended that you work with your DBA to defragment your Venafi Platform database.

Enrollment of certificates with domain components is not allowed by default

Version 19.1 offers full-feature support for certificates with domain components. When you upgrade to 19.1, policy for allowing domain components on certificates is disabled by default. Therefore, if your users are uploading user-provided CSRs with domain component attributes, you must update your policy settings to allow domain components for those folders in Aperture (only).

Last SSL/TLS validation results removed upon upgrade

Version 19.1 introduces new dedicated validation tables in the database for SSL/TLS validation results. Existing validation results are not migrated to the new validation tables. For certificates that have validation enabled, Trust Protection Platform will re-populate SSL/TLS validation results after the next daily validation.

SSL/TLS validation results can no longer be read with POST Config/Read via REST API

If you are using Config WebSDK to read validation results, you will need to update to new WebSDK calls in 19.1 to continue to read validation data. The new REST API endpoint is:

GET Certificates/{guid}/ValidationResults

All Adaptable scripts will be blocked from working until they are re-saved in the web UI

To increase the security of the Venafi Platform, the system now securely stores the last known good hash of all the Adaptable PowerShell scripts. This prevents unauthorized edits of the adaptable scripts by Windows Server Administrators from being automatically executed by the Venafi Platform. Therefore, on upgrade, all adaptable scripts will stop processing.

NOTE If you are upgrading from version 18.4, this change only applies to Adaptable Workflow. This is because all other Adaptable solutions were updated in 18.4.

After upgrading to version 19.1, but before restarting the Venafi Platform Windows services, do the following:

Confirm that the correct (and the exact same) version of the script is on all Venafi servers in the cluster.
Using either WebAdmin or Aperture (depending on the feature), re-open the screen where you specify the Adaptable PowerShell script, and then re-save the page.
Start the Venafi Platform Windows services.

New updates to Adaptable solutions need to be manually replicated on all servers in the cluster and will also require administrator acknowledgement whenever the associated PowerShell script changes.

For more information, see: Protecting against unapproved changes to Adaptable scripts.

Server Agent now proactively manages its memory resources

Server Agent version 19.1 now manages the Windows Service Recovery Options directly for its own Windows service. Any previous customization of these settings are overwritten during the upgrade. This is part of the proactive resource management feature.

Server Agent also introduces a new process thread on Linux, Solaris, HPUX, and AIX. This thread is called vagent_pg. It is in addition to the vagent process and is started as needed.

During the normal operation of the proactive resource management feature of the server agent, you might see occasional logs (vagent_msg_1117) indicating that the Venafi Agent service (or thread) has been restarted. If this occurs on your systems more than once a month, please contact Venafi Customer Support.

Special 19.1 upgrade steps for Server Agent for Windows using Agent Upgrade work

NOTE: If you upgrade your Venafi Server Agents by re-deploying the MSI then you can skip this section.

Microsoft Visual C++ 2013 Redistributable end-of-mainstream support occurs on April 9, 2019. Venafi Server Agent for Windows versions 18.2 and newer already include the Microsoft Visual C++ 2017 runtime, but versions 18.1 and older do not.

To upgrade your Server Agents running on Windows from version 18.1 or older to version 19.1, you must do one of the following:

Upgrade your Venafi Platform Server to version 18.2, 18.3, or 18.4 first. Then upgrade your Venafi Server Agents (on Windows) using Agent Upgrade work in Aperture. Once the agents are upgraded to 18.2, 18.3, or 18.4 and have reported back to the Venafi Platform server as upgraded, you can upgrade your Venafi Platform servers to version 19.1 and proceed to upgrade the Server Agents (on Windows) to 19.1 using Agent Upgrade work.

NOTE The Agent Upgrade Configuration feature will upgrade Server Agents to match the current version of your Trust Protection Platform server. If your Trust Protection Platform server is already at version 19.1, then you must use the following method to upgrade your Windows-based Server Agents.
After upgrading your Venafi Trust Protection Platform server to version 19.1, use the Server Agent for Windows installation file to upgrade the agents to 19.1. The agent installation file can be deployed manually or by using automated tools such as SCM, Puppet, etc.

References:

Server Agent clears internal event log store (Events.sq3) when you upgrade to 19.1

During the upgrade to Server Agent 19.1, the agent's internal events log is cleared. This applies to Server Agent on all operating systems. The logs will begin to accumulate again after a successful upgrade to 19.1. This does not affect events that are written to SysLog or Windows Events.

Potential longer upgrade time if upgrading from 18.3 or earlier

Due to the re-write of the work queue framework, existing queued actions against certificates and keys must be migrated to the new framework. If there is a significant amount of pending actions in the system at the time of upgrade, you might experience longer than normal upgrade times. For example, if you have 20,000 queued actions when you upgrade, the upgrade time will take approximately five minutes longer to make the migration.

If you are upgrading from version 18.2 or earlier, significant changes made in Trust Protection Platform affect how the database stores vaults and references to objects. These changes require a migration of how data is stored in the database. Depending on the size of your database, and the configuration of AlwaysOn, you can expect the SQL migration scripts to take from 10 to 25 minutes longer than usual for larger deployments.

Unix/Linux Server Agent now setting file permissions on installed certificate keystores

Previous versions of the Venafi Platform and Server Agent did not honor permission configuration of certificate keystores when installed by the Server Agent.

Starting with version 18.4, these permission configurations for certificate installation for Unix/Linux agents will be honored. Therefore, you should verify that the permissions that you have in place are correct.

Your users can now request additional SAN types in Aperture

In previous versions of Venafi Platform, Aperture only supported DNS SANs to be requested on a certificate. Beginning with 18.3, in addition to DNS SANs, Aperture now supports IP, URI, UPN, and email SAN types. If you don't want your users to be able to request these types of SANs, or if you are concerned that they might be confused by the new options, visit your Certificate Policy settings in Aperture to disable the use of specific SAN types.

**SSH Self-Service Keys feature replaces External Key feature**

Beginning with version 18.3, the new Self-Service Keys feature has replaced the External Keys feature. Therefore, after upgrading, you can no longer create external keys. Instead, create self-service keys. This is typically done when you're resolving orphaned keys or when you have a user with a device that is not in the Trust Protection Platform inventory. After upgrading, you'll have the option of migrating each external key to a self-service key (from the Keyset Details page in Aperture).

User Portal now shows the current version of certificates

Beginning in version 18.3, your users can now request and download certificates, including previous versions, directly from the portal. In addition, the user experience has been improved through significant visual enhancements.

NOTE Because of these enhancements, some of the product terminology has been updated; so if you have customized the terminology or styles in your implementation of the User Portal, you should revisit and update your customization to match these changes. For more information, visit https://support.venafi.com/hc/en-us/articles/228093008.

Amazon integrations updated to leverage new credential type

Prior to the 18.3 release, password credentials were used to configure different Amazon integrations, such as Certificate Authority, Provisioning Driver, and AWS EC2 Instance Monitoring. During the upgrade from 18.2 or earlier, these integrations are migrated automatically so that they leverage the new Amazon credential type. However, if you have any WebSDK integrations that automate creating or editing these types of objects in Trust Protection Platform, you'll need to update your WebSDK integration. This is because these integrations now require the use of the Amazon Credential type in order to support ADFS SAML authentication into Amazon Web Services.

Permissions updated for running Server Agent on Linux/Unix

The permissions that the Server Agent installs with have been updated. Specifically, user and group permissions have been updated so that they are set to 0 (where in some places they were previously set to 5). Therefore, if you'd had non-root users interacting with the Server Agent previous to upgrading from 18.2 or earlier, then you might need to revisit their sudo permissions after the latest version of the Server Agent is installed.

**System must provide iconv functionality for Server Agents running on Linux**

In previous versions, Server Agent implemented its own iconv functionality. Beginning with Venafi Platform version 18.3, Server Agent requires that your system libc provides the iconv functionality. You should make sure that iconv is available before upgrading either Trust Protection Platform or Server Agent. Current versions of glibc (The GNU C Library) provide the required functionality. The required files should be installed as part of the glibc package.

Workflow approval logs

Beginning with release 18.2, permissions were added and are now required in order to see the approval and rejection logs in WebAdmin. On upgrade, only Master Admins will have permission to see the log. If there are approvers or other users that should see these special log views, you will need to give those users and groups view/read permissions.

Workflow approver update interval has been modified

In versions 18.1 or earlier of Venafi Platform, if the approvers for a workflow changed for an existing workflow ticket, the ticket would be deleted and recreated so that the updated approver could act on the workflow ticket. In order to improve the scalability and stability of the platform when there are many outstanding workflow tickets requiring approval, the interval for updating existing tickets has changed from 1 minute to 4 hours. Therefore, if the approver changes, it will take Venafi Platform approximately 4 hours to respond to updates regarding who the approvers are for existing workflow tickets.

Referrer and origin checking

To enhance the security of the web consoles, both Aperture and WebAdmin have been enhanced to check that the referrer or the origin in the HTTP headers is not null. In addition to not being null, if either has a non-null value, then the fully qualified domain name must match the fully qualified domain name of the Venafi Platform server hosting the web console. Some corporate proxies are configured to remove or modify the referrer and/or origin from the HTTP header. Some users may install browser plugin/extensions that also remove/modify the referrer/origin. After upgrade, users or organizations in these situations will experience issues with the web UI consoles.

If you are getting a 403 Forbidden error in either Aperture or WebAdmin, then your organization or specific user likely has a browser plugin or corporate proxy that is stripping out both referrer and origin information from HTTP headers. For information on how to troubleshoot this issue, visit:

https://docs.venafi.com/Docs/current/TopNav/Content/Aperture/tr-aperture-tpp.php

WebSDK integrations for Disabled and In Error

In order to optimize performance for certain certificate queries, Disabled and In Error were promoted from being attributes within the config_contains tables to permanent columns within the config_objects table. Even though all parts of the product, including WebSDK, were updated to ensure that this was a safe change, it is highly recommended that customers spend more time testing their WebSDK integrations in lower environments before upgrading to production in 18.2.

Master admins permissions changed

Beginning with release 18.2, Master Admins can no longer have their permissions accidentally or intentionally removed at certain locations in the Policy tree. This change occurred because of the number of customers unintentionally making changes to the permissions of Master Admin accounts that resulted in a call to a Venafi Support Engineering in order to reverse the problem. Adding extra permissions to Master Admins causes considerable slowdowns for Aperture, WebAdmin, and Custom Reports created by that user. By completely removing the ability to change Master Admins permissions in subsets of the tree, all Master Admins will see an increased performance benefit. It is important to note that when managing permissions, it is still possible to add Master Admins to the permissions control for a specific object and its children, but those permissions assignments will be ignored.

AWS certificate installation/provisioning driver changes it default provisioning behavior

Beginning with version 18.2, the default behavior of the Amazon Web Services certificate installation and provisioning driver changed in the way that it provisions non-Amazon issued certificates. By default, certificates are now provisioned to the Amazon Certificate Manager (ACM) store instead of the IAM store. However, you can still provision certificates to the IAM store if you modify the Provision To setting.

Generally, Venafi does not change default behavior because it can create issues; however, in order to align Venafi Platform functionality with Amazon's own recommendations, an exception was made:

“ACM is the preferred tool to provision, manage, and deploy your server certificates. With ACM you can request a certificate or deploy an existing ACM or external certificate to AWS resources. Certificates provided by ACM are free and automatically renew. You can use ACM to manage server certificates from the console or programmatically. For more information about using ACM, see the AWS Certificate Manager User Guide.”

“Use IAM as a certificate manager only when you must support HTTPS connections in a region that is not supported by ACM. IAM securely encrypts your private keys and stores the encrypted version in IAM SSL certificate storage. IAM supports deploying server certificates in all regions, but you must obtain your certificate from an external provider for use with AWS. You cannot upload an ACM certificate to IAM. Additionally, you cannot manage your certificates from the IAM Console.”

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_server-certs.html

Supported/compatible browsers changed

In order to bring greater parity between the web browsers supported by Venafi and the web browsers used by Venafi customers, we have updated support for Google’s Chrome browser from compatible to supported.

In addition, we have changed support of Mozilla Firefox from supported to compatible, and beginning with Venafi Platform version 18.2, we had updated the already compatible Firefox ESR version to Firefox ESR 60.

Server Agent: Remote Mount Point Scanning

In previous versions of the Server Agent, NFS and CIFS mount points on Windows and *NIX operating systems were always scanned, even when not configured to do so. Beginning with Trust Protection Platform 18.2, Server Agent properly honors the agent certificate discovery work configuration as to whether to scan NFS and CIFS mount points or not. In addition, specific detection has been added for file systems mounted via NTFS junction points and their scanning is controlled via the same expanded option. If you use Server Agent to do certificate discovery, you should review your work definition with regards to the configuration of the scanned systems.

NOTE The *NIX portion of this fix was introduced in the 18.1 version of the Server Agent. The Windows portion of the fix was introduced in the 18.2 version of the Server Agent.

18.2 User Agent for macOS and Windows requires Venafi Platform 18.2 or higher

While our general rule is that the Venafi Platform must be the same version or newer than the User Agent, we are specifically calling it out this release because of the overhaul done in this specific version. With the introduction of macOS and non-domain joined windows support, new APIs were introduced that require a Venafi Platform 18.2 or higher server for successful communication. Older versions of User Agent will still function with a Venafi Platform 18.2 server.

Deprecation of Windows Server 2008 R2

With the release of Trust Protection Platform 17.4, support for Windows Server 2008 R2 was removed. Trust Protection Platform 18.1 and 18.2 take advantage of new functionality available in Windows Server 2012 and 2016. Therefore, you can no longer install on Windows Server 2008 R2. Please upgrade or replace your Windows systems with Windows 2012 R2 or Windows 2016 servers before attempting to upgrade to Trust Protection Platform 18.1 or 18.2.

Deprecation of Microsoft SQL 2008 R2

Before upgrading to Trust Protection Platform 17.4 or later, verify that the Microsoft SQL Server hosting the Trust Protection Platform database is running Microsoft SQL Server 2012 SP2, 2014, or 2016 with the latest patches applied.

For more information, see Error: "This Upgrade Is Not Allowed Due To An Incompatible Version Of SQL Server"

Venafi Advanced Key Protect required for HSM Remote Key Generation

If you have been provisioning certificates with keys remotely generated in a Gemalto SafeNet HSM and want to continue doing so, you must enable Advanced Key Protect after upgrading to Trust Protection Platform 18.1 or higher.

Upgrade process - Windows Services

Beginning with version 18.1 Venafi Platform includes a new and enhanced install and upgrade process. Following an upgrade, the new installer does not start Venafi Windows Services automatically. Instead, use the new Venafi Configuration Console on the Product node to start and stop services.

Upgrade process - Windows Authentication to SQL Server

In previous versions of Trust Protection Platform, when using Windows Authentication for communicating with the Venafi MSSQL Server, the Windows user who performed the Trust Protection Platform upgrade was also required to have appropriate permissions for database access. However, starting with Trust Protection Platform 18.1, this is no longer true. This is because the newly enhanced installation and upgrade process allows database communication to be proxied through the database credentials specified at installation.

Server Agent: RPM files are now signed

Starting with Server Agent 18.1, Server Agent RPM files are now signed with an RPM V4 signature. In order to properly validate the RPM files before installation or upgrade, import the Venafi RPM signing key before installation/upgrade. Validation of RPM V4 signatures might fail on versions of RHEL that are earlier than 6.0. In addition, if you customize the RPM, the signature will no longer be valid.

Oracle Access Manager/SSO Authentication Refactor

Where Trust Protection Platform is configured to authenticate via custom header attributes, Oracle Access Manager integration has been refactored in 17.4 and renamed Pass-through Authentication. If you are integrating with Oracle Access Manager, Siteminder, or any other Single-Sign-On solution that integrates via custom HTTP headers, you will need to update your configuration before the integration will work on 17.4.

For more information, see: Pass-Through Authentication For SSO Updated In 17.4

DigiCert CA platform support

The DigiCert integration driver was updated in 17.3 to support CertCentral. It no longer supports the legacy Enterprise platform. If you are currently using the Enterprise platform, please coordinate with DigiCert to migrate to the new CertCentral platform before upgrading to 18.2.

DataPower integration support

The DataPower integration driver was updated in 17.3 to support the new REST interface available on newer versions of DataPower. If you are using older versions that support the legacy SSH Command Line Interface (CLI), consider upgrading so that you can take advantage of new functionality in the integration. Legacy versions using SSH CLI are expected to continue to work, but are no longer actively tested.

Venafi License Report

Beginning with Trust Protection Platform 17.3, the Venafi License Report is automatically sent to Venafi. For more information, see Info: License Report Changes FAQ For 17.3

Deprecated functionality:

Click here for the latest KB listing deprecated functionality in the current and past versions of Trust Protection Platform.
https://support.venafi.com/hc/en-us/articles/115001578651

Functionality scheduled for deprecation in future releases:

Click here for the latest KB article on features scheduled for deprecation in future releases of Trust Protection Platform.
https://support.venafi.com/hc/en-us/articles/115001662292