Cannot Pair VSatellite Worker Using A Fully Qualified Domain Name

Error:

When attempting to pair a VSatellite Worker with a Vsatellite host during the configuration of a Microsoft Active Directory Certificate Service certificate authority using a fully qualified domain name (FQDN), you receive an unexpected error message with a referenceId.

Symptom:

When adding a Microsoft Certificate Authority as a Certificate Authority in Venafi as a Service, two pieces of infrastructure are required: a VSatellite host and a VSatellite worker. When attempting to pair the worker to the host, after setting the name as an FQDN with the proper port, you receive an unexpected error. During this time you may note you can successfully resolve the names of VSatellite worker from the VSatellite host and vice versa from a terminal or command prompt.

Cause:

The VSatellite host relies on will use DNS to resolve the names of the target system, however, the DNS settings it relies on are found in /etc/resolv.conf on the VSatellite host. These settings are copied by the kubernetes pod coredns. This pod functions independently of the host system's configuration. Assuming the name of the host and port are correct and all services are otherwise functional and online, one of the following scenarios is likely:

• The resolv.conf file has a setting that cannot be resolved by the coredns pod.
• Your DNS settings have changed from the time the pod was originally setup.

Resolution:

The steps you might take will be impacted by your VSatellite host operating. For example, a Ubuntu system programmatically generates a resolv.conf file and modern versions point to a 127* address for DNS resolution, even when a static DNS server was otherwise configured. This file can be made static by installing additional an additional service called resolvconf to manage resolv.conf to essentially make it static. Other systems such as RHEL will generate resolv.conf as well, but allow management of its settings through other means such as modifying /etc/sysconfig/network/config. The following steps will not dive into this level of configuration. Rather, the resolutions will focus on steps that can be taken without modifying the underlying host system.

Use an IP Address Instead

The fastest resolution, requiring no system modifications is simply to use an IP address instead of an FQDN. The downside is IP address change more frequently than names do which could leave you in a predicament in the future if your IP addresses do change, causing you to need to rejoin the MSCA and worker to Venafi as a Service.

The resolv.conf File Has A Setting That Cannot be Resolved by the coredns Pod

Stop the pairing process, uninstall the vsatellite worker and fix the coredns pod.

On systems that auto-manage /etc/resolv.conf and point to a local address like 127.0.0.53, such as modern Ubuntu systems, you can install additional services to make the configuration static. However, another route to solve the problem exists: modify the coredns pod.

Run the following command to view your current coredns configuration and note the "forward" section configuration:

kubectl -n kube-system get cm/coredns -o yaml

Note how it points to ". /etc/resolv.conf"

In this instance the file can be edited to reflect a proper DNS server by using the following command and replacing /etc/resolv.conf with the IP address of a valid DNS server:

1. kubectl edit configmap coredns -n kube-system
2. This will open a VI like edit and the same commands will be used.
3. Using your arrows, go down to the line: forward . /etc/resolve.conf
4. Type A to begin editing that line. The cursor will move to the end of the line.
5. Delete "/etc/resolv.conf" and replace it with the IP address of the DNS server.
6. After replacing the line, hit escape to get out of edit mode.
7. Your modified file should resemble the following (forward . ip.add.goes.here):
</ol start="8">

8. Now exit the editor by typing a colon (:) and then typing wq to write and quit the editor. There is no need to restart anything.
9. Now begin the VSatellite Worker installation again and perform the pairing process using your fully qualified domain name.

Your DNS Settings Have Changed From the Time the Pod was Originally Setup

Sometimes network configurations change and DNS server addresses must be modified. As noted above, the coredns pod will take the information from /etc/resolv.conf when it is first configured. If your resolv.conf file had a particular nameserver entry that has since changed, your host system will work as expected, but coredns and your connection to MSCA may suddenly stop working or you will not be able to pair your VSatellite worker using a fully qualified name anymore. 

If your DNS configuration has changed on the host system, you can follow the steps in the previous section to modify the coredns pod or you can simply delete the pod and it will be recreated automatically with the newly updated settings.

To delete the pod (and to force it to be recreated) follow these steps:

1. Run the following command: kubectl get pods -A and note the full name of the coredns pod. It should be something like coredns-d76bd69b-mf6r8. Note the GUID value after coredns will be different on your system.
2. Now delete the pod with this command, replacing the coredns-guid value with your own, and wait a moment: kubectl delete pod coredns-d76bd69b-mf6r8 -n kube-system
3. After the pod is deleted, re-run kubectl get pods -A to get a new listing of your pods and note the AGE column should reflect a newly created pod for coredns.
4. Now exit the editor by typing a colon (:) and then typing wq to write and quit the editor. There is no need to restart anything.
5. Now begin the VSatellite Worker installation again and perform the pairing process using your fully qualified domain name.

Summary:

After identifying the best path to follow to resolve your DNS resolution problem for the coredns pod, you should be able to successfully pair your VSatellite Worker to the VSatellite host using fully qualified domain names. Assuming all services are on, available, and accessible, you should be able to quickly establish a connection between VaaS and Microsoft AD CS.

If you are still running into problems, please reach out to support@venafi.cloud for further assistance.