How to Install a Wildcard SSL on Windows Server

• Who: Anyone interested in securing your Azure cloud server with SSL (Wildcard or single domain)
• What: How to Install a Wildcard SSL on Windows Server in Azure
• With: FileMaker 16+ (these instructions will likely work with past versions as well)
• Why: Security is a priority with any solution, but especially  if you connect to your server over a WAN (as opposed to your local network).

This article is intended to be a deep dive, step by step, “how to” article walking through the process of setting up a Wildcard SSL certificate in FileMaker server and Windows server in the Azure cloud (if your Windows Server is not in the Azure cloud, most of this article will still apply and be useful).  The certificate will be installed on both FileMaker Server and Windows Server in order to enable encrypted connections directly to FileMaker Server and also to web pages hosted on Windows Server!  I will focus on using a Wildcard SSL certificate, but the same process can be used for installing a single domain certificate.  As mentioned, if your server is in-house this article still applies to you, but you can skip the Azure specific stuff.  Let’s begin!

 

Intro Concepts and Vocabulary

For those who are new to SSL or need a refresher, let’s start by defining some of the basic concepts, terms, and reasons behind SSL (if you just want to get to the steps, scroll down a little further).

SSL certificates are currently the accepted method for encrypting web traffic between a server and an end point.  The end point could be a web page, mobile app, FileMaker client, etc.  The main functions of an SSL certificate are to 1) encrypt the data traveling over the web and 2) verify to end users that the web domain they are connecting to belongs to the owner’s of that domain.   In other words, domain owners must go through a process to prove they own that domain before they are issued a certificate from a Certificate Authority.  This second reason is the key difference between FileMaker’s built in certificate and one issued through a Certificate Authority.  The FileMaker Server documentation states:

FileMaker Server provides a standard SSL certificate signed by FileMaker, Inc. that does not verify the server name. The FileMaker default SSL certificate is intended only for test purposes. A custom SSL certificate is required for production use.

It might be worth it to note that the pre-installed FileMaker certificate and a custom certificate both encrypt the web connection between the end points, but the FileMaker issued certificate never validated the domain owner.  Not validating creates the possibility that a connection to the domain could be spoofed or in some way hacked by a third party!

Fortunately, these days there are some inexpensive options when purchasing certificates, so in a professional/production environment there is no reason not to get one.  In my case, I wanted to encrypt parts of our company’s web site, our web apps, and also FileMaker Server!  Each of these entities are related, but distinct.  The web site is hosted remotely at a hosting company with the domain www.mycompany.com, but the apps are hosted on the same Azure server where we installed FileMaker Server.  In order to validate all three of these separate entities, I decided to get a Wildcard SSL certificate which will validate an unlimited number of subdomains.  In other words, we can configure it so that not only is www.mycompany.com encrypted and validated, but so are apps.mycompany.com and also fm.mycompany.com (later we will see how to apply this fm subdomain to the public IP associated with the Azure server.  This allows us to point the url fm.mycompany.com directly to the FileMaker server in the cloud!).

 

Requirements

Before we get into the details, I am going to assume you have already installed FileMaker Server on a Windows Server and can connect a FileMaker client successfully, and that you already have a registered domain name.  With that, let’s begin by acquiring the certificate!

 

Choosing the Certificate

Wildcard certificates are more expensive than single domain certs, so after some research I found the cheapest wildcard SSLs at www.cheapsslsecurity.com which is a reseller of the big name certificate companies.  The one I found is called the Comodo PositiveSSL Wildcard (there are no affiliate links in this article).

Quick side note about FileMaker “tested” certificates: there has been a semantic change regarding FileMaker’s stance on certificates from “supported” certificates to “tested” certificates (heard this on FileMaker Talk podcast: Episode 136) which was intentional because before, developers felt like they could only use “supported” certificates.  That said, Comodo has a couple of “tested” certificates, but the Comodo PositiveSSL Wildcard is not one of them.  I’m happy to be report, however, that it works great and is easy to install with these instructions!

In order to get the certificate, we need to first begin the process by generating something called a certificate signing request (or CSR).  CSR’s can be generated from a number of different places including your Windows Server, the SSL certificate sales website, or FileMaker Server, but since FileMaker Server can be a bit finicky when it comes to implementing SSL, it is much easier to start from FileMaker Server.  Trust me on this one!

What if?
-What if you already started the CSR from Windows? No problem, you can still just start over in FM Server and ignore the previous CSR from Windows.
-What if you already have a single domain SSL like: www.mydomain.com? No problem, you can either get a single subdomain SSL like: fm.mydomain.com or you can use a Wildcard SSL for multiple subdomain use while leaving the original single domain SSL.

 

Generating the CSR

• • Open the FileMaker Server Admin Console
  • Click on the Database Server menu item, then the Security tab.
  • Under the “SSL Connections” section, check the box “Use SSL for database connections”.  Press the Save button.
    

 

• • Restart the FileMaker Server service.
    
    

 

• Log back into FileMaker Server and go back to the Database Server menu item, then the Security tab.
• Click the “Create Request” button.  If you have an existing certificate, you will need to press the “Start Over” button, otherwise you will be taken to the signing request window where you will fill out fields for the CSR.  If you are using a WildCard SSL, be sure to use the asterisk like this: *.mydomain.com If you are using a single domain SSL, you would use something like: fm.mydomain.com Be sure to only use alphanumeric characters in your password or you will get an error!
  
• Press the “Create” button. This will create the following files in /FileMaker Server/CStore/ :
  • serverRequest.pem: CSR required for the SSL purchase process.
  • serverKey.pem: private key file required for the certificate import process.

   

• Press the Download button to save a copy of serverRequest.pem which we will use to purchase the SSL from Cheapsslsecurity.com.  If you had a problem with your CSR button, you could also press the “Start Over” button.  Starting over doesn’t mess anything up.
  

 

Purchase the WildCard SSL

• • Go to the website: https://cheapsslsecurity.com/sslproducts/wildcardssl.html
  • Click on the “Add to Cart” button for the Comodo PositiveSSL Wildcard.
  • Once you have purchased your SSL, you will be taken to the Order Process page.  Step one is to choose your SSL authentication option.  Remember in the beginning I mentioned that an SSL not only encrypts the web connection, but it verifies to end users that the web domain they are connecting to belongs to the owner’s of that domain?  This is the step that establishes that you actually own the domain you will use the SSL on.
  • I recommend using email authentication.  It is quick and easy.
    • If you are setting it all up for someone else like a client and you don’t have direct access to the pre-authorized email addresses associated with the WHOIS for the domain, you can simply have someone who does have access forward you the emails to continue the process.
      

 

• For Step 2, you will need to open the serverRequest.pem file you downloaded from FileMaker Server in a text editor and copy/paste it into the text area for step 2.  The first line and last line that say “—–BEGIN CERTIFICATE REQUEST—–” and “—–END CERTIFICATE REQUEST—–” are necessary parts of the CSR. Do not remove them.  It should look something like this:
  
• On Step 3, be sure to choose “Other” as the type of server platform you are using.  This is important as it sets it up the way FileMaker expects it with the private key!
• If you messed up any of the steps above, you can always come back to the Order Process page and redo this section.  In fact, you can even come back months later if something goes wrong with your certificate and you need to re-generate the certificate.
• If you use a different CSR method or a different certificate vendor, this info might be helpful:
  • If you are asked to provide the cert domain name again, be sure to use the asterisk for wildcards: *.mydomain.com
  • If you are asked to choose a RSA key size, choose 2048.
  • If you are asked to select a signature hash, use SHA-2  ( or SHA-2 with SHA-1 root).

 

Prepping the Certificates

• After finishing the order, you will receive an email from Comodo (the Certificate Authority, not the reseller) which will contain a validation code and a link to the Comodo site where you will need to apply the code.
• After validating the code, you will receive another email from Comodo with a zip file containing your new certificate and also some intermediate certificates.
  

  Intermediate certificate chaining is the process of combining multiple intermediate certificates and is one of the confusing parts about using less expensive SSL certificates.  More expensive SSL certs have a higher degree of authentication steps so they might require fewer intermediate certs or none at all.  I’ve read that it’s likely that this is done by design to get you to spend more on the more expensive extended validation certs (that don’t need multiple intermediary certs).  Fortunately, we have excellent instructions on how to do the intermediate certificate chaining for the Comodo PositiveSSL Wildcard!

• Download the zip file and unpack the files into a directory easy to access from a windows command line.
• If you bought the Comodo PositiveSSL Wildcard, you should get 4 files within the zipped download: your main certificate, and 3 intermediate certificates.  Your main certificate should be something like: STAR_mydomain_com.crt
  • The “STAR” stands for the asterisk, but if you bought a single domain cert it would be something like: FM_mydomain_com.crt

 

Creating the Intermediate Certificate Chain

In order for our less expensive certificate to validate with a Certificate Authority, we need to use the extra intermediate certificates to lead our end points up a chain to the final Certificate Authority.  Since FileMaker server only “allows” for one intermediate certificate to be uploaded with the main certificate, we need to bundle these three together in the proper chained order.  Hat tip to the Devside.net author who provides the specific steps to accomplish the certificate chain for the Comodo PositiveSSL cert.

• Open the command line with elevated privileges and navigate to the folder where you unpacked the certs.  For example:
  

  c:
  cd certs\
  (the above would put you in the folder->  c:\certs)

• Use the “copy” command to concatenate the 3 intermediate certificates together:
  

  copy /B COMODORSADomainValidationSecureServerCA.crt + COMODORSAAddTrustCA.crt + AddTrustExternalCARoot.crt PositiveSSL.ca-bundle

• The order of the certificates in the “copy” command above is important.
• You will now have a chained intermediate certificate with the name: PositiveSSL.ca-bundle

 

Importing the Certificate to FileMaker Server

At this point, you have prepped your FileMaker Server to expect a certificate, purchased the certificate, and bundled the intermediate certificates into a chain.  Now we can go back to FileMaker server with your certificate and bundled intermediate chain files.  If the server is in the cloud, be sure to copy these files from your local computer to the server.

• Open the FileMaker Server Admin Console and Click on the Database Server menu item, then the Security tab.
• Click on the Import Certificate button to open this dialog window:
  
• Upload the following files:
  • Signed Certificate: is the main certificate you just received: STAR_mydomain_com.crt
  • Private Key: is the serverKey.pem located in /FileMaker Server/CStore/
  • Intermediate Certificate: is the new bundled file: PositiveSSL.ca-bundle
  • Private Key Password: is the password you used during the CSR creation.
  • Click Save
• Restart FileMaker Server.

 

Bind Your Certificate to Your Default Windows Web Site

One of the magic parts about starting the CSR from FileMaker Server is that upon importing the cert, it also automatically imports to Windows Server! If you didn’t already have other web sites or services on the server and it is simply dedicated to FileMaker server, you’re done configuring the certificate on Windows!

What if?
-What if you had to change FileMaker Server’s web ports because you already had port 80 and 443 used for something else (either on the server itself or on the router in front of the server)? No problem, all you need to do is bind the new certificate to the ports you set up for FileMaker Server.

• Open IIS on Windows Server.
• In the “Connections” pane on the left, navigate to YOURSERVER->Sites->FMWebSite.
• In the “Actions” page on the right, click “Bindings”.
• For the ports you used on the FM Server setup, click on the port number and press the “Edit” button.
• In the Host Name field, put your cert’s domain name: *.mydomain.com or fm.mydomain.com
• In the SSL Certificate dropdown, choose your new certificate and press OK.
• You have now bound the certificate to that port, so in order to reach your FM server, you will need to add your port to the end of your IP or domain name.  For example, if you used the port 9443 as your SSL port when setting up FM Server and you want to connect to your server via a static IP, you might use: 173.10.10.15:9443, or via a subdomain like this: fm.mydomain.com:9443.

 

Update DNS to Point Your Subdomain Name to Your Server IP

Ultimately, we want to connect to our FileMaker server by using the subdomain: fm.mydomain.com instead of using an IP address.  We can do this by creating an A record within our DNS records.

At this point technically your FileMaker Server has a newly installed custom certificate and will encrypt the connection between the server and an end point if you connect via IP address, but since the certificate validation will be looking for the certificate name as the connecting address, you will still get an error icon using the certificate. In order to get the “green padlock” icon signifying a properly configured and validated certificate, we need to update our DNS record with a with our subdomain.

With both Azure and AWS, it is simple and free to use a static IP address (if you have an in-house server, hopefully you also have a static IP address.  If not, do some research on using dynapic dns updaters like no-ip.com).

In order to point our new subdomain cert fm.mydomain.com to our FM server’s IP address, we go to our DNS record manager and create an A record.  There are two likely DNS record managers.  If you simply purchased the domain name but are not hosting a web site, then you can go to your domain registrar to update DNS records (GoDaddy for example).  If you’ve already set up a web site with a web host (like BlueHost or HostGator) and are adding a subdomain, you can go to your web site’s cpanel to update DNS records and add a subdomain.

• GoDaddy Example:
  • Log in to GoDaddy
  • Click on your domain name’s DNS button:
    
  • Click the Add button at the bottom.
    • Select “A”
    • Host: use your subdomain prefix.  Let’s simply use: fm (this is the prefix for: fm.mydomain.com)
    • Points to: use your static IP.
    • TTL: leave it at 1 Hour
    • Press Save
• Cpanel Example:
  • Log in to your web host’s cpanel.
  • In your Cpanel, look for the DNS section and click on the “Simple Zone Editor”.
  • Name field: type fm.mydomain.com
  • Address field: type your static IP
  • Press the “Add an A Record” or Save button (whichever you have).

 

Connecting to Your FileMaker Server

Now you can connect to your FileMaker server from FileMaker using the hostname: fm.mydomain.com!

• Open FileMaker Pro or Advanced
• In the Launch Center, click on the Hosts tab.
• At the bottom left, click on the + symbol to open the “Add Favorite Host” window.
  • In the Host’s Internet Address, use: fm.mydomain.com

That’s it!  Now when you or a client connects to your FileMaker Server via the WAN, you will get that elusive “green padlock” and your connection will be encrypted!  If you want to connect to WebDirect, you would simply use: https://fm.mydomain.com/fmi/webd.
Note that since we are connecting via SSL encryption, you must specify the “https” in the url. 

If you want to do some custom web publishing, you can create your custom pages in the HTTPServer\conf folder on your FileMaker server.  The full path on my server is: C:\Program Files\FileMaker\FileMaker Server\HTTPServer\conf.  The “conf” directory is set up by FileMaker server as the default public www directory on your Windows Server, so all web pages/sites should go here.  For example, if you have several custom sites like: intranet.mydomain.com, client1.mydomain.com and, client2.mydomain.com, you can simply navigate to them via url’s like: https://fm.mydomain.com/intanet/ or https://fm.mydomain.com/client1/.

 

Conclusion

Wow, it’s been quite a detailed journey going through the steps to set up our SSL certificate!  If you’ve followed along until the end, thank you for your time;  I hope this guide serves you well!  Here are a few final ideas to consider:

• There are many variations in SSL vendor web sites, web hosts, DNS managers, cloud providers, etc.  If your personal variations are a little different, the underlying principals still apply, so with a little digging you should be able to find the equivalent tools and resources at your respective providers.
• If you used a custom port for SSL on your server, be sure to append the port to all your urls.  For example:
  https://fm.mydomain.com:9443/fmi/webd
• Firewall: if you haven’t set up your Windows Server firewall and/or you disabled it to make sure all connections were working properly, you will find detailed instructions on setting up the Windows Firewall for FileMaker Server here.  There is no reason to leave the solid Windows Firewall disabled! 😉
• Troubleshooting- If you run into problems, look out for these things:
  • Be sure IP forwarding is enabled on the NIC in Azure.
  • Try clearing your browser cache.
  • Try flushing your local DNS cache.  On windows, in the command prompt use: ipconfig /flushdns
  • Go over the Windows Firewall instructions to make sure your current implementation is correct.

Happy File Making!