• Who: Anyone interested in securing your FileMaker 17 Server with SSL (Wildcard or single domain)
  • What: How to Install SSL Certificate on FM 17 Server Admin Console
  • With: FileMaker 17+ (FM Server 17 introduced the new Server Admin Console)
  • Why: Security is a priority with any solution, but especially if you connect to your server over a WAN (as opposed to your local network).

This article is intended to be an update to the previous deep dive article which walked through the process of setting up a Wildcard SSL certificate in FileMaker Server 16. However, in this article we will highlight the small but significant differences that come with the new Server Admin Console in FileMaker 17.  Ultimately, following this guide you should end up with a successfully installed SSL certificate on FileMaker Server 17. The certificate will be installed on both FileMaker Server and Windows Server in order to enable encrypted connections directly to FileMaker Server (from FM Pro) and also to web pages hosted on Windows Server! I will focus on using a single domain name SSL certificate, but the same process can be used for installing a wildcard certificate.

For the most part, everything that applied in the previous article still applies.  However, a key difference is that the new FileMaker 17 Server Admin Console does not provide a way to begin the certificate signing request (or CSR).  In the previous article, I noted that it was much easier to get things right by starting the CSR from FM Server (as opposed to Windows Server), but since that isn’t an option now, what do we do?  It turns out, that we can (and should) create the CSR from the FileMaker CLI (command line interface).

What if?

-What if you already have an SSL that was working with FM Server 16? Great, if you upgraded to FM Server 17 on the same physical server, then all you have to do is import the previous/existing cert files into the FM Server 17 Admin Console! If you are using a new Windows server, then simply import the cert files and make sure to redirect your DNS settings to your new IP address.
-What if you already started the CSR from Windows or another source? No problem, you can still just start over in FM Server and ignore the previous CSR from the other source.

Generating the CSR
  • Open the Windows command prompt (in win Server 2016 right click the start button and choose Command Prompt (Admin))
  • Before you create the CSR, determine the URL your certificate will be used for.  For example: fm.mycompany.com (if you are using a wildcard, you should use *.mycompany.com).
  • You should also decide on the password you will use for importing your private key. Only alpha-numeric characters.
  • In the command prompt window, type:
    fmsadmin certificate create fm.mycompany.com –keyfilepass password .
    • The variables in red above are your personal setup info, the default color items are keywords and should be input exactly as seen.
    • Please note that the “–keyfilepass” above has 2 dashes. Some browsers automatically put these together as a “long dash”. If you get the error: “Encryption password for the private key file is not specified”, it is likely because you only have 1 dash. Another reason to get this error is because you used a special character like an exclamation point in the keyfilepass. Stick to alphanumeric characters to avoid this issue.
  • When prompted, provide the username and password for the FileMaker admin account.
  • If you receive a message that says: “Private key file already exists, please remove it and run the command again”:
    • In the command prompt, type: fmsadmin certificate delete .
    • When prompted “really delete certificate?”, type “y”  (no quotes).
    • Provide admin login credentials.
    • When done, you will be prompted to “Restart the FileMaker Server service to apply the change.”
    • Restart the FileMaker Server service (easy way is to search for it).
      Example for Win Server 2012:
      Example for Win Server 2016 (click on search icon in Task Bar)
    • In the Services window, right click the FileMaker Server item and choose “Restart”.
    • Go back to the command prompt and type the command above again to create the CSR.
  • After the CSR is created, the command prompt will simply go back to it’s default path.  There is no real indication that anything happened, but in fact two new files have been created:
    • serverRequest.pem: CSR required for the SSL purchase process.
    • serverKey.pem: private key file required for the certificate import process.
    • These new files are located in the Cstore of your FileMaker Server and named. To find the Cstore directory, navigate to /FileMaker Server/CStore/ (most likely in c:/Program Files/FileMaker/FileMaker Server/CStore ).
Purchase the SSL
    • Go to the website: https://cheapsslsecurity.com/sslproducts/domainvalidatedssl.html
    • Click on the “Add to Cart” button for the Comodo PositiveSSL.
    • Once you have purchased your SSL, you will be taken to the Order Process page. Step one is to simply state whether this is a new or CSR or a renewal.  I will choose “New”.
    • In step two, choose your SSL authentication option. If you refer to the previous article, you’ll note that in the beginning I mentioned that an SSL not only encrypts the web connection, but it verifies to end users that the web domain they are connecting to belongs to the owners of that domain. This is the step that establishes that you actually own the domain you will use the SSL on.
    • I recommend using email authentication. It is quick and easy.
      • If you are setting it all up for someone else like a client and you don’t have direct access to the pre-authorized email addresses associated with the WHOIS for the domain, you can simply have someone who does have access forward you the emails to continue the process.
  • For Step 3 (ignore the numbers in the pics, they are older and don’t reflect the new step 1 of simply indicating New or Renewal), you will need to open the serverRequest.pem file from FileMaker Server in a text editor and copy/paste it into the text area for step 3. The first line and last line that say “—–BEGIN CERTIFICATE REQUEST—–” and “—–END CERTIFICATE REQUEST—–” are necessary parts of the CSR. Do not remove them. It should look something like this:
  • On Step 4, be sure to choose “Other” as the type of server platform you are using. This is important as it sets it up the way FileMaker expects it with the private key!
  • Press the Continue button.
  • If you messed up any of the steps above, you can always come back to the Order Process page and redo this section. In fact, you can even come back months later if something goes wrong with your certificate and you need to re-generate the certificate.
  • If you’re using a different certificate vendor, this info might be helpful:
    • If you are asked to provide the cert domain name again, be sure to use the asterisk for wildcards: *.mydomain.com
    • If you are asked to choose a RSA key size, choose 2048.
    • If you are asked to select a signature hash, use SHA-2 ( or SHA-2 with SHA-1 root).
  • You should now be on the “Verify Your URL” page.  Fill out the relevant contact info and agree to the site’s policy and press the Continue button.
Prepping the Certificates
  • After finishing the order, you will receive an email from Comodo (the Certificate Authority, not the reseller) which will contain a validation code and a link to the Comodo site where you will need to apply the code.
  • After validating the code, you will receive another email from Comodo with a zip file containing your new certificate and also some intermediate certificates.

    Intermediate certificate chaining is the process of combining multiple intermediate certificates and is one of the confusing parts about using less expensive SSL certificates. More expensive SSL certs have a higher degree of authentication steps so they might require fewer intermediate certs or none at all. I’ve read that it’s likely that this is done by design to get you to spend more on the more expensive extended validation certs (that don’t need multiple intermediary certs). Fortunately, we have excellent instructions on how to do the intermediate certificate chaining for the Comodo PositiveSSL!

  • Download the zip file and unpack the files into a directory easy to access from a windows command line.
  • If you bought the Comodo PositiveSSL, you should get 4 files within the zipped download: your main certificate, and 3 intermediate certificates. Your main certificate should be something like: fm_mydomain_com.crt
    • If you bought a wildcard cert it would be something like: STAR_mydomain_com.crt
Creating the Intermediate Certificate Chain

In order for our less expensive certificate to validate with a Certificate Authority, we need to use the extra intermediate certificates to lead our end points up a chain to the final Certificate Authority. Since FileMaker server only “allows” for one intermediate certificate to be uploaded with the main certificate, we need to bundle these three together in the proper chained order. Hat tip to the Devside.net author who provides the specific steps to accomplish the certificate chain for the Comodo PositiveSSL cert.

  • Open the command line with elevated privileges and navigate to the folder where you unpacked the certs. For example:

    c:
    cd certs\
    (the above would put you in the folder-> c:\certs)

  • Use the “copy” command to concatenate the 3 intermediate certificates together:

    copy /B COMODORSADomainValidationSecureServerCA.crt + COMODORSAAddTrustCA.crt + AddTrustExternalCARoot.crt PositiveSSL.ca-bundle

    (COMODO has recently been acquired in 2019, so if you are bundling the new intermediate certs, it will look like this):

    copy /B SectigoRSADomainValidationSecureServerCA.crt + USERTrustRSAAddTrustCA.crt + AddTrustExternalCARoot.crt PositiveSSL.ca-bundle

  • The order of the certificates in the “copy” command above is important.
  • You will now have a chained intermediate certificate with the name: PositiveSSL.ca-bundle
Importing the Certificate to FileMaker Server

At this point, you have prepped your FileMaker Server to expect a certificate (by doing the CSR), purchased the certificate, and bundled the intermediate certificates into a chain. Now we can go back to FileMaker server with your certificate and bundled intermediate chain files. If the server is in the cloud, be sure to copy these files from your local computer to the server.

  • Open the FileMaker 17+ Server Admin Console and Click on the Configuration menu item, then the SSL Certificate option on the left to show the SSL Certificate section.
  • Click on the Import Custom Certificate button to open this dialog window:
  • Upload the following files:
    • Signed Certificate: is the main certificate you just received: fm_mydomain_com.crt
    • Private Key: is the serverKey.pem located in /FileMaker Server/CStore/
    • Intermediate Certificate: is the new bundled file: PositiveSSL.ca-bundle
    • Private Key Password: is the password you used during the CSR creation.
    • Click Import.
  • Restart FileMaker Server.
Bind Your Certificate to Your Default Windows Web Site

One of the magic parts about starting the CSR from FileMaker Server is that upon importing the cert, it also automatically imports to Windows Server! If you didn’t already have other web sites or services on the server and it is simply dedicated to FileMaker server, you’re done configuring the certificate on Windows!

What if?
-What if you had to change FileMaker Server’s web ports because you already had port 80 and 443 used for something else (either on the server itself or on the router in front of the server)? No problem, all you need to do is bind the new certificate to the ports you set up for FileMaker Server.

  • Open IIS on Windows Server.
  • In the “Connections” pane on the left, navigate to YOURSERVER->Sites->FMWebSite.
  • In the “Actions” page on the right, click “Bindings”.
  • For the ports you used on the FM Server setup, click on the port number and press the “Edit” button.
  • In the Host Name field, put your cert’s domain name: *.mydomain.com or fm.mydomain.com
  • In the SSL Certificate dropdown, choose your new certificate and press OK.
  • You have now bound the certificate to that port, so in order to reach your FM server, you will need to add your port to the end of your IP or domain name. For example, if you used the port 9443 as your SSL port when setting up FM Server and you want to connect to your server via a static IP, you might use: 173.10.10.15:9443, or via a subdomain like this: fm.mydomain.com:9443.
Update DNS to Point Your Subdomain Name to Your Server IP

Ultimately, we want to connect to our FileMaker server by using the subdomain: fm.mydomain.com instead of using an IP address. We can do this by creating an A record within our DNS records.

At this point technically your FileMaker Server has a newly installed custom certificate and will encrypt the connection between the server and an end point if you connect via IP address, but since the certificate validation will be looking for the certificate name as the connecting address, you will still get an error icon using the certificate. In order to get the “green padlock” icon signifying a properly configured and validated certificate, we need to update our DNS record with a with our subdomain.

With both Azure and AWS, it is simple and free to use a static IP address (if you have an in-house server, hopefully you also have a static IP address. If not, do some research on using dynapic dns updaters like no-ip.com).

In order to point our new subdomain cert fm.mydomain.com to our FM server’s IP address, we go to our DNS record manager and create an A record. There are two likely DNS record managers. If you simply purchased the domain name but are not hosting a web site, then you can go to your domain registrar to update DNS records (GoDaddy for example). If you’ve already set up a web site with a web host (like BlueHost or HostGator) and are adding a subdomain, you can go to your web site’s cpanel to update DNS records and add a subdomain.

  • GoDaddy Example:
    • Log in to GoDaddy
    • Click on your domain name’s DNS button:
    • Click the Add button at the bottom.
      • Select “A”
      • Host: use your subdomain prefix. Let’s simply use: fm (this is the prefix for: fm.mydomain.com)
      • Points to: use your static IP.
      • TTL: leave it at 1 Hour
      • Press Save
  • Cpanel Example:
    • Log in to your web host’s cpanel.
    • In your Cpanel, look for the DNS section and click on the “Simple Zone Editor”.
    • Name field: type fm.mydomain.com
    • Address field: type your static IP
    • Press the “Add an A Record” or Save button (whichever you have).
Connecting to Your FileMaker Server

Now you can connect to your FileMaker server from FileMaker using the hostname: fm.mydomain.com!

  • Open FileMaker Pro or Advanced
  • In the Launch Center, click on the Hosts tab.
  • At the bottom left, click on the + symbol to open the “Add Favorite Host” window.
    • In the Host’s Internet Address, use: fm.mydomain.com

That’s it! Now when you or a client connects to your FileMaker Server via the WAN, you will get that elusive “green padlock” and your connection will be encrypted! If you want to connect to WebDirect, you would simply use: https://fm.mydomain.com/fmi/webd.
Note that since we are connecting via SSL encryption, you must specify the “https” in the url.

If you want to do some custom web publishing, you can create your custom pages in the HTTPServer\conf folder on your FileMaker server. The full path on my server is: C:\Program Files\FileMaker\FileMaker Server\HTTPServer\conf. The “conf” directory is set up by FileMaker server as the default public www directory on your Windows Server, so all web pages/sites should go here. For example, if you have several custom sites like: intranet.mydomain.com, client1.mydomain.com and, client2.mydomain.com, you can simply navigate to them via url’s like: https://fm.mydomain.com/intanet/ or https://fm.mydomain.com/client1/.

Conclusion

Wow, it’s been quite a detailed journey going through the steps to set up our SSL certificate! If you’ve followed along until the end, thank you for your time; I hope this guide serves you well! Here are a few final ideas to consider:

  • There are many variations in SSL vendor web sites, web hosts, DNS managers, cloud providers, etc. If your personal variations are a little different, the underlying principals still apply, so with a little digging you should be able to find the equivalent tools and resources at your respective providers.
  • If you used a custom port for SSL on your server, be sure to append the port to all your urls. For example:
    https://fm.mydomain.com:9443/fmi/webd
  • Firewall: if you haven’t set up your Windows Server firewall and/or you disabled it to make sure all connections were working properly, you will find detailed instructions on setting up the Windows Firewall for FileMaker Server here. There is no reason to leave the solid Windows Firewall disabled! 😉
  • Troubleshooting- If you run into problems, look out for these things:
    • Try clearing your browser cache.
    • Try flushing your local DNS cache. On windows, in the command prompt use: ipconfig /flushdns
    • Go over the Windows Firewall instructions to make sure your current implementation is correct.

Happy File Making!