Security at GlobNIX / Spodhuis

Prior to the advent of Let's Encrypt, my approach was to have almost all of my sites use HTTPS with certificates from a private CA described below. For bootstrapping, and to expose my notes, the site used, by contrast, a TLS/SSL certificate (PKIX) issued by an established Certificate Authority, for which I paid money.

Now that Let's Encrypt is widely available, any website intended for public use will normally use a certificate issued by that CA. A few experimental sites, or services using name formats not supported by LE, or otherwise doing Weird Things, still use one of my private CAs.

GlobNIX Certificate Authority

This is not for general public use. This Certificate Authority (or CA) is not warranted to be fit for purpose, useful or secure. It's for one person who maintains a number of TLS-enabled services to issue their own certs. It's also used for self-education, so in the process many mistakes will probably be made and certainly have been made.

You should be able to just click on the .crt file to load a CA certificate into most popular browsers. The PEM version is encoded as plain-text and often put into /etc/ssl/certs/ and then the c_rehash command run to create the necessary links.

If you wish to verify the globnixCA4.crt key, are prepared to do things the long way around, and have an OpenPGP (eg, GnuPG) public key-ring with a trust-paths to Phil, then instead download and save the globnixCA4.pem key and the globnixCA4.pem.asc signature upon the key. Then verify the key with either of:

prompt$ gpg --verify globnixCA4.pem.asc
prompt$ gpg --verify globnixCA4.pem.asc globnixCA4.pem

(the second is just more explicit) which should report something like the below, but with more sensible email addresses:

gpg: Signature made Tue 02 Sep 2014 04:15:55 AM EDT
gpg:                using RSA key 0x4D1E900E14C1CC04
gpg: Good signature from "Phil Pennock <censored@example.tld>"
gpg:                 aka "Phil Pennock <despise-spammers@webharvesters.invalid>"

After that, you just convert the key into .crt form and load it into your web-browser. You can do this in one of two ways. The easy way and the backdoor way. First, the conversion and an example of how to peek inside the certificate to see what data is present:

prompt$ openssl x509 -inform pem -outform der -in globnixCA4.pem -out globnixCA4.crt
prompt$ openssl x509 -inform pem -in globnixCA4.pem -noout -text

Browser-based install

The easy way works in any browser which hasn't been locked down. You point the browser at the file and the browser will auto-import it. You can use a file:/// URL to do this. This approach works in Firefox, MSIE, Opera, etc. (I don't remember if it works in Camino and Safari; I think it does and just loads the cert into the appropriate keychain). For remote retrieval you should need to ensure that the file is served with the correct MIME type (application/x-x509-ca-cert).

Installing without using the browser

The backdoor way involves manipulating the keystore without using the browser. Two known approaches are described here: graphically (MacOS and Windows) and command-line (any Mozilla-derived browser, Java-derived management systems such as the Android phone).

MacOS and Windows

On MacOS X this can be done with Keychain Access; select the keychain to import into, then add a cert, and load from the file. It should be fairly intuitive to manage. You can access the keychain via the command-line using the certtool command. On modern MS Windows there's an Internet configuration pane, typically reached from MSIE preferences, which provides access to loading certs into the system cert DB for this user; you can also manage the certs via the Microsoft Management Console (mmc) by loading a cert manager snap-in pointed to the appropriate keystore. It has been some time since I last did this, so no further details are provided here.

Mozilla command-line (eg, OLPC laptop)

Any Mozilla-based browser will be using the Netscape Security Services backend for managing the certificates; there are command-line tools to manage the files which keys & certs are held in (within your profile directory); if you manage the certs from the command-line, you'll probably need to close the browser (completely) and restart to pick up the changes. If you can use the normal GUI interface, that's much easier. I've had to use the command-line tools three times; I forget the reason for the first time, but the second time was to load the keys into the Sugar Gecko-derived browser on an OLPC laptop, which comes with just one CA by default; the third time was for an Android G1 phone.

The required command-line tool is certutil; which packaged unit of software provides this tool depends upon the OS. On FreeBSD, it's security/nss; on RPM systems, it appears to be nss-tools, such that on an OLPC system the command to install the tools was: yum install nss-tools. On a Debian-derived system, libnss3-tools and so apt-get install libnss3-tools.

For this approach, you can even work directly from the PEM files. The first command just lists all known certs. I then show use of openssl to extract the name from the certificate, to pick a reasonable nickname. Then I import the certificate. In this example, the -t cTC sets the trust for the loaded certificate, declaring it to be a valid CA (c), a trusted CA to issue client certs (T) and a trusted CA to issue server certs (C).
Adjust as appropriate for how much you trust me.

$ certutil -L -d ~/.sugar/default/gecko/
$ openssl x509 -noout -subject -in globnixCA4.pem
$ certutil -A -t cTC -d ~/.sugar/default/gecko/  -n 'GlobNIX Certificate Authority 4' -i globnixCA4.pem 

New NSS command-line (eg, Google Chrome on Linux)

The Mozilla NSS libraries, as of 3.12.x, support a new shared storage system based on SQLite. The current scheme, described in the Mozilla wiki page NSS_Shared_DB and used at present by Google Chrome Linux builds use these in the directory ~/.pki/nssdb/. The files are named cert9.db & key4.db.

You can reference the SQLite using either a sql: prefix on the directory or by changing the default DB type using an environment variable, setting NSS_DEFAULT_DB_TYPE=sql. One problem is that in current Chrome Linux alphas, no password is set on the NSS DBs; resetting the password to be the empty string will get rid of the error. (Thanks to Wan-Teh Chang for pointing out the solution to me). Chrome will successfully use DBs with an empty password.

$ cd ~/.pki/nssdb
$ certutil -d sql:$PWD -L
$ export NSS_DEFAULT_DB_TYPE=sql
$ certutil -d $PWD -L
$ certutil -A -t cTC -d $PWD -n 'GlobNIX Certificate Authority 4' -i ~/globnixCA4.pem
certutil: could not authenticate to token NSS Certificate DB.: An I/O error occurred during security authorization
### Because that failed, we change from no password to an empty password.
$ certutil -d $PWD -W
Enter a password which will be used to encrypt your keys.
The password should be at least 8 characters long,
and should contain at least one non-alphabetic character.

Enter new password: [Just press Enter/Return, empty password]
Re-enter password:  [Ditto]
$ certutil -A -t cTC -d $PWD -n 'GlobNIX Certificate Authority 4' -i ~/globnixCA4.pem


As of Android 2.1, you can install a certificate via the browser, but can not set trust privileges on that certificate. This allows you to have CA certs available for situations where a CA cert is chosen manually. At this time, the only use I know of for this is 802.1x security for WiFi networks; confirmed as of Android 2.2 (Froyo), that's: Settings → Wireless & networks → Wi-Fi settings → Add Wi-Fi network and set Security to 802.1x EAP to see the option to choose a certificate which you've installed just by clicking on it.

As far as I am aware (and I have no special insight into Android), to use your own CA for use with websites, you'll still need to do the manual install which I describe below.

For full installation on an Android phone, you need root access. Typically, this might be by having an unlocked phone such as the ADP1. You'll need the Android SDK, a JDK and the Bouncy Castle security provider. You need the SDK to be able to transfer files back and forth across a USB cable, the JDK to get the keytool command and the Bouncy Castle java files to get support for the file format used for the certificate store.

These notes will include OS vendor-specific paths by way of example, adjust as needed; the notes also assume that you haven't previously messed with an Android phone, so will get you started.

Versions: These commands were tested with the 1.0/1.1 system images; the OTA update for the 1.5 (Cupcake) release put a regular production build on my ADP1 so I've lost root access for now, until I invest the time into sorting this out (I've stuck with official images until now). With 1.5, there is a new SDK which provides an adb root command; after upgrading the SDK, the error message from that is what finally clued me in that the permission problems were caused by a revert to regular prod builds. (A previous update, which I believe was 1.0 to 1.1, kept me on dev builds).

  1. Get adb working such that adb devices will show your phone; for me, using an Ubuntu-derivative OS, I created /etc/udev/rules.d/50-android.rules containing:
    SUBSYSTEM=="usb_device", SYSFS{idVendor}=="0bb4", GROUP="plugdev"
    and /etc/init.d/udev reload; connect the Android phone after doing this; then adb kill-server to kill an old server and adb devices will then start a server and list the device. If not, get this working for you before continuing.
  2. Run adb shell and use su (no password) to become root. Mount /system read-write with:
    mount -w -o remount /dev/block/mtdblock3 /system
    (but use the correct device for you). From a host command-line, run:
    adb pull /system/etc/security/cacerts.bks .
    to pull the file to the current directory “.” for editing. Make a backup copy of this file, for recovery!
  3. Confirm that you can see the contents of the file, with:
        keytool \
             -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider \
             -storepass changeit -keystore cacerts.bks -list
    That's a fixed password used. If you get a java.lang.NullPointerException then you omitted the password. If you get complaints about the provider not being found, then make sure that you have it installed (it's not common; for me, the .deb named libbcprov-java was needed) and that it's in your classpath; eg, by dropping a symlink to /usr/share/java/bcprov.jar into the directory <jdk-dir>/jre/lib/ext/. If you edit <jdk-dir>/jre/lib/security/ to list
    on the end of the security.provider.<N> list then you won't need to explicitly specify the -provider in commands; examples below will assume that this has been done (for brevity), add it back in if you choose to not make this trust assertion.
  4. Add the CA cert with:
        keytool -storetype BKS -storepass changeit -keystore cacerts.bks \
             -import -file globnixCA4.pem -trustcacerts
    Answer yes at the prompt. Consider using an -alias option when adding the cert.
  5. Put the file back on the Android phone:
    unix$ adb push cacerts.bks /sdcard/new-cacerts.bks
    phone# dd if=/sdcard/new-cacerts.bks of=/etc/security/new-cacerts.bks
    phone# chmod 644 /etc/security/cacerts.bks
    phone# rm /sdcard/new-cacerts.bks
    phone# mv /etc/security/new-cacerts.bks /etc/security/cacerts.bks
    phone# mount -r -o remount /dev/block/mtdblock3 /system
  6. Exit the adb shell (EOF (Ctrl-D) twice), disconnect phone, test. If the browser crashes when you visit an SSL-enabled site, then you might have forgotten the chmod step. Oops. Note that this site does not use a globnixCA cert (see intro above). If you're choosing to install my CA cert, then you might try the rather sparse page currently at which uses an SSL cert issued by globnixCA$N.