antipaucity

Last year I posted a simple script for keeping your Let’s Encrypt SSL certificates current.

In conjunction with my last post sharing the “best” SSL configs you can use with Apache on CentOS, here is the current state of the cron’d renewal script I use.

systemctl stop httpd.service
systemctl stop postfix
~/letsencrypt/letsencrypt-auto -t -n --agree-tos --keep --expand --standalone certonly --rsa-key-size 4096 -m user@domain.tld -d domain.tld
# you can append more [sub]domains to a single cert with additional `-d` directives ([-d otherdomain.tld [-d sub.domain.tld...]])
#...repeat for every domain / domain group
systemctl start httpd.service
systemctl start postfix

I have this script running @weekly in cron. You should be able to get away with doing it only every month or two .. but I like to err on the side of caution.

I’m stopping and starting Postfix in addition to httpd (Apache on my system) for only two reasons: first, I am using some of the LE-issued certs in conjunction with my Postfix install; second, because I don’t know if Dovecot and my webmail system need to make sure Postfix is restarted if underlying certs change.

In follow-up to previous posts I’ve had about SSL (specifically with Let’s Encrypt), here is the set of SSL configurations I use with all my sites. These, if used correctly, should score you an “A+” with no warnings from ssllabs.com. Note: I have an improved entropy package installed (twuewand). This is adapted from the Mozilla config generator with specific options added for individual sites and/or to match Let’s Encrypt’s recommendations.

Please note: you will need to modify the config files to represent your own domains, if you choose to use these as models.

[/etc/httpd/conf.d/defaults.conf]

#SSL options for all sites
Listen 443
SSLPassPhraseDialog  builtin
SSLSessionCache         shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout  300
Mutex sysvsem default
SSLRandomSeed startup builtin
SSLRandomSeed startup file:/dev/urandom  1024
# requires twuewand to be installed
SSLRandomSeed startup exec:/bin/twuewand 64
SSLRandomSeed connect builtin
SSLRandomSeed connect file:/dev/urandom 1024
SSLCryptoDevice builtin
# the SSLSessionTickets directive should work - but on Apache 2.4.6-45, it does not
#SSLSessionTickets       off
SSLCompression          off
SSLHonorCipherOrder	on
# there may be an unusual use case for enabling TLS v1.1 or 1 - but I don't know what that would be
SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
SSLOptions +StrictRequire
SSLUseStapling on
SSLStaplingResponderTimeout 5
SSLStaplingReturnResponderErrors off
SSLStaplingCache        shmcb:/var/run/ocsp(128000)

#all unknown requests get domain.tld (over http)
<VirtualHost *:80>
    DocumentRoot /var/html
    ServerName domain.tld
    ServerAlias domain.tld *.domain.tld
    ErrorLog logs/domain-error_log
    CustomLog logs/domain-access_log combined
    ServerAdmin user@domain.tld
    <Directory "/var/html">
         Options All +Indexes +FollowSymLinks
         AllowOverride All
         Order allow,deny
         Allow from all
    </Directory>
</VirtualHost>

SetOutputFilter DEFLATE
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript text/css text/php

[/etc/httpd/conf.d/z-[sub-]domain-tld.conf]

<Virtualhost *:80>
    ServerName domain.tld
# could use * instead of www if you don't use subdomains for anything special/separate
    ServerAlias domain.tld www.domain.tld
    Redirect permanent / https://domain.tld/
</VirtualHost>

<VirtualHost *:443>
    SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem
# if you put "fullchain.pem" here, you will get an error from ssllabs
    SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem
    DocumentRoot /var/www/domain
    ServerName domain.tld
    ErrorLog logs/domain-error_log
    CustomLog logs/domain-access_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
    ServerAdmin user@domain.tld

# could put this in defaults.conf - I prefer it in each site config
    SSLEngine on

<Files ~ "\.(cgi|shtml|phtml|php3?)$">
    SSLOptions +StdEnvVars
</files>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>

SetEnvIf User-Agent ".*MSIE.*" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

    <Directory "/var/www/domain">
         Options All +Indexes +FollowSymLinks
         AllowOverride All
         Order allow,deny
         Allow from all
    </Directory>

</VirtualHost>

I use the z....conf formatting to ensure all site-specific configs are loaded after everything else. That conveniently breaks every site into its own config file, too.

The config file for a non-https site is much simpler:

<VirtualHost *:80>
    DocumentRoot /var/www/domain
    ServerName domain.tld
    ServerAlias domain.tld *.domain.tld
    ErrorLog logs/domain-error_log
    CustomLog logs/domain-access_log combined
    ServerAdmin user@domain.tld
    <Directory "/var/www/domain">
         Options All +Indexes +FollowSymLinks
         AllowOverride All
         Order allow,deny
         Allow from all
    </Directory>
</VirtualHost>

If you’re running something like Nextcloud, you may want to turn on Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains" in the <VirtualHost&gt directive for the site. I haven’t decided yet if I should put this in every SSL-enabled site’s configs or not.

In my how-to for Let’s Encrypt, I gave an example script that can be called via cron (or manually) which will renew Let’s Encrypt SSL certificates under CentOS 6.

If you want to do it on CentOS 7 (which is what I am now running), use the following:

cd ~/letsencrypt
git pull
systemctl stop httpd.service
~/letsencrypt/letsencrypt-auto --agree-tos --keep --rsa-key-size 2048 --standalone certonly -m user@domain.tld -d domain.tld [-d sub.domain.tld [-d ...]]
systemctl start httpd.service


Now, what does this script do? Step by step:

1. clear-out the last grab of the Let’s Encrypt git repo (there’s probably a better way to do this, but I don’t know what it is)
2. go to root’s home (/root)
3. clone-down the Let’s Encrypt toolset
4. stop httpd (Apache in my case, though you might be running nginx or something else
5. run the cert tool in automated form:
   1. agree to terms of service
   2. keep current cert if it doesn’t need to be updated
   3. key size of 2048 bits
   4. run the standalone webserver to verify “ownership” of the domain
   5. generate just the cert
   6. administrative email (optional, but “encouraged”)
   7. domain(s) to issue cert for (must be individually identified with successive -d flags; LE does not support wildcard certs)
6. restart httpd

I set mine to run @weekly in cron – @monthly is likely good enough, but since it’s “free” to run, running slightly more than is necessary seems good to me. Plus, if you’re getting SSL certs for many domains all being served from the same server, they may have different expiration dates, so running more often is better.

My crontab entry for renewing certs:

@weekly /root/renew-le-ssl.sh

There’s been quite a bit of excitement surrounding Let’s Encrypt recently – a truly 100% free SSL issuer.

Last week I helped a friend of mine get his first Let’s Encrypt certificate generated and configured for his website. One of the things I found incredibly frustrating is that Let’s Encrypt does not have a package for Red Hat/CentOS/Fedora! Ignoring such a massive installed base seems monumentally dumb – so I hope that they correct it soon. Until they do, however, here’s a tutorial that should cover the gotchas for getting Let’s Encrypt to work on a CentOS 6 server with Apache 2.

The documentation (as of 06 Jan 2015) on the Let’s Encrypt website is in error in a few places (or, at least, not as correct as is could/should be). One big thing to note, for example, is that it says Python 2.6 is supported (the current release for RHEL/CentOS 6). If you run the certificate generator without the --debug flag, though, it will error-out saying Python 2.6 is not supported.

While I used an existing CentOS 6 server, I’ll start this tutorial as I have many others – by telling you to go get a CentOS 6 server from Digital Ocean or Chunk Host.

Preliminaries

Login as root (or a sudo-privileged account – but root is easier), and install Apache, Python, and SSL – yum install httpd python mod_ssl.

Also enable the EPEL repository: yum install epel-repository (available from the CentOS Extras repository. I’m going to assume you are familiar with configuring Apache, and will only provide the relevant snippets from ssl.conf herein.

Now that the basics are done, let’s move to Let’s Encrypt. I ran the tool in interactive mode (which is going to require ncurses to be available – it’s probably already installed on your system) – but you’ll want to add a crontab entry since Let’s Encrypt certs expire after 90 days, so I’ll compact the interactive session into a single command-line call at the end, which you’ll need to “know” how to do, since the --help argument doesn’t do anything yet (that I could find).

Initial Certificate Creation

First, grab the latest Let’s Encrypt from GitHub:
git clone https://github.com/letsencrypt/letsencrypt && cd letsencrypt

Stop Apache: service httpd stop. Let’s Encrypt is going to try to bind to ports 80 and 443 to ensure you have control the domain.

Now run the letsencrypt-auto tool – in debug mode so it’ll work with Python 2.6: ./letsencrypt-auto --debug certonly.

Use certonly because the plugins to automate installing for Apache and Nginx don’t work on CentOS yet.

Enter your domain name(s) for which you want to issue a certificate. If you accept incoming connections to www.domain.tld and domain.tld, be sure to put both in the list (likewise, if you have, say, blog.domain.tld that you want included).

Enter an administrative email address.

When the tool finishes, it’ll put symlinks in /etc/letsencrypt/live/domain.tld, with the “actual” certs in /etc/letsencrypt/archive/domain.tld. We’re going to reference the symlinks in /etc/letsencrypt/live/domain.tld next.

Edit /etc/httpd/conf.d/ssl.conf (I prefer emacs – but use whatever you prefer), and add the following lines in your VirtualHost directive:
SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem
SSLCACertificateFile /etc/letsencrypt/live/domain.tld/cert.pem


Restart Apache – service httpd start.

Try hitting https://domain.tld in your web browser – and you should be golden!

Automating Renewal

Create a small shell script called renew-LE-certs.sh somewhere you’ll remember where it is – like /root:
service httpd stop
# add additional '-d' entries for more subdomains
/path/to/letsencrypt/letsencrypt-auto --debug --keep --agree-tos --rsa-key-size 2048 certonly -m ssladmin@domain.tld -d domain.tld -d www.domain.tld
service httpd start

For your crontab entry, do the following to setup monthly cert renewal:
@monthly /path/to/renew-LE-certs.sh

I noticed I haven’t updated my previous post on keeping my Let’s Encrypt certs updated since building-out a couple new servers.

systemctl stop httpd.service
/bin/certbot renew --preferred-challenges http-01,dns-01 --must-staple
systemctl start httpd.service
systemctl restart postfix

Not a big change, but one worth making

If you haven’t installed certbot yet, you should

After a friend of mine told me he wanted to deploy Jitsi on my main webserver, and me saying “sure”, I decided I wanted to get it up and running on a new server both so I knew how to do it, and to avoid the latency issues of videoconferencing from central North America to Germany and back.

Before I go into how I got it working, let me say that the official Quick Start guide is good – but it doesn’t cover anything but itself.

Here’s the basic setup:

• CentOS 7
• Docker
• Apache 2.4

What To Do:

Once you have your new CentOS instance up and running (I used Vultr), here’s everything you need to install:

yum -y install epel-release && yum -y upgrade && yum -y install httpd docker docker-compose screen bind-utils certbot git haveged net-tools mod_ssl

I also installed a few other things, but that’s because I’m multi-purposing this server for Squid, and other things, too.

Enable Apache, firewalld, & Docker:

systemctl enable httpd && systemctl enable docker && systemctl enable firewalld

Now get your swap space setup:

fallocate -l 4G /swapfile && chmod 0600 /swapfile && mkswap /swapfile && swapon /swapfile

Add the following line to the bottom of your /etc/fstab:

/swapfile swap swap default 0 0

Restart your VPS:

shutdown -r now

Get your cert from Let’s Encrypt (make sure you’ve already setup appropriate CAA & A records for your domain and any subdomains you want to use):

certbot -t -n --agree-tos --keep --expand --standalone certonly --must-staple --rsa-key-size 4096 --preferred-challenges dns-01,http-01 -m <user>@<domain.tld> -d <jitsi.yourdomain.tld>

Create a root crontab entry to run certbot frequently (I do @weekly ~/renew-le.sh)

Go to the home directory of whatever user you plan to run Jitsi as:

su - <jitsi-user>

Begin the Quick Start directions:

• git clone https://github.com/jitsi/docker-jitsi-meet && cd docker-jitsi-meet
• mv env.example .env
• Change the timezone in .env from Europe/Amsterdam if you want it to show up in a sane timezone (like Etc/UTC)
• mkdir -p ~/.jitsi-meet-cfg/{web/letsencrypt,transcripts,prosody,jicofo,jvb}
• docker-compose up -d

Now configure Apache for SSL. Start with this reference I posted.

But in the [sub]domain-specific conf file z-[sub]domain-tld.conf, add proxy and authentication lines (so that only people you allow to use your video conference can actually use it):

ProxyPreserveHost on
ProxyPass / http://localhost:8000/ nocanon
ProxyPassReverse / http://localhost:8000/
ProxyRequests       off
ServerAdmin warren@warrenmyers.com
AllowEncodedSlashes NoDecode
<Proxy http://localhost:8000/*>
    Order deny,allow
    Allow from all
    Authtype Basic
    Authname "Password Required"
    AuthUserFile /etc/httpd/.htpasswd
    Require valid-user
</Proxy>
RewriteEngine       on
RewriteRule        ^/meetwith/(.*)$ http://%{HTTP_HOST}/$1 [P]
ProxyPassReverseCookiePath /meetwith /

Reload your configs, and make sure they’re happy, fixing any errors that may exist:

apachectl graceful

Setup at least one user who’ll be able to access the site:

htpasswd -B -c /etc/httpd/.htpasswd <user>

You should also configure firewalld to allow only what you want (http, https, ssh):

firewall-cmd --zone=public --add-service=http && firewall-cmd --zone=public --add-service=https && firewall-cmd --zone=public --add-service=ssh

With any luck, when you now navigate to https://[sub.]domain.tld in your web browser, and enter your username and password you created with htpasswd, you’ll get the Jitsi welcome page!

Other Resources:

• Check out the Jitsi subreddit – r/jitsi
• Jitsi on Twitter: @jitsinews

About 2 years ago, I started running Pi-hole as a DNS resolver and ad-blocker. Then last year, I ditched it.

After seeing a recent post by Troy Hunt, though, I thought it might be worth revisiting..but I needed a better way to control how it worked.

Enter OpenVPN – a service I already run on three endpoints. Here’s what I did:

Install Pi-hole per the usual (curl -sSL https://install.pi-hole.net | bash if you’re feeling brave, curl -sSL https://install.pi-hole.net, inspect, then run, if you’re feeling a little more wary).

This time, though, I set my upstream DNS providers to Cloudflare (1.1.1.1) and Quad9 (9.9.9.9) instead of Freenom and Google.

I also did a two-step install – once with Pi-hole listening on the primary network interface on my OpenVPN endpoint (ie the public IP), and then, once I made sure all was happy, I flipped it to listen on tun0 – the OpenVPN-provided interface. This means Pi-hole can only hear DNS queries if you’re connected to the VPN.

Why the change from how I’d done it before? Two reasons (at least):

First, if you leave Pi-hole open to the world, you can get involved in DNS amplification attacks. That is muy no bueno.

Second, sometimes I don’t care about ads – sometimes I do. I don’t care, for example, most of the time when I’m home. But when I’m traveling or on my iPhone? I care a lot more then.

Bonus – since it’s only “working” when connected to my VPN, it’s super easy to check if a site isn’t working because of Pi-hole, or because it just doesn’t like my browser (hop off the VPN, refresh, and see if all is well that wasn’t when on the VPN).

Changes you need to make to your OpenVPN’s server.conf:


push "dhcp-option DNS 10.8.0.1"


This ensures clients use the OpenVPN server as their DNS resolver. (Note: 10.8.0.1 might not be your OpenVPN parent IP address; adjust as necessary.) Restart OpenVPN after making this change.

My setupVars.conf for Pi-hole:


PIHOLE_INTERFACE=tun0
IPV4_ADDRESS=10.8.0.1/24
IPV6_ADDRESS=
QUERY_LOGGING=true
INSTALL_WEB_SERVER=true
INSTALL_WEB_INTERFACE=true
LIGHTTPD_ENABLED=false
WEBPASSWORD=01f3217c12bcdf8aa0ca08cdf737f99cd68a46dbdc92ce35fd75f39ce2faaf81
DNSMASQ_LISTENING=single
PIHOLE_DNS_1=1.1.1.1
PIHOLE_DNS_2=1.0.0.1
PIHOLE_DNS_3=9.9.9.9
DNS_FQDN_REQUIRED=true
DNS_BOGUS_PRIV=true
DNSSEC=false
CONDITIONAL_FORWARDING=false


I tried getting lighttpd to only listen on on port 443 so I could use Let’s Encrypt’s SSL certs following a handful of tutorials and walk-throughs, but was unsuccessful. So I disabled lighttpd, and only start it by hand if I want to check on my Pi-hole’s status.

Speaking of which, as I write this, here is what the admin console looks like:

Hope this helps you.