antipaucity

fighting the lack of good ideas

turn on spf filtering with postfix and centos 7

After running my new server for a while, I was noticing an unusually-high level of bogus email arriving in my inbox – mail that was being spoofed to look like it was coming from myself (to myself).

After a great deal of research, I learned there is a component of the DNS specification that allows for TEXT or SPF records. Sender Policy Framework was developed to help mail servers identify whether or not messages are being sent by authorized servers for their representative domains.

While there is a huge amount of stuff that could be added into a SPF record, what I am using for my domains is:

"v=spf1 mx -all"

Note: some DNS providers (like Digital Ocean) will make you use a TEXT record instead of a dedicated SPF record (which my registrar / DNS provider Pairnic supports).

If they require it be via TEXT record, it’ll look something like this: TXT @ "v=spf1 a include:_spf.google.com ~all"

Starting with this old how-to I found for CentOS 6, I added the policy daemon for Postfix (though it’s now in Python and not Perl) thusly:

yum install pypolicyd-spf

(I already had the EPEL yum repository installed – to get it setup, follow their directions, found here.)

Then I edited the master.cf config file for Postfix, adding the following at the bottom:

policy unix - n n - 0 spawn user=nobody argv=/bin/python /usr/libexec/postfix/policyd-spf

Note: those are actually tabs in my config file – but spaces work, too.

When you’re done with your edits and record additions, restart Postfix:

systemctl restart postfix

Then you’ll see messages like this in your /var/log/maillog file:

Apr 23 18:58:59 khopesh postfix/smtpd[18199]: NOQUEUE: reject: RCPT from unknown[197.27.40.169]: 550 5.7.1 <warren@datente.com>: Recipient address rejected: Message rejected due to: SPF fail - not authorized. Please see http://www.openspf.net/Why?s=mfrom;id=warren@datente.com;ip=197.27.40.169;r=warren@datente.com; from=<warren@datente.com> to=<warren@datente.com> proto=ESMTP helo=<[197.27.40.169]>

And if you follow the directive to go visit the “Why” page on OpenSPF, you’ll see something like this explanation:


Why did SPF cause my mail to be rejected?

What is SPF?

SPF is an extension to Internet e-mail. It prevents unauthorized people from forging your e-mail address (see the introduction). But for it to work, your own or your e-mail service provider’s setup may need to be adjusted. Otherwise, the system may mistake you for an unauthorized sender.

Note that there is no central institution that enforces SPF. If a message of yours gets blocked due to SPF, this is because (1) your domain has declared an SPF policy that forbids you to send through the mail server through which you sent the message, and (2) the recipient’s mail server detected this and blocked the message.

warren@datente.com rejected a message that claimed an envelope sender address of warren@datente.com. warren@datente.com received a message from 197.27.40.169 that claimed an envelope sender address of warren@datente.com.

However, the domain datente.com has declared using SPF that it does not send mail through 197.27.40.169. That is why the message was rejected.


helping a magpierss-powered site perform better

I rely on MagpieRSS to run one of my websites. (If you'd like to see the basic code for the site, see my GitHub profile.)

One of the drawbacks to Magpie, and dynamic websites in general, is they can be bottlenecked by external sources – in the case of Magpie, those sources are the myriad RSS feeds that Datente draws from.

To overcome some of this sluggishness, and to take better advantage of the caching feature of Magpie, I recently started a simple cron job to load every page on the site every X minutes – this refreshes the cache, and helps ensure reader experience is more performant. By scheduling a background refresh of every page, I cut average page load times by nearly a factor of 10! While this is quite dramatic, my worst-performing page was still taking upwards of 10 seconds to load a not-insignificant percentage of the time (sometimes more than a minute!) 🙁

Enter last week's epiphany – since RSS content doesn't change all that often (even crazy-frequent-updating feeds rarely exceed 4 updates per hour), I could take advantage of a "trick", and change the displayed pages to be nearly static (I still have an Amazon sidebar that's dynamically-loaded) – with this stupidly-simple hack, I cut the slowest page load time from ~10-12 seconds to <1: or another 10x improvement!

"What is the 'trick'," you ask? Simple – I copied every page and prefixed it with a short character sequence, and then modified my cron job to still run every X minutes, but now call the "build" pages, redirecting the response (which is a web page, of course) into the "display" pages. In other words, make the display pages static by building them in the background every so often.

If you'd like to see exactly how I'm doing this for one page (the rest are similar), check out this stupidly-short shell script:

(time (/bin/curl -f http://datente.com/genindex.php > ~/temp.out)) 2>&1 | grep real

(The time is in there for my cron reports.)

Compare the run time to the [nearly] static version:

(time (/bin/curl -f http://datente.com/index.php > ~/temp.out)) 2>&1 | grep real

how did i never know about .ssh/config?

I’m sure folks have tried to explain this to me before, but it wasn’t until today that it finally clicked – using .ssh/config will save you a world of hurt when managing various systems from a Linux host (I imagine it works on other platforms, too – but I’ve only started using it on CentOS).

Following directions I found here, I started a config file on a server I use as a jump box. In it I have an entry for my web server, and I’ll be adding other frequently-accessed servers to it as time goes on.

Thanks, nerderati, man pages … and whomever else tried to explain this to me before but I didn’t grok.

wordpress plugins i use

As promised last time, I now have a page dedicated to the WordPress plugins I use.

Check it out, here.

use prettypress if you’re running a wordpress blog

Like my list of used Chrome Extensions, I’m building a list of recommended WordPress plugins.

But until I get it done, I have to give some pretty big props to PrettyPress. It’s a plugin that lets you edit in Visual, Text, and Markdown – the markup format of sites like reddit, GitHub,, GitLab, and the Stack Exchange family.

prettypress-screenshot

a couple months late – but my prediction was pretty close

Tesla’s Model 3 is debuting at $35,000.

That is distinctly in the range of most “normal” people to obtain.

It should be shipping sometime in 2017. I’d buy one.

improve your entropy pool in linux

A few years ago, I ran into a known issue with one of the products I use that manifests when the Red Hat Linux server it’s running on has a low entropy pool. And, as highlighted in that question, the steps I found 5 years ago didn’t work for me (turns out modifying the t parameter from ‘1’ to ‘.1’ did work (rngd -r /dev/urandom -o /dev/random -f -t .1), but I digress (and it’s no longer correct in CentOS 7 (the ‘t’ option, that is))).

In playing around with the Mozilla-provided SSL configurator, I noticed a line in the example SSL config that referenced “truerand”. After a little Googling, I found an opensource implementation called “twuewand“.

And a little more Googling about adding entropy, and I came across this interesting tutorial from Digital Ocean for “haveged” (which, interestingly-enough, allowed me to answer a 6-month-old question on Server Fault about CloudLinux).

Haveged “is an attempt to provide an easy-to-use, unpredictable random number generator based upon an adaptation of the HAVEGE algorithm. Haveged was created to remedy low-entropy conditions in the Linux random device that can occur under some workloads, especially on headless servers.”

And twuewand “is software that creates hardware-generated random data. It accomplishes this by exploiting the fact that the CPU clock and the RTC (real-time clock) are physically separate, and that time and work are not linked.”

For workloads that require lots of entropy (generating SSL keys, SSH keys, PGP keys, and pretty much anything else that wants lots of random (or strong pseudorandom) seeding), the very real problem of running out of entropy (especially on headless boxes or virtual machines) is something you can face quite easily / frequently.

Enter solutions like OpenRNG which are hardware entropy generators (that one is a USB dongle (see also this skh-tec post)). Those are awesome – unless you’re running in cloud space somewhere, or even just a “traditional” virtual machine.

One of the funny things about getting “random” data is that it’s actually very very hard to get. It’s easy to describe, but generating “truly” random data is incredibly difficult. (If you want to have an aneurysm (or you’re like me and think this stuff is unendingly fascinating), go read the Wikipedia entry on “Cryptographically Secure Pseudo Random Number Generator“.)

If you’re in a situation, though, like I was (and still am), where you need to maintain a relatively high quantity of fairly decent entropy (probably close to CSPRNG level), use haveged. And run twuewand occasionally – at the very least when starting Apache (at least if you’re running HTTPS – which you should be, since it’s so easy now).