fighting the lack of good ideas

a semi-permanent psa on passwords

Passwords should never expire:

Passwords should not be changed often:

Password “complexity” is – mostly – a joke:

You have been breached:

Passphrases are better than passwords – and will generate them for you (it will also generate random passwords that pass complexity requirements)

Use a password manager of some kind

do you leak?

It would seem I have configured OpenVPN, Squid proxy, and, to a lesser extent, Pi-hole well – none of the major sites that report IP, DNS, and other connection-related security issues find anything out of the ordinary when I’m either running “just” proxied, or VPN, or VPN+proxy.

You should check yourself hereon:

  2. (ironic this site isn’t serving itself over https)

And, of course, if you just want to see what your pubic IP address is, go hit my service –

rethinking pi-hole (again)

About 2 years ago, I started running Pi-hole as a DNS resolver and ad-blocker. Then last year, I ditched it.

After seeing a recent post by Troy Hunt, though, I thought it might be worth revisiting..but I needed a better way to control how it worked.

Enter OpenVPN – a service I already run on three endpoints. Here’s what I did:

Install Pi-hole per the usual (curl -sSL | bash if you’re feeling brave, curl -sSL, inspect, then run, if you’re feeling a little more wary).

This time, though, I set my upstream DNS providers to Cloudflare ( and Quad9 ( instead of Freenom and Google.

I also did a two-step install – once with Pi-hole listening on the primary network interface on my OpenVPN endpoint (ie the public IP), and then, once I made sure all was happy, I flipped it to listen on tun0 – the OpenVPN-provided interface. This means Pi-hole can only hear DNS queries if you’re connected to the VPN.

Why the change from how I’d done it before? Two reasons (at least):

First, if you leave Pi-hole open to the world, you can get involved in DNS amplification attacks. That is muy no bueno.

Second, sometimes I don’t care about ads – sometimes I do. I don’t care, for example, most of the time when I’m home. But when I’m traveling or on my iPhone? I care a lot more then.

Bonus – since it’s only “working” when connected to my VPN, it’s super easy to check if a site isn’t working because of Pi-hole, or because it just doesn’t like my browser (hop off the VPN, refresh, and see if all is well that wasn’t when on the VPN).

Changes you need to make to your OpenVPN’s server.conf:

push "dhcp-option DNS"

This ensures clients use the OpenVPN server as their DNS resolver. (Note: might not be your OpenVPN parent IP address; adjust as necessary.) Restart OpenVPN after making this change.

My setupVars.conf for Pi-hole:


I tried getting lighttpd to only listen on on port 443 so I could use Let’s Encrypt’s SSL certs following a handful of tutorials and walk-throughs, but was unsuccessful. So I disabled lighttpd, and only start it by hand if I want to check on my Pi-hole’s status.

Speaking of which, as I write this, here is what the admin console looks like:

admin console screenshot

Hope this helps you.

finally starting to get some good docs amassed

I had a decent library of documentation, templates, hand-offs, slide decks, etc in my pre-Splunk consulting life (technically, I still have them).

It’s nice to be finally getting a decent collection to draw from for my customers in my post-automation consulting life.

you can’t disaggregate

Had a customer recently ask about to disaggregate a Splunk search that had aggregated fields because they export to CSV horribly.

Here’s the thing.

You can’t disaggregate aggregated fields.

And there’s a Good Reason™, too: aggregation, by definition, is a one-way street.

You can’t un-average something.

Average is an aggregation function.

So why would you think you could disaggregate any other Splunk aggregation operation (like values or list)?

You can’t.

And you shouldn’t be able to (as nice as the theoretical use case for it might be).

So what is a body to do when you have a use case for a clean-to-export report that looks as if it had been aggregated, but every field in each row cleanly plunks-out to a single comma-separated value?

Here’s what I did:

{parent search}
| join {some field that'll exist in the subsearch}
[ search {parent search}
 | stats {some stats functions here} ]
| fields - {whatever you don't want}
| sort - {fieldname}

What does that end up doing?

The subsearch is identical to the outer search, plus whatever filtering/where/|stats you might want/need to do.

Using the resultant, filtered set, join on a field you know will be unique [enough].

Then sort however you’d like, and remove whatever fields you don’t want in the final display.

Of course, be sure your subsearch will complete in under 60 seconds and/or return fewer than 10,000 lines (unless you’ve modified your Splunk limits.conf)

stats values vs stats list in splunk

Splunk’s | stats functions are incredibly useful and powerful.

There are two, list and values that look identical…at first blush.

But they are subtly different. Here’s how they’re not the same.

values is an aggregating, uniquifying function.

list is an aggregating, not uniquifying function.

“Whahhuh?!” I hear you ask.

Here’s a prime example – say you’re aggregating on the field IP_addr all user values.

Your search might contain the following chunk: | stats values(user) as user by IP_addr. So for each unique IP address, you will collate a uniquified list of users. Maybe you have the following two IP addresses: & And you have the following user-IP address pairings: kingpin11, fergus97, gerfluggle, kingping11, jbobgorry

values will aggregate all of the following users associated with IP addresses: & gerfluggle, jbobgorry, kingping11; & fergus97.

That’s nice – it’s pretty.

But it exports in lousy form if you need to further process the data in another tool (eg Microsoft Excel).

When Splunk exports those results in a CSV, instead of getting a nice, processable file, you get tabs separating what would otherwise be individual items that have all been grouped into one field.

Enter list.

list doesn’t uniquify the values given to it, so while you still only get one line per IP address (since that was our by clause in the snippet above), you get as many IP addresses listed as there are users (in this example).

This makes for an exportable, more processable set of results that a tool like Excel can ingest to perform further analysis with relatively little reformatting needed.

Come back tomorrow for how to get the export to work “out of the box”.

a fairly comprehensive squid configuration for proxying all the http things

After combing through the docs and several howtos on deploying the Squid proxy server – none of which really did everything I wanted, of course – I’ve finally gotten to the format below.

Installing Squid is easy-peasy – it’s in the standard package repos for the major platforms (CentOS/Fedora/RHEL, Ubuntu/Debian, etc) – so just run yum install squid or apt install squid on your platform of choice (my exact install command on Ubuntu 18.04 was apt -y install squid net-tools apache2-utils).

What I wanted was an “open” (password-protected) proxy server with disk-based caching enabled that would cover all of the ports I could reasonably expect to run into.

Why “open”? Because I want to be able to turn it on and off on various mobile devices which may (or may not) have stable-ish public IPs.

Here is the config as I have it deployed, minus sensitive/site-specific items (usernames, passwords, port, etc), of course:

A working /etc/squid/squid.conf

acl SSL_ports port 443
acl SSL_ports port 8443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 777		# multiling http
acl Safe_ports port 8080

auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/.htpasswd
auth_param basic children 15
# after "realm", put some descriptive, clever, or otherwise-identifying string that will appear when you login
auth_param basic realm Insert Incredibly Witty Title Here
auth_param basic credentialsttl 5 hours
acl password proxy_auth REQUIRED
http_access allow password

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

#http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
# commented-out to allow "open" use (ie password authenticated)
#http_access deny all

# Squid normally listens to port 3128
# change this line if you want it to listen on something other port
http_port 3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256
# format is      
cache_dir ufs /tmp/squid-cache 768 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .		0	20%	4320

via off
forwarded_for off

request_header_access Allow allow all 
request_header_access Authorization allow all 
request_header_access WWW-Authenticate allow all 
request_header_access Proxy-Authorization allow all 
request_header_access Proxy-Authenticate allow all 
request_header_access Cache-Control allow all 
request_header_access Content-Encoding allow all 
request_header_access Content-Length allow all 
request_header_access Content-Type allow all 
request_header_access Date allow all 
request_header_access Expires allow all 
request_header_access Host allow all 
request_header_access If-Modified-Since allow all 
request_header_access Last-Modified allow all 
request_header_access Location allow all 
request_header_access Pragma allow all 
request_header_access Accept allow all 
request_header_access Accept-Charset allow all 
request_header_access Accept-Encoding allow all 
request_header_access Accept-Language allow all 
request_header_access Content-Language allow all 
request_header_access Mime-Version allow all 
request_header_access Retry-After allow all 
request_header_access Title allow all 
request_header_access Connection allow all 
request_header_access Proxy-Connection allow all 
request_header_access User-Agent allow all 
request_header_access Cookie allow all 
request_header_access All deny all

Finalize your Squid server system settings

Things you need to do once you do the above (prepend sudo to each command below if youre not logged-in as root:

  1. Enable Squid to start at boot: systemctl enable squid
  2. Create the cache directories: squid -z
  3. Create a DNS entry for your proxy host (if you want it usable outside your home network, and don’t want to reference it by IP address only)
  4. Create the authentication file (located at /etc/squid/.htpasswd in this example): touch /etc/squid/.htpasswd
  5. Create a username and password: htpasswd -c /etc/squid/.htpasswd (don’t forget this username/password combination!)
  6. Start Squid: systemctl start squid

Configure your browser to use your new proxy

Here’s where you need to go and what you need to change in Firefox:

  1. Navigate to about:preferences
  2. Click on Settings… under Network Proxy
  3. Enter your proxy host details:

To verify your proxy settings are correct, visit with both the proxy off, and then again with it on.

If your reported IP address changes between visits (with the second check being your Squid server IP) – congratulations! You have successfully deployed a Squid proxy caching server.