antipaucity

fighting the lack of good ideas

chelsea troy – designing a course

Via the rands-leadership Slack (in the #i-wrote-something channel), I found an article written on ChealseaTroy.com that was [the last?] in her series on course design.

While I found part 9 interesting, I was bummed there were no internal links to the other parts of the series (at least to previous parts (even if there may be future parts not linked in a given post)).

To rectify that for my 6 readers, and as a resource for myself, here is a table of contents for her series:
  1. What will students learn?
  2. How will the sessions go?
  3. What will we do in a session?
  4. Teaching methods for remoteness
  5. Why use group work?
  6. Dividing students into groups
  7. Planning collaborative activities
  8. Use of surveys
  9. Iterating on the course
She also has some other related, though not part of the “series”, posts I found interesting:
  1. Learning to teach a course
  2. Planning and surviving a 3-hour lecture
  3. Resources for programming instructors
  4. Syllabus design

If you notice future entries to this series (before I do), please comment below so I can add them 🤓

comparing unique anagrams?

How useful would determining similarity of words by their unique anagrams be? For example: “ROBERT” uniquely anagrams to “BEORT”; “BOBBY” and “BOOBY” both uniquely anagram to “BOY”.

Is there already a comparison algorithm that uses something like this?

What potentially “interesting” discoveries might be made about vocabularical choices if you analyzed text corpora with this method?

splunk: match a field’s value in another field

Had a Splunk use-case present itself today on needing to determine if the value of a field was found in another – specifically, it’s about deciding if a lookup table’s category name for a network endpoint is “the same” as the dest_category assigned by a Forescout CounterACT appliance.

We have “customer validated” (and we all know how reliable that kind of data can be… (the customer is always wrong) names for network endpoints.

These should be “identical” to the dest_category field assigned by CounterACT … but, as we all know, “should” is a funny word.

What I tried (that does not work) was to get like() to work:

| eval similar=if(like(A,'%B%') OR like(B,'%A%'), "yes", "no")

I tried a slew of variations around the theme of trying to get the value of the field to be in the match portion of the like().

What I ended-up doing (that does work) is this:

| eval similar=if((match(A,B) OR match(B,A)), "yes", "no")

That uses the value of the second field listed to be the regular expression clause of the match() function.

Things you should do ahead of time:

  • match case between the fields (I did upper() .. lower() would work as well)
  • remove “unnecessary” characters – in my case, I yoinked all non-word characters with this replace() eval: | eval A=upper(replace(A,"\W",""))
  • know that there are limitations to this comparison method
    • “BOB” will ‘similar’ match to “BO”, but not “B OB” (hence removing non-word characters before the match())
    • “BOB” is not ‘similar’ to “ROB” – even though, in the vernacular, both might be an acceptible shortening of “ROBERT”
  • if you need more complex ‘similar’ matching, checkout the JellyFisher add-on on Splunkbase

Thanks, also, to @trex and @The_Tick on the Splunk Usergroups Slack #search-help channel for working me towards a solution (even though what they suggested was not the direction I ended up going).

how-to timechart [possibly] better than timechart in splunk

I recently had cause to do an extensive trellised timechart for a dashboard at $CUSTOMER in Splunk.

They have a couple hundred locations reporting networked devices.

I needed to report on how many devices they’ve reported every day over the last 90 days (I would have liked to go back further…but retention is only 90 days on this data).

My initial instinct was to do this:

index=ndx sourcetype=srctp site=* ip=* earliest=-90d
| timechart limit=0 span=1d dc(ip) by site

Except…that takes well over an hour to run – so the job gets terminated at ~60 minutes.

What possible other approaches could be made?

🤔

Well.

Here are a few that I thought about:

  1. Use multisearch, and group 9 10d searches together.
    • I’ve done things like this before with good success. But it’s … ugly. Very, very ugly.
    • You can almost always accomplish what you want via stats, too – but it can be tricky.
  2. Pre-populate a lookup table with older data (a la option 1 above, but done “by hand”), and then just append “more recent” data onto the table in the future.
    • This would give the advantage of getting a longer history going forward
    • Ensuring “cleanliness” of the table would require some maintenance scheduled searches/reports … but it’s doable
  3. Something else … that “happens” to work like a timechart – but runs in an acceptable time frame.
  4. Try binning _time
    1. Tried – didn’t work 🤨

So what did I do?

I asked for ideas.

If you’re regularly (or irregularly) using Splunk, you should join the Splunk Usergroups Slack.

Go join it now, if you’re not on it already.

Don’t worry – this blog post will be here when you get back.

You’ve joined? Good good. Look me up – I’m @Warren Myers. And I love to help when I can 🤠.

I asked in #search-help.

And within a couple minutes, had some ideas from somebody to use the “hidden field” date_day and do a | stats dc(ip) by date_day site. Unfortunately, this data source is JSON that comes-in via the HEC.

Poo.

Lo and behold!

I can “fake” date_day by using strftime!

Specifically, here’s the eval command:

| eval date=strftime(_time,"%Y-%m-%d")

This converts from the hidden _time field (in Unix epoch format) to yyyy-mm-dd.

This is the 🔑!

What does this line do? It lets me stats-out by day and site (just like timechart does … but it runs way faster (Why? I Don’t Know. He’s on third. And I Don’t Give a Darn! (Oh! That’s our shortstop!)).

How much faster?

At least twice as fast! It takes ~2200 seconds to complete, but given that the timechart form was being nuked at 3600 seconds, and it was only about 70% done … this is better!

The final form for the search:

index=ndx sourcetype=srctp site=* ip=* earliest=-90d@ latest=-1d@
| table site ip _time
| eval date=strftime(_time,"%Y-%m-%d")
| stats dc(ip) as inventory by date site

I’ve got this in a daily-scheduled Report that I then draw-into Dashboard(s) as needed (no point in running more often, since it’s summary data that only “changes” (at most) once a day).

Hope this helps somebody – please leave a comment if it helps you!

dorss

After years of thinking about it, I finally got around to it.

I’ve rewritten my RSS feed driven website https://datente.com to run on Python from PHP.

I’m sure there is much room for improvement in the approach – and would appreciate any constructive feedback you may have. Here’s the GitHub repo: https://github.com/volcimaster/dorss.

sshuttle – a simple transparent proxy vpn over ssh

I found out about sshuttle from a random tweet that happened to catch my eye.

Here’s the skinny (from the readme):

  • Your client machine (or router) is Linux, FreeBSD, or MacOS.
  • You have access to a remote network via ssh.
  • You don’t necessarily have admin access on the remote network.
  • The remote network has no VPN, or only stupid/complex VPN protocols (IPsec, PPTP, etc). Or maybe you are the admin and you just got frustrated with the awful state of VPN tools.
  • You don’t want to create an ssh port forward for every single host/port on the remote network.
  • You hate openssh’s port forwarding because it’s randomly slow and/or stupid.
  • You can’t use openssh’s PermitTunnel feature because it’s disabled by default on openssh servers; plus it does TCP-over-TCP, which has terrible performance.

Here’s how I set it up on my Mac

Install homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Install sshuttle (as a regular user):

brew install sshuttle

Test the connection to a server you have:

sudo sshuttle -r <user>@host.tld -x host.tld 0/0 -vv

I also made sure that my target server could be connected-to via certificate for my local root user – but you can use a password if you prefer.

Check your IP address:

curl https://ipv4.cf

Once you make sure the connection works, Ctrl-C to end the session.

Then setup an alias in your shell’s .profile (for me, it’s .bash_profile):

alias vpn='sudo sshuttle -r <user>@domain.tld -x domain.tld 0/0'

Other things you can do

According to the full docs, there are a lot more things you can do with sshuttle – including running it on your router, thereby VPN’ing your whole LAN through an endpoint! You can also run it in server mode.

This is a super useful little utility!

basic dockerized jitsi deployment with an apache reverse proxy on centos

After a friend of mine told me he wanted to deploy Jitsi on my main webserver, and me saying “sure”, I decided I wanted to get it up and running on a new server both so I knew how to do it, and to avoid the latency issues of videoconferencing from central North America to Germany and back.

Before I go into how I got it working, let me say that the official Quick Start guide is good – but it doesn’t cover anything but itself.

Here’s the basic setup:

What To Do:

Once you have your new CentOS instance up and running (I used Vultr), here’s everything you need to install:

yum -y install epel-release && yum -y upgrade && yum -y install httpd docker docker-compose screen bind-utils certbot git haveged net-tools mod_ssl

I also installed a few other things, but that’s because I’m multi-purposing this server for Squid, and other things, too.

Enable Apache, firewalld, & Docker:

systemctl enable httpd && systemctl enable docker && systemctl enable firewalld

Now get your swap space setup:

fallocate -l 4G /swapfile && chmod 0600 /swapfile && mkswap /swapfile && swapon /swapfile

Add the following line to the bottom of your /etc/fstab:

/swapfile swap swap default 0 0

Restart your VPS:

shutdown -r now

Get your cert from Let’s Encrypt (make sure you’ve already setup appropriate CAA & A records for your domain and any subdomains you want to use):

certbot -t -n --agree-tos --keep --expand --standalone certonly --must-staple --rsa-key-size 4096 --preferred-challenges dns-01,http-01 -m <user>@<domain.tld> -d <jitsi.yourdomain.tld>

Create a root crontab entry to run certbot frequently (I do @weekly ~/renew-le.sh)

Go to the home directory of whatever user you plan to run Jitsi as:

su - <jitsi-user>

Begin the Quick Start directions:

  • git clone https://github.com/jitsi/docker-jitsi-meet && cd docker-jitsi-meet
  • mv env.example .env
  • Change the timezone in .env from Europe/Amsterdam if you want it to show up in a sane timezone (like Etc/UTC)
  • mkdir -p ~/.jitsi-meet-cfg/{web/letsencrypt,transcripts,prosody,jicofo,jvb}
  • docker-compose up -d

Now configure Apache for SSL. Start with this reference I posted.

But in the [sub]domain-specific conf file z-[sub]domain-tld.conf, add proxy and authentication lines (so that only people you allow to use your video conference can actually use it):

ProxyPreserveHost on
ProxyPass / http://localhost:8000/ nocanon
ProxyPassReverse / http://localhost:8000/
ProxyRequests       off
ServerAdmin warren@warrenmyers.com
AllowEncodedSlashes NoDecode
<Proxy http://localhost:8000/*>
    Order deny,allow
    Allow from all
    Authtype Basic
    Authname "Password Required"
    AuthUserFile /etc/httpd/.htpasswd
    Require valid-user
</Proxy>
RewriteEngine       on
RewriteRule        ^/meetwith/(.*)$ http://%{HTTP_HOST}/$1 [P]
ProxyPassReverseCookiePath /meetwith /

Reload your configs, and make sure they’re happy, fixing any errors that may exist:

apachectl graceful

Setup at least one user who’ll be able to access the site:

htpasswd -B -c /etc/httpd/.htpasswd <user>

You should also configure firewalld to allow only what you want (http, https, ssh):

firewall-cmd --zone=public --add-service=http && firewall-cmd --zone=public --add-service=https && firewall-cmd --zone=public --add-service=ssh

With any luck, when you now navigate to https://[sub.]domain.tld in your web browser, and enter your username and password you created with htpasswd, you’ll get the Jitsi welcome page!

Other Resources: