Had a Splunk use-case present itself today on needing to determine if the value of a field was found in another – specifically, it’s about deciding if a lookup table’s category name for a network endpoint is “the same” as the dest_category assigned by a Forescout CounterACT appliance.

We have “customer validated” (and we all know how reliable that kind of data can be… (the customer is always wrong) names for network endpoints.

These should be “identical” to the dest_category field assigned by CounterACT … but, as we all know, “should” is a funny word.

What I tried (that does not work) was to get like() to work:

| eval similar=if(like(A,'%B%') OR like(B,'%A%'), "yes", "no")

I tried a slew of variations around the theme of trying to get the value of the field to be in the match portion of the like().

What I ended-up doing (that does work) is this:

| eval similar=if((match(A,B) OR match(B,A)), "yes", "no")

That uses the value of the second field listed to be the regular expression clause of the match() function.

Things you should do ahead of time:

• match case between the fields (I did upper() .. lower() would work as well)
• remove “unnecessary” characters – in my case, I yoinked all non-word characters with this replace() eval: | eval A=upper(replace(A,"\W",""))
• know that there are limitations to this comparison method
  • “BOB” will ‘similar’ match to “BO”, but not “B OB” (hence removing non-word characters before the match())
  • “BOB” is not ‘similar’ to “ROB” – even though, in the vernacular, both might be an acceptible shortening of “ROBERT”
• if you need more complex ‘similar’ matching, checkout the JellyFisher add-on on Splunkbase
  • it supports Soundex, Levenshtein distance, and a variety of other comparison functions

Thanks, also, to @trex and @The_Tick on the Splunk Usergroups Slack #search-help channel for working me towards a solution (even though what they suggested was not the direction I ended up going).

I made a mistake 4 years ago.

I said vampires and zombies couldn’t [long] coexist. Because they’d be competing for the same – dwindling – food source: the living (vs them both being undead).

But I was wrong.

If the universe in which they exist is a mash-up of that of Twilight and iZombie … it could work.

The iZombie universe has zombies that can avoid going “full Romero” by maintaining a steady supply of brains – and it’s not much they need to eat to stay “normal”.

The Twilight universe has vampires that can survive on animal blood (or, one presumes, by hitting-up blood banks).

So if you were to have “brain banks” the way you have “blood banks” – I could see it working.

Now we just need some iZombie-Twilight hybrid vambie/zompire creatures running around.

I recently had cause to do an extensive trellised timechart for a dashboard at $CUSTOMER in Splunk.

They have a couple hundred locations reporting networked devices.

I needed to report on how many devices they’ve reported every day over the last 90 days (I would have liked to go back further…but retention is only 90 days on this data).

My initial instinct was to do this:

Except…that takes well over an hour to run – so the job gets terminated at ~60 minutes.

What possible other approaches could be made?

🤔

Well.

Here are a few that I thought about:

1. Use multisearch, and group 9 10d searches together.
   • I’ve done things like this before with good success. But it’s … ugly. Very, very ugly.
   • You can almost always accomplish what you want via stats, too – but it can be tricky.
2. Pre-populate a lookup table with older data (a la option 1 above, but done “by hand”), and then just append “more recent” data onto the table in the future.
   • This would give the advantage of getting a longer history going forward
   • Ensuring “cleanliness” of the table would require some maintenance scheduled searches/reports … but it’s doable
3. Something else … that “happens” to work like a timechart – but runs in an acceptable time frame.
4. Try binning _time
   1. Tried – didn’t work 🤨

So what did I do?

I asked for ideas.

If you’re regularly (or irregularly) using Splunk, you should join the Splunk Usergroups Slack.

Go join it now, if you’re not on it already.

Don’t worry – this blog post will be here when you get back.

You’ve joined? Good good. Look me up – I’m @Warren Myers. And I love to help when I can 🤠.

I asked in #search-help.

And within a couple minutes, had some ideas from somebody to use the “hidden field” date_day and do a | stats dc(ip) by date_day site. Unfortunately, this data source is JSON that comes-in via the HEC.

Poo.

Lo and behold!

I can “fake” date_day by using strftime!

Specifically, here’s the eval command:

| eval date=strftime(_time,"%Y-%m-%d")

This converts from the hidden _time field (in Unix epoch format) to yyyy-mm-dd.

This is the 🔑!

What does this line do? It lets me stats-out by day and site (just like timechart does … but it runs way faster (Why? I Don’t Know. He’s on third. And I Don’t Give a Darn! (Oh! That’s our shortstop!)).

How much faster?

At least twice as fast! It takes ~2200 seconds to complete, but given that the timechart form was being nuked at 3600 seconds, and it was only about 70% done … this is better!

The final form for the search:

index=ndx sourcetype=srctp site=* ip=* earliest=-90d@ latest=-1d@
| table site ip _time
| eval date=strftime(_time,"%Y-%m-%d")
| stats dc(ip) as inventory by date site

I’ve got this in a daily-scheduled Report that I then draw-into Dashboard(s) as needed (no point in running more often, since it’s summary data that only “changes” (at most) once a day).

Hope this helps somebody – please leave a comment if it helps you!

I omitted something kinda big when I wrote my one-time UBI proposal last year.

I neglected to address welfare reform.

Welfare would have to be changed for UBI to even have a half a prayer of working.

The “easy” way to do this would be to phase-in reduced welfare benefits on a prorated-equivalent basis for the UBI payment you receive.

Surely there are many other ways to address welfare as part of the one-time universal basic income – suggest them below!

Do I have to participate?

And I missed a second point, too – this should be something you can opt-out of. Just like I wrote about Social Security lo those many moons ago.

No one should be forced to participate – though I strongly suspect most people would rather participate than not.

What about when the program starts?

A third missed point in last year’s thought experiment – a prorated one-time UBI for every citizen over 18 when the program starts. Take the average life expectancy of a USian of, say, 75 years. Subtract 18 to get 57 – there is your basis “100%” one-time payment.

There also needs to be a phase-out cap on one-time benefits at age 74 (ie, when you turn 75, you are no longer eligible to receive a payout).

Now take your age, subtract 18, and divide by 57, and subtract from 100% to get your prorated payment. Are you 27? (27-18)/57 = ~15.8%. 100%-15.8% = 84.2%.

84.2% of $100,000 is $84,200.

Same process if you’re 50: (50-18)/57 = ~56.1%. 100%-56.1% = 43.9%.

43.9% of $100,000 is $43,900.

What if you’re 80? Congratulations! You’ve outlived the average American!

What we have now is a completely neurotic population obsessed with security and safety and crime and drugs and cleanliness and hygiene and germs… there’s another thing… germs.

Where did this sudden fear of germs come from in this country? Have you noticed this? The media, constantly running stories about all the latest infections – salmonella, e-coli, hanta virus, bird flu – and Americans, they panic easily so now everybody’s running around, scrubbing this and spraying that and overcooking their food and repeatedly washing their hands, trying to avoid all contact with germs. It’s ridiculous and it goes to ridiculous lengths. In prisons, before they give you a lethal injection, they swab your arm with alcohol! It’s true! Yeah! Well, they don’t want you to get an infection! And you could see their point; wouldn’t want some guy to go to hell and be sick! It would take a lot of the sportsmanship out of the whole execution. Fear of germs… why these fucking pussies! You can’t even get a decent hamburger anymore! They cook the shit out of everything now cause everybody’s afraid of food poisoning! Hey, where’s your sense of adventure? Take a fucking chance will you? You know how many people die in this country from food poisoning every year? 9000… that’s all; it’s a minor risk! Take a fucking chance… bunch of goddamn pussies! Besides, what do you think you have an immune system for? It’s for killing germs! But it needs practice… it needs germs to practice on. So listen! If you kill all the germs around you, and live a completely sterile life, then when germs do come along, you’re not gonna be prepared. And never mind ordinary germs, what are you gonna do when some super virus comes along that turns your vital organs into liquid shit? I’ll tell you what you’re gonna do… you’re gonna get sick, you’re gonna die, and you’re gonna deserve it cause you’re fucking weak and you got a fucking weak immune system!

Let me tell you a true story about immunization okay?

When I was a little boy in New York City in the 1940s, we swam in the Hudson River and it was filled with raw sewage okay? We swam in raw sewage! You know… to cool off! And at that time, the big fear was polio; thousands of kids died from polio every year but you know something? In my neighbourhood, no one ever got polio! No one! Ever! You know why? Cause we swam in raw sewage! It strengthened our immune systems! The polio never had a prayer; we were tempered in raw shit! So personally, I never take any special precautions against germs. I don’t shy away from people that sneeze and cough, I don’t wipe off the telephone, I don’t cover the toilet seat, and if I drop food on the floor, I pick it up and eat it! Yes I do. Even if I’m at a sidewalk café! In Calcutta! The poor section! On New Year’s morning during a soccer riot! And you know something? In spite of all that so-called risky behaviour, I never get infections, I don’t get them, I don’t get colds, I don’t get flu, I don’t get headaches, I don’t get upset stomach, you know why? Cause I got a good strong immune system and it gets a lot of practice. My immune system is equipped with the biological equivalent of fully automatic military assault rifles with night vision and laser scopes, and we have recently acquired phosphorous grenades, cluster bombs, and anti-personnel fragmentation mines.

So when my white blood cells are on patrol recon ordering my blood stream seeking out strangers and other undesirables, if they see any, ANY suspicious looking germs of any kind, they don’t fuck around!
They whip out their weapons; they wax the motherfucker and deposit the unlucky fellow directly into my colon! Into my colon! There’s no nonsense, there’s no Miranda warning, there’s none of that “three strikes and you’re out” shit, first defense, BAM… into the colon you go! And speaking of my colon, I want you to know I don’t automatically wash my hands every time I go to the bathroom okay? Can you deal with that? Sometimes I do, sometimes I don’t. You know when I wash my hands? When I shit on them! That’s the only time. And you know how often that happens? Tops, TOPS, 2-3 times a week tops! Maybe a little more frequently over the holidays, you know what I mean? And I’ll tell you something else my well-scrubbed friends… you don’t need to always need to shower every day, did you know that? It’s overkill, unless you work out or work outdoors, or for some reason come in intimate contact with huge amounts of filth and garbage every day, you don’t always need to shower. All you really need to do is to wash the four key areas; armpits, asshole, crotch, and teeth. Got that? Armpits, asshole, crotch, and teeth. In fact, you can save yourself a whole lot of time if you simply use the same brush on all four areas!

https://www.youtube.com/watch?v=X29lF43mUlo https://www.lingq.com/sv/lesson/george-carlin-fear-of-germs-235986

4.5 years ago, I switched hosts (though not providers (that had happened 4.5 years prior to that)) for my “big” server.

Now I’m about to do it again.

After years of thinking about it, I finally got around to it.

I’ve rewritten my RSS feed driven website https://datente.com to run on Python from PHP.

I’m sure there is much room for improvement in the approach – and would appreciate any constructive feedback you may have. Here’s the GitHub repo: https://github.com/volcimaster/dorss.