antipaucity

fighting the lack of good ideas

ron popeil never had it so good – lex>>fwd talk 04 sep 2014

Co-presented by Wes Eklund and myself

https://docs.google.com/presentation/d/1ureSEkjr9qd8k4Q2hY2ASLo-uIcfRe1L63IvT7eAiGM/edit?usp=sharing

a sample cfengine promise bundle – delete “large” files

body common control
{
        bundlesequence => { "file_remover" };
}

bundle agent file_remover
{
        vars:
                "bigfile" int => "1g";
        files:
                "/home"
                delete => tidy,
                file_select => files_too_big,
                depth_search => recurse("inf");
        reports:
                "Deleted files over $(bigfile) in size from /home";
}

body depth_search recurse(d)
{
        depth => "$(d)";
}

body delete tidy
{
        dirlinks => "delete";
        rmdirs => "false";
}

body file_select files_too_big
{
        search_size => irange("$(bigfile)","inf");
        file_result => "size";
}

what to automate

I have been in the world of automation for quite a while. Specifically in the realms of server, datacenter, and cloud automation – but I’ve been interested and/or involved in other tasks that tend towards automation (even for a short period of time) for far longer than just my post-college time in the world of HPSA and its related ilk.

One of the first questions customers ask us when we arrive onsite (heck – even way back in the technical presales cycle) is NOT what can be automated, but rather what should we automate and/or what can we automate first.

Analyzing the environment and finding some prime, low-hanging fruit to target in an initial automation push is vital.

To quote Donald Knuth, “We should forget about small efficiencies, say about 97% of the time; premature optimization is the root of all evil.”

In the realm of automating, that means picking on tasks that, while the tools at hand can make quick work of them, are done so infrequently as to not warrant an immediate focus, since the ROI on infrequently-done tasks is not going to be readily seen* should be skipped.

This is part of where being a good architect comes to bear.

No.

That’s not true.

This is where being a good listener and collator comes to bear. In a future post I’ll talk more fully about the art of architecting – but for today’s topic, let’s focus on the true key personality traits you must display to get a successful project started, implemented, and running.

You need to listen. You do not need to “hear” – you need to process what is being said, ask it back, take notes, ask for clarifications, etc. In the counseling world, this is called “active listening“. In the rest of life, it’s called being an attentive, thoughtful, caring, intelligent, adult human being.

When you hear a customer say they have a real problem with some task or other (beware – managementspeak coming!) – ie, they have “pain points” in various places, ask about what those individual tasks are actually comprised of. Investigate what can be touched today, what can be planned-for tomorrow, and what needs to be tabled for a future engagement (for you architects and sales folks reading, this translates into “what can we sell them later – after this project is successful?” – how can we build and strengthen this relationship?).

Take these notes and conversations you have to your colleagues and tease-out coherent lines of attack. Collate all the notes form everyone involved into commonalities – what has everyone heard a customer say? What did only one guy hear? What order did each person hear them in?

After you’ve listened, after you’ve taken your notes, after you’ve powwowed with your colleagues - then comes the fun part of any engagement: the actual automation!

Bring your cleaned-up and trimmed-down notes back to the customer in an easily-digestible form, and give a solid plan for what we will do now, what we want to do soon, and what really needs to wait to be done til later. Put an N, S, or L next to each item on your list. – it’s a first-cut priority draft. Then ask your customer for how they view those tasks, and listen to what they say are their priorities (including “real” dates, if any exist). You may need to reorganize your list, but keeping it involved in all project discussions will show you’re truly paying attention to them.

And at the end of the day, everyone’s favorite topic is themselves. Always – even shy people want to hear themselves bragged-up, talked-about, promoted, and given attention.

When you showcase your individual focus and attention on your customer, it will show in their willingness to accept you into their closer rings of trust – their readiness to receive you as a “trusted advisor”, which is what you want to be for them: you want to be who they can talk to about problems they’re seeing in their environment (current or potential) so you can bring your expertise to bear on their issues.

The role of any consultant who wants to be more than a mere grunt is not so much technical or business acumen, but that of their business therapist and/or best friend. You want to be able to say with Frasier Crane, “I’m listening”. And you want them to know that you really are.

Some of the early steps you can take today to bring yourself there are to:

  • avoid electronic distraction in meetings
  • document everything you do for work
  • be detailed
  • know industry trends, what competitors are doing, etc
  • treat everyone you come in contact with at a customer as if they were the most important person there
  • anticipate what you may be asked, and where you want to go
  • never speak authoritatively about that which you do not know
  • learn – be a “Lifelong Learner”: the day you stop learning is the day you stop growing, and the day you stop being reliable to others

*Unless, of course, those infrequent tasks are only infrequent because they’re “hard”, and therefore automating them will yield a solid ROI by allowing them to be done more often

programming your home by mike riley

Mike Riley’s entry in The Pragmatic Programmers series, Programming Your Home – automating with Arduino, Android, and your computer – was a lot of fun.

While I am not really in a position to do many of the mini projects given in the book (wrong type of house plus we rent), reading some of the project ideas did give me some inspiration for other activities. One of those is a Buffer-like tool I’m now writing to queue tweets over-and-above what the free level of Buffer will allow (and on a different schedule from my Buffer-fed queue). In conjunction with python-twitter, cron, and simple email messages, I’ve got a system started to which I can email things I would like to be posted, and they will go out when the cron job runs.

The Arduino is an impressive embedded platform – one that has also rekindled another long-time interest I’ve had in robotics. Years back, I recall seeing Sally Struthers advertising for one of those learn-at-home groups, and one of the options was robotics. (By “years back”, I mean 20+ years ago – probably more like 25 years ago, at this point.) I used to own a copy of Robot Builder’s Bonanza – and read it cover-to-cover a couple times. I loved watching Battlebots on TV. I’ve always wanted to buy/use LEGO Mindstorms.

Using robots to automate daily activities (and, of course, for fun) has been a fascination since I first saw Lost In Space and myriad other scifi shows and movies.

Riley does a great job of not demanding you be an expert programmer (or even a programmer at all) with the fully-implemented code examples in the book. He also does a good job of indicating what you’ll likely have to tweak on your own – and what you can probably just leave alone in the examples. Add to this the “extra credit challenges”, and I highly recommend it to anyone interested in home automation, embedded development, robotics, or just general programming/scripting.

There are some other interesting Python snippets throughout the book – that don’t have to be used in the context of an Arduino (like using Google’s SMTP server (via authentication)).

automation {gp}

The way people moved up the ladder in IT during my early days (starting in 1975) was to take on new projects that allowed them time to master the new software and become the local expert. As you became the local expert on many new software products, management became very comfortable giving you more and more of these new software projects. Now, at this critical time you needed to train your replacement for some of the software that you had become the local expert because you could not maintain forward momentum with the tremendous drag caused by being the local expert for so many software offerings. Of course, you would get management to agree to lighten your workload because your current workload didn’t allow you to work on their next “Pet” project (after all you are now their go to guy). Also, you would push to give up the software that was either too time consuming and/or not part of your future automation plans. If you didn’t give up something (holding onto knowledge is your power trip), you would fail to meet management’s expectations and someone else would get the new projects and your plans would stall (cut off from new software knowledge and isolated).

When you start to automate, you automate your existing manual processes usually by using a wrapper because it is the fastest way of showing progress. However, after demonstrating that you can automate longstanding processes, you now have the evidence to convince management to allow you to automate your next project (beginning to end) because you will become the local expert not only on the new software but also on the automation software. After successfully completing the automation beginning to end, you are in a position to push for the policy that states that all new software/applications will be automated as part of its installation/configuration steps. Now, you push for automation friendly software/applications because wrappers will no longer be acceptable automation options.

The philosophy of automation document was compiled during IBM user group meetings from around 1987 through 1989 when automation was a very hot topic.

The Philosophy of Automation

Automation is not a technical problem it is a people problem.

When you initially automate, you convert your process flow documents into executable code that consistently runs on a prearranged schedule, or through a monitor, or an error message. These executable processes enforce your process rules every time.

You cannot automate processes that are all over the place.

When you automate your processes they will be transformed.

Because you automate your processes automation never ends

Automate as close to the source as possible.

Problems with automated processes occur infrequently but are more difficult to solve than manual processes.

The combining of problem, change, and asset management with automated process management and root cause analysis, improves quality and allows you to consistently meet your service levels.

All automation code is throwaway code.

Automation is very exciting

Automation is very rewarding.

Everyone is on the automation team


Guest Post from one of my colleagues, Dave B

automating or automation?

I have been working in the realm of “automation” – specifically data center automation – for several years.

Merriam-Webster defines “automating” thusly:

  1. to operate by automation
  2. to convert to largely automatic operation <automate a process>

Notice the subtle difference with M-W’s definition of “automation“:

  1. the technique of making an apparatus, a process, or a system operate automatically
  2. the state of being operated automatically
  3. automatically controlled operation of an apparatus, process, or system by mechanical or electronic devices that take the place of human labor

These words tend to be used interchangeably – but they are different. Most of the customers I have worked with think they are “doing automation”, when in actuality they are only barely starting to “automate”.

What do I mean by this?

Most customers that bring automation tools in-house (whether simple cron jobs or complex tools like HPSA) take their current, manual processes and merely write wrappers around them to make them “automatic”. That is the first step of automation – but it’s only the first step.

Too many people try to take new tools and make them fit their current processes, procedures, and policies – rather than seeing what policies, procedures, and processes are either made redundant by the new tools, or can be improved, shortened, or – wait for it - automated!

Think of a physical example – you’re a Shaker Cabinetmaker in the late 1800s. You’re making end tables. Cutting dovetails with a dovetail saw. Sanding with a block and sandpaper. Cutting pieces to size by hand. Drilling mortises and cutting tenons with a manual auger and small saw. Lathing on a treadle-powered lathe.

Jump forward 100 years. You’re WGBH in Boston wanting to come up with a how-to program to air weekly. You find a fellow named Norm Abram, and pay him to do the show. You put New Yankee Workshop on TV for years. Norm visits places like Hancock Shaker Village for inspiration. But Norm doesn’t use loads of hand tools – he uses radial arm saws, drill presses, table saws, routers, joiners, etc. Why? Because he wants to make a prototype or three in a few days. And then wants to film a show in two (or occasionally longer for big projects) and be able to give you the confidence, along with a set of measured drawings, that you could, more-or-less, replicate what he did in your own home.

Which approach is better? Neither – they both yield end tables. Which approach is more repeatable – especially by someone with little experience? That’d be Norm’s method: the automated method. But the New Yankee Workshop is only slightly down the road of automating the entire process.

Automation (in the furniture world, at least) is found at Ikea. Thousands of identical Ingos and Karls roll out the door every year. Produced by automated machinery.

The job of the Information Technologist has a great deal of art to it – but it’s also a science: at the core of everything that a computer does is logic (perhaps poor logic, but logic nonetheless). Everyone in information technology (especially, though it’s applicable to myriad other industries) should be striving to make themselves replaceable – because no one is irreplaceable. I’ve seen it come true in scores of settings: the person who makes themselves “irreplaceable” never gets promoted, and is eventually replaced by someone else: either management removes him or finds a way around him, or he leaves the organization.

Therefore, preemptively make yourself replaceable. This was Ken Moellman’s campaign when he ran for State Treasurer in KY: to eliminate the very job he was running for.

There’s a secret to making yourself replaceable – and that is that when you can show that you’re replaceable, especially in the world I work in, you tend to be promoted. You also tend to grow because you’re learning more. Because you learn more and grow, you become more valuable. Because you become more valuable, you can move up the chain as you like.

Do artisan works still have a market? Of course – go look at any “artsy” type store that showcases “local craftsmen” in almost any part of the country: they’re offering their hard work for your consideration … at a hard price.

Is the artisan chair fundamentally any better than the Ikea chair? Maybe, maybe not. It sure looks better. But it’s not as repeatable.

And repeatable tasks get automated because if you have to do it twice, you need to write it down. And if you have to do it more than twice, you need a process anyone can follow. But processes are always open for refinement and replacement. The process to get a piece of lumber from a tree is not completely dissimilar to day from how we did it before the advent of the sawmill – but with laser-guided blades, the sawmill of 2013 can optimize lumber out of a log in a way very very few people ever could .. and can cut the log into its constituent boards faster than any person.

The process for manually provisioning a RHEL server is pretty simple – but shortly after introducing Anaconda, Red Hat introduced the kickstart (modeled after jumpstart). Microsoft, likewise, has unattend.txt and unattend.xml (for either the DOS or WinPE methods of installing). And SuSE, HPUX, and AIX have their systems (AutoYaST, Ignite, and NIM).

Why do these tools exist? It’s so administrators can rapidly deploy machines without having to do a lot of extra setup work by hand. The same can be asked for why do chainsaws, table saws, and circular saws exist? It’s so you can cut wood more rapidly and more repeatably than by hand. You can fell a tree with an axe. You can fell a tree with a saw. But for a single person, felling a tree with a chainsaw is best. Or you can use a Tiger Cat.

The first step of automating needs to be building wrappers to reliably repeat manual processes.

The second, and far more important, step in the paradigm shift from manual methods to automation, is to systematically go through all of your processes, procedures, and policies and see what can be refined, what can be replaced, and, most importantly, what can be removed.

What legacy activities are you doing that should be eliminated, updated, or cleaned-up?

delivering solutions – “shipping is a feature!”

Back in 2009, Joel Spolsky wrote an article called The Duct Tape Programmer. Of everything he has written, I think this is the very pinnacle, and it is summed in one simple sentence in the middle: “Shipping is a feature.”

I’ve referenced this article twice before (in Feb and Sep of ’11).

Why is this so important in my mind?

I went back to school in 2003 to complete my bachelor’s degree in CIS. I had graduated in 2001 with an AAS in CIS from HVCC, and after finding nothing in 2 years of searching better than the job I had at Hertz, I decided a 4 year degree might help. I graduated from Elon University in Dec 2006 with my newly-minted BA in CIS. During my tenure at school I discovered that I didn’t really like the development end of Computer Science, and I instead preferred the analytical and integrational aspects of systems work – tying disparate tools together, improving internal workflow, etc – to help make individuals’ lives better and easier. In other words, I enjoyed finding ways to automate time-consuming and repetitive tasks to allow myself (and others) to focus on more interesting work – like figuring out how to automate more tasks to move up the chain.

I worked for a few places while I was at school (two different departments at the school itself, a pair of non-profits, and some freelance side work doing web site development). When I graduated, therefore, it was only natural that I ended-up with a pair of offers to work with automation tools – one from a company called Opsware, and one from a place called Network General. For a variety of reasons, I chose Opsware.

It wasn’t long after I started in Support for Opsware’s Server Automation [System] product that I became more and more sold on the product, and grew bored doing support – troubleshooting is fun, but with the paucity of good support tickets*, large similarity of cases coming from customers, etc .. it just wasn’t “me”. Shortly after HP purchased Opsware I put in to move from Support to Professional Services – to, hopefully, get a chance to work with harder integrational problems than I would ever see helping people over the phone and via email.

Beginning March of 2008^ I moved from Support to ProServe, and did start to get a taste for the bigger systemic problems that could be solved with the Opsware HP BTO suite. While with HP, I had the opportunity to do the global delivery of HPSA 7.5 for HSBC – performing both installation and onsite mentoring/training in Chicago, NYC, London, and Hong Kong. I also did the replacement install of HPSA 7.0 (a non-upgrade-to release) for Home Depot in Atlanta to manage their 2200 stores. There were some other customers I worked with, too – but those were the two biggest.

One of the issues that has arisen with [nearly] every customer I have ever worked with is that they want what they’ve agreed to pay for in the Statement of Work (SOW) signed, sealed, and delivered by the end of the project – and if it’s not, they want good reasons to sign a CO (change order) to modify the SOW.

And it’s no surprise. When you cost someone nearly 7 cents per second to work for them, they want to see results!

One of the constraints, therefore, that needs to be constantly watched is scope creep – the insidious tendency for all projects to go beyond their intended purpose (violating law 47 of the 48 laws), exceed budget, and never deliver what is really needed.

My primary goal when I work with a customer is not, perhaps paradoxically, to “make them happy”. One thing I learned when working in support is that the customer is never right!. You may have to pretend that they’re kinda right – but they’re always wrong. They do not know what they want. They do not know what they need. And they certainly do not know what is wrong if you ask them.

My primary goal when I work with a customer is to deliver what they have paid for. When possible, I will change course slightly (following proper CO processes) – but I want them to get what they have agreed to pay for. Ideally, especially now that I am in the architecture end of the world much more than ‘just’ delivery, I can work with them in the pre-sales process to get the SOW to something that approximates what they need. But I always aim to give them what they have paid for. Everything else is window dressing.

At the end of a project – whether as outside consultants, students, internal employees, at home, for work, etc – what needs to be seen is what was paid for.

Ship. Deliver.

Without those two, nothing gets done.


* I have grown so frustrated with support processes that I spent time a couple years ago writing a small eBook that includes a section on how to make good tickets. I’ve also written on ways to improve your support organization before.
^ Just realized that means I’ve been doing ProServe or PS-like work for 5 years running, and have been with the automation suite for more than 6 now.
! Before you become too concerned – I do realize there are a few good customers out there. But they are just that – few, and VERY far between.