HashiCorp Field CTO Weekly: YACP (Yet Another ChatGPT Post), Amazonian Architectures, So Long and Thanks For All The Fishing Poles
Volume 112
(image courtesy of Love, Death And Robots on Netflix. I consider it to be some of the best short animation created, not safe for kids (mostly))
I, For One, Welcome Our Robot Overlords
Unless you have been locked up, on a rock climbing trip or just generally checked out of the world, you have probably heard about ChatGPT. There are a large amount of folks, many, not even in tech, talking about ChatGPT (ahem, James).
It’s so popular, you can’t even get in (when I started writing this anyway). This reminds me of the reason I even do what I do for a living. Back in college, I use to hang out on this BBS called ISCABBS (Iowa Student Computing Association BBS) run out of the University of Iowa. At one point, the queue for ISCA would be 1000 people (or more) and you could wait in that queue for hours. But you could shortcut the queue if you had your own client, often times taking only 2-20 minutes to connect versus hours. And that client only ran on Unix systems. So, I got a Unix account by borrowing one from a friend at the University of Minnesota (instead of just asking my school…I know), compiled the client in her account and began bypassing the queue and also learning small bits of automation. From there I got my first job out of college as a Solaris Admin, and I’ve been doing something related ever since. It was that need to skip the lines to goof off that lead me to my current career, and I’m wondering if my desire for knowledge of machine learning will be furthered by my need to skip the queue and ask ChatGPT things like “What is the answer to life, the universe and everything”. Stand by for more information (unless you’ve found the shortcut, then just contact me).
My last job was in Ad Tech, and what’s interesting to me is that machine learning has been figuring out what kind of shirt I should buy or listening for whether or not I want to go to the Great Wolf Lodge (on a single friend’s phone mind you), or helping find swing voters in swing counties for over a decade now. But it wasn’t until the rest of us could ask an AI for practical information and actually get back a plausible response, that folks really started paying attention.
Immediately, everyone feared for their jobs, or wondered if AI could be trusted. ChatGPT can code, it can suggest architectures, it can write stories (or college essays), it can give information to pass to other AIs like image generator Stable Diffusion or DALL-E, or music generators such as Dance Diffusion or Jukebox.
And then there’s this gem where it’s rendering Terraform Code:
Heck, even HashiCorp’s CEO, Dave McJannet, mentioned ChatGPT several times today in a recent company All-Hands meeting. These are truly interesting times.
Personally, I’m in the same camp as someone like Andrej Karpathy who talks about the impact of AI on human work on Lex Fridman's Podcast. The tools we build don’t replace us, they augment us. But more importantly in the case of AI, the models need data in order to function properly. Human input is important to creating human interfaces. Sure, we may be inching ourselves closer and closer to Skynet, but I’m more apt to think the ending will be more like that dreamt of in the movie Her (which is another of my all-time favorites sci-fi movies).
Design Your Way to Nirvana
To get to the point of AI domination or salvation is going to require a massive amount of compute and storage. The new workloads that we dream up for our AI compadres will very likely require some architecture changes and this reminds me of the recent release of Amazon’s Distributing Computing Manifesto. This document is highly fascinating to me because many of the patterns that were revolutionary at the time are not just commonplace today, but are also under their own updates with today’s advances in technology. There are a few key things that don’t really change though, and that’s the discipline of determining why to change in the first place and what to measure to ensure those changes are actually effective.
One quote stands out as a guidepost for others working on their own transformation:
The specification, design and development of services and their interfaces is not something that should occur in a haphazard fashion. It has to be carefully coordinated so that we do not end up with the same tangled proliferation we currently have.
A few weeks ago I referenced the The Duckbill Group’s article on how proper architecture is so critical to right-sizing cloud spend, but I think there’s a hidden edict here in that proper architectures are just more efficient in general. As technologies are always changing, finding the right components to create the most efficient architectures often means we’re having to do periodic technology reviews to see if the new shiny things actually make sense to integrate into our platforms.
I have a lot of customers whose first foray into the cloud is a Lift-n-Shift. And while some of you may scoff at such an atrocity (admit it, you’re probably scoffing because you also did this and learned something from it), I actually don’t think it’s that terrible of an idea. First of all, if you’re new to the cloud, it’s highly likely you’re comfortable with the way you’re doing things in your data centers and have the actual talent to operate your infrastructure in a like manner. Sure, you’re probably better off in the long run using more cloud efficient/native tooling, but if you go from VMs in your DC to Service Mesh’d Kubernetes across GCP and Azure, it’s highly likely more than a few heads are going to explode along the way.
You already have a baseline for what it costs to run your applications in the datacenter, and you can compare that with what it costs to run it in the cloud in the same fashion. It’s highly likely it’ll cost you more for similar performance unless you’re diligent about not over-provisioning too much. And as you begin to understand how to automate the building and updating of your infrastructure in the cloud (especially the observability bits), it’ll be far easier to find the bottlenecks and pick the right architecture to get around those bottlenecks much like Amazon did in building AWS in the first place. However, I think it’s prudent to start a pilot program in your cloud of choice, practice building automation (with Terraform, of course) and integrating it into your workflows and then “rapidly” (or more rapidly than you would have without automation) iterate on making improvements to work towards whatever goal you set when you decided to go to the cloud.
And this leads me to the core question I think everyone should be asking themselves when moving workloads to the cloud: Why am I doing this in the first place?
And now, because I actually have access to ChatGPT again (the “at capacity” image above was real), let’s ask it about cloud migrations.
Not to say that folks won’t realize point #1, but there’s a difference between price and cost. It’s likely the price you pay at first may be higher, but you may reduce cost in other dimensions like time-to-acquire infrastructure (immediate vs. months), or people-hours required to stand up the required infrastructure. If you’re disciplined enough to use right-sized resources and turn them off when not needed, you may also save in the price category.
It’s the last three answers that are more likely why folks go to the cloud in the first place. You may state your goals with objectives like:
We’d like to automatically scale our customer order infrastructure before we launch an ad campaign during the Super Bowl to allow for millions of request a second with zero downtime.
We’d like to be able to shift 5 critical workloads away from our primary datacenter in less than an hour and have them fully functional.
We’d like to be able to go from one software release a week to 2 releases per day with no downtime per normal release and less than 5 minutes when there are significant changes.
If you’re already operating in the cloud and have defined these kinds of objectives (and hopefully met them), what else can you improve? If you’re already in the cloud and haven’t defined these kinds of objectives, consider it. If you’re not in the cloud yet, please define some kind of metrics to improve before jumping in, and measure progress over time. Overall, there are stages to improving the architectures you build, but understanding what kind of problem you’re trying to solve with laser-like focus is imperative to your success.
One last thing I like to point out is that there is nothing wrong asking for help. There are numerous technology partners who’ve helped folks not only migrate their first workloads to the cloud, or augmented staff for process improvement projects or to create Terraform modules for a new offering, or to help integrate existing software with new service offerings. Often time these partners have a depth of experience you may not have in your company due to repetition alone. If you’re stuck improving your architecture to solve your problems, consider seeking out a partner to help you.
So Long, Farewell, Auf Wiedersehen, Goodbye
Our illustrious leader, the OG Field CTO at HashiCorp, OG FCTO Newsletter creator and contributor, and All Around Great Guy, Brent Holden has decided to leave HashiCorp to pursue another opportunity. I’ve probably, and embarrassingly, gushed about him in other public forums, so I’m going to have ChatGPT do all the rest of the work for me.
Aquaman Out.
P.S. Link Following is Low
While looking over the metrics for the newsletter, I noticed that not too many folks actually clicked the links in the blog posts. While it’s certainly acceptable to just consume the blog itself, the links give a bit more color to the conversation and are largely the inspiration for writing the blog in the first place (at least for me). I’m including the full links separately with a small explanation of what they’re about.
Splash page for the ChatGPT interface. You will need an OpenAI account to interact with ChatGPT
Information about the BBS which I learned a lot of things Internet and a major motivation for me working in IT.
https://en.wikipedia.org/wiki/ISCABBS
How to Save your Job From ChatGPT
https://www.bloomberg.com/opinion/articles/2022-12-10/how-to-save-your-job-from-chatgpt
Can you trust ChatGPT?
https://www.nytimes.com/2022/12/10/technology/ai-chat-bot-chatgpt.html
Stable Diffusion - AI image generation
https://stability.ai/blog/stable-diffusion-v2-release
Dall-E 2 - AI image generation
Dance Diffusion - AI music generation
OpenAI Jukebox - AI music generation
https://openai.com/blog/jukebox/
Lex Fridman’s interview with Andrej Karpathy, AI scientist who helped build Tesla’s self-driving capabilities
https://lexfridman.com/andrej-karpathy/
Amazon’s Distributed Computing Manifesto
https://www.allthingsdistributed.com/2022/11/amazon-1998-distributed-computing-manifesto.html
HashiCorp Field CTO Blog Vol 106: Winter is Coming
https://cloudedvision.substack.com/p/hashicorp-field-cto-weekly-winte