CloudSkills.fm

098: Infrastructure Development with Brandon Olin

Episode Summary

Mike’s guest today is Brandon Olin, an MVP in Cloud and Datacenter Management, Site Reliability Engineer at Stack Overflow, and author of Building PowerShell Modules.

Episode Notes

Learning new frameworks is always tough, but it’s incredibly important to build your foundation from the basics. Mike’s guest today is Brandon Olin, an MVP in Cloud and Datacenter Management, Site Reliability Engineer at Stack Overflow, and author of Building PowerShell Modules. Stack Overflow provides community-building and knowledge-management within its client companies through Azure technology. Brandon specializes in Infrastructure Development and is here to talk about different frameworks that he uses to do that!

In this episode, we talk about…

Resources from this episode:

Episode Transcription

Mike Pfeiffer:
Hey, what's up everybody? It's Mike Pfeiffer and you're listening to the Cloud Skills FM Podcast. All right everybody, welcome back to another episode of Cloud Skills FM. As usual, appreciate you tuning in. In today's episode, I've got Brandon Olin with me. He is a fellow MVP. He's a site reliability engineer over at Stack Overflow, and he's got a book out. It is Building PowerShell Modules, and that's out at Leanpub. Super excited to have Brandon on the show. Brandon, how you doing?

Brandon Olin:
Very good, Mike. How are you?

Mike Pfeiffer:
Doing good, man. It's been a crazy year, but I'm counting the days until January 1st.

Brandon Olin:
It has. I'm hoping for a better 2021.

Mike Pfeiffer:
Yep, me too. But how has your year been by the way? I know you worked on the book and stuff, but it must have been kind of crazy for you, just like the rest of us, working from home and dealing with this stuff. Have you been overloaded with work, because everything that's going on with COVID and the demand for tech resources?

Brandon Olin:
It's interesting. In my situation, I already worked from home prior to the pandemic, so I was used to that setup. My day-to-day workday did not change so much. What did change is personal life. Now the kids are at home and not in school, so dealing with that.

Mike Pfeiffer:
That had to be tough, man.

Brandon Olin:
That's an adjustment. Yeah. But so many people are dealing with that. Yeah. I mean, as far as work goes, I'm, I guess lucky to work for a company that is already remote friendly and works in a remote-first way, so it didn't really throw us off our track when everyone had to work from home, because a good majority of the company was already remote.

Mike Pfeiffer:
Right. Okay. That's awesome. Yeah, it seems like you guys must have quite a bit to do over there, because you're just supporting so many users on your site. What's it like to work behind the scenes there at Stack Overflow?

Brandon Olin:
It's interesting. Me specifically, like you said, I'm a site reliability engineer. I focus specifically on the enterprise product that Stack Overflow has. If people don't know, Stack Overflow offers... We have a product called Stack Overflow for Teams, which is kind of like a private version of Stack Overflow for business. We're Q&A and knowledge management and community building within your company. I support the enterprise product, which we host for companies in Azure. So I don't work on the public platform, is what we call it internally, but I'm on the larger team that does. I assist them where I can with that, but mostly, my focus is in Azure.

Mike Pfeiffer:
Got it. Okay. Yeah. That's been coming up a lot over the last year or two, right? People asking, "Well, what do SREs do in their job?" And stuff like that. And so of course, we know that you're big into automation. What's it like for you? Has it been a big departure from what you were doing in the past, in CIS admin type stuff and things like that?

Brandon Olin:
In some ways, it's very similar. SRE can mean many different things to different companies, just like DevOps does. For us specifically, an SRE, it's literally in the title. It's site reliability. Our job is to keep the site up and to apply automation and engineering principles to do that, to make that as reliable as we can. We also do just traditional CIS admin work, the stuff in the background that we maintain, like any other CIS admin would as well.

Mike Pfeiffer:
Right. And so I'm assuming, in terms of automation, to get things deployed and stuff, are you using PowerShell a lot or are you using other frameworks and tools?

Brandon Olin:
From the enterprise side, we use a lot of PowerShell and Terraform to build out our infrastructure. That's our familiar tool set that we have. But at the larger SRE team, we have stuff in Puppet, we have stuff in Python, we have stuff in Go and PowerShell. We use a lot of mix of the different languages and technologies.

Mike Pfeiffer:
Oh, that's interesting. Does that get kind of difficult to try to stay on top of all these different things, or you just kind of learn just enough to where you can be effective [inaudible 00:04:05] and then kind of have your bread and butter, which would be PowerShell and Terraform type thing?

Brandon Olin:
I think you learn enough to be effective. So I know Python, I'm not an expert in Python. I can read Go and can edit Go, but I'm not a Go expert by any means. And the same for other people. Other people really know Python and Go, they know a little bit of PowerShell, they can get by. But the nice thing is, we all kind of assist each other and support each other on whatever we're trying to do.

Mike Pfeiffer:
That kind of takes me into my next question, which super interesting, your book, because Building PowerShell Modules, it definitely takes you from just doing the basics to becoming a PowerShell developer, obviously. Can you talk about the book and then what people would expect to learn going through this? What do you take them through and what's the big idea?

Brandon Olin:
Yeah. So kind of the goal of the book was to show all the steps that someone would need or may want to do when they wanted to create a really high quality open source PowerShell module. And so I'm focusing less, so much on the actual PowerShell code. It's not like a how to do a PowerShell, how to code in PowerShell book. There are plenty of great books that already do that. This was more about, okay, I know some PowerShell, I want to create a module. What are the steps or the best practices with kind of the conventions out there in the community, to do that?

Brandon Olin:
And this includes, how do I lay out a PowerShell module, like file structure? How do I test it? How do I kind of create the developer inner loop with the build tasks and test tasks, and how do I document it? How do I put it on GitHub with a read me? And what does a license file do? What is GitHub actions? Those issue templates, there's all kinds of other things that are ancillary to PowerShell itself, but you would probably want, or may want to consider when you want to create a good open source project. So that's what it kind of lays out in detail.

Mike Pfeiffer:
This is perfect because... And by the way, this is not premeditated at all. I'm learning about this book for the first time on this episode, for anybody listening. But yesterday we were having a conversation inside our community about, how do you get some experience? And I'm always saying, "Well, go build an open source project." This is like the actual guide. If you have no idea what to do, maybe you know some outer shell, but this puts it into practice for you. I love that.

Brandon Olin:
Yeah. And I try not to be too opinionated in there. Well, in some cases I am, but everyone has their own opinions on, this is the way you should do it. But the great thing is, there are so many ways you can lay these things on. I try to lay it out where it's worth. These are your options and this is the pros and cons of each, or this is why you may want to consider using this. But there are valid reasons not to do that as well. But for someone new out there, there's just so much, they don't know where to start. So they need a guide to get them where they want to go, and then as they get more experienced, they can realize, okay, I actually want to do something differently now that I have a little more context about what I'm trying to do, and that's totally fine. But sometimes you need a little guide to get you there.

Mike Pfeiffer:
Yeah. I agree. Because to your point, you might make it halfway through, or even at the end, and be like, you know what? That was awesome, but now I'm going to go with this other direction. Or you might become obsessed, and then now you've got all this perspective to go build these cool creative solutions.

Brandon Olin:
Right.

Mike Pfeiffer:
So what's it like... Kind of take me through, for anybody listening that maybe has never done this before. Going to get a repo set up, I got my code in there. You mentioned GitHub actions. How does that fit into this, if I'm making changes to this code I'm working on?

Brandon Olin:
Yeah. So GitHub actions, if people don't know what it is, it's a version of CICD or continuous integration, continuous delivery platform. And it kind of revolves around running these various tasks in response to events that happen in GitHub. So imagine I pushed some code into GitHub, that's an event, I can trigger an action based on that. Or someone does a pull request because they want to make some changes to the code base, you can do another action that does something different on that. You can do really crazy things like run actions when someone posts an issue, and you can run some custom code that responds to that action.

Brandon Olin:
You can do a lot of different things with it. It's really very powerful. From a simple kind of build and test perspective, you want to... You have GitHub actions to build your PowerShell module. And I kind of go through in the book, what does actually building a PowerShell module look like, because you're not... I mean, if you're writing PowerShell, you're not compiling code like you would in C-Sharp, but there is a build process involved. There's a test process involved that you should do after you build things like that. And your Github actions can run automation to go perform those things.

Mike Pfeiffer:
That's very cool, because if you're an infrastructure person with not really a lot of perspective of software development, and you go through this, then you're going to kind of learn the patterns of any development team, how they integrate changes to code and get that to kick off a build process. Like you said, compiling C-Sharp and stuff and running tests, which is important. But what does the build process look like for a PowerShell module? You just copying files around or what's going on in the build?

Brandon Olin:
Usually you're copying files around, is kind of, at the end of the day, what could happen. Your build process, the way I look at it, is I've written some code, I want to go test it. So my build process would be taking all of those separate PowerShell files and combining them into an output PowerShell module. That's my finished artifact that I want to go test. And then running some pester tests against that PowerShell module to validate that it's correct, and does what I expect it to do. So that's your typical build and test loop. Well, you can extend that to where, I want to create some HTML or markdown documentation based on my comment based help in PowerShell. That's a part of it. That's another task as part of a build process, that you can insert into there. You can publish it to the gallery, or you can do lots of things like that.

Mike Pfeiffer:
Yeah. And so that's cool, because that actually answers one of the questions I was going to ask you, because you said the gallery there. So as you get that thing kind of built, you can publish it over PowerShell gallery and make it available to everybody. But you mentioned the testing, I'm curious about that because there's not a whole lot of guidance for infrastructure code testing templates. There's stuff out there, but it doesn't seem like it's really gone mainstream. What's the testing look like for this? You mentioned pester, how does that all go together?

Brandon Olin:
Are you talking about infrastructures code testing or for PowerShell testing, because those are-

Mike Pfeiffer:
Well, I guess in this case, PowerShell testing. I'm just thinking, as infrastructure concerned folks, we don't really have a ton of guidance around testing stuff. But in your framework here and the way you're doing things, what are you doing with pester? And what does that... What should people know about it if they're doing this stuff?

Brandon Olin:
Sure. So really, high level testing kind of consists of a couple of different categories. So you have unit tests, which test specific inputs and outputs to various functions. You have integration tests, which kind of test how a function interacts with maybe another function, or maybe interacts with the outside world, like you call in a REST API or something like that. And pester, you can use to kind of simulate calling those functions with various inputs and outputs. So if I have a function that adds two plus two, I can write a pester test to say, okay, I'm going to provide you with two plus two, I expect four back. If I get five back, that's a problem. And then you just kind of extend that metaphor out to every function. It's like, I give you some input, I expect this certain output. Pester is going to validate that for me.

Mike Pfeiffer:
Yeah. That's a lot of work, right? To write all the tests after you've written the code, but eventually you're going to get more quality out of the code you're writing, right? Isn't that one of the main benefits to this?

Brandon Olin:
Right. I mean, you're getting assurance that your functions work as expected. And as you modify them in the future, you can run your test and you don't have to do a lot of manual testing to validate all of your functionality, you can just run your test suite. If you've written it correctly, you can be pretty confident that if I make this change, I've run my test suite, everything else passes, I can release that.

Mike Pfeiffer:
That's cool, man. I might have to go look at your book, because it's awesome. Is pester available on a GitHub actions hosted runner or whatever, when you're doing a workflow? Or do you have to do some special stuff to get that thing prepped to do what you need to do?

Brandon Olin:
You can run a generic PowerShell task and then run pester. Pester is actually installed in GitHub action, the runners themselves, by default. The nice thing is, GitHub actions have a lot of common tools already installed. They have PowerShell, they're going to have C-sharp, they're going to have [Known 00:12:53] they're going to have Terraform. They're going to have lots of tools that are common to developers, already on those runners, and pester is one of them that's already on there. But you could of course, run a PowerShell script to go get it. Some people have created their own actions. That's another kind of cool thing about actions, is that you can create them and publish them as Docker containers. And I've written a couple of those as well. I'm sure there's one specific to pester, where you could say, just go download this pester Docker container and run it from there. So you have a certain set of dependencies.

Mike Pfeiffer:
Yeah. Right. So you could bake in your own custom stuff if you need it, and it's not going to normally be there.

Brandon Olin:
Right.

Mike Pfeiffer:
It makes a lot of sense. That's cool, man. Really cool stuff. So let's switch gears just a little bit. Before we hit record, you were talking about, you're doing a bunch of stuff with Terraform and testing Terraform code. What's that all about? What are you doing with Terraform? And let's talk about the testing process for that.

Brandon Olin:
Sure. So just like any other software development, when you're doing the infrastructures code, you are developing and you should probably follow the same kind of developer patterns and workflows. So yes, I am creating VMs and load balancers and databases in Azure, not C-Sharp function methods and PowerShell functions and things like that, but I do want to test that. If I'm writing Terraform, as an example, to deploy all this infrastructure, I should have tests to make sure that once that infrastructure is deployed, it is what I actually expected. Maybe I fat-fingered some configuration or a name [inaudible 00:14:26] configuration. I want my test to validate that, what I build is actually what I intended. So testing the infrastructures code is, in a lot of ways, very similar to testing standard software, but there are some nuances with it because you're building infrastructure. Now you're not testing just [inaudible 00:14:43] in isolation on your machine. You are interacting with a third party in AWS and then Azure or things like that.

Mike Pfeiffer:
Got it. Okay. And has that been a pretty straightforward process or has it been a lot of trial and error? What's been your experience so far? Is it pretty straightforward in terms of Terraform? Because Terraform configurations are pretty easy to read, in my opinion, once you know the basic framework. Is that kind of the same way with the testing as well?

Brandon Olin:
I think the testing, once you see it, you get it. But if you haven't seen it before, it may be kind of strange to how you would test with this stuff. But when you actually look at it and you're like, oh, this is actually a pretty simple concept to look at. So I run Terraform, I build some things, my pester suite would go query Azure for that infrastructure and validate that it was what I expected. And doing the same kind of assertions that you would do in pester with standard PowerShell, you do that with Terraform. But I think, if you've never done it before, it may be a little strange concept. But I think once you see it, you're like, okay, this clicks pretty quickly.

Mike Pfeiffer:
Right. And is that stuff, mainly you have been doing at work, and not really so much outside of the technical community? Because you're writing a book, right? But is that stuff at work that you've been doing?

Brandon Olin:
I do a lot of that at work.

Mike Pfeiffer:
Okay.

Brandon Olin:
So we have some automated... And just like with standard software development, you write Terraform modules and reusable components, you run them through their own test suite and their own release process. That's where the testing comes into play. But I've also been involved with the PowerShell, the latest PowerShell conference book, volume three, that came out a month or two ago. And my chapter was specifically about testing Terraform with pester. And if people don't know, the PowerShell conference book is a community kind of project, where a bunch of people go and contribute a chapter, as if they were going to do a conference talk. It's kind of built as like a conference in a book type format. So you have many different speakers who would write a chapter about something PowerShell related, and all the proceeds of that go to the onboarding scholarship program from DevOps stuff, devopscollective.org, to get people into the PowerShell conference.

Mike Pfeiffer:
That's fantastic. Yeah. We'll link that up in the show notes so people can check that out. That sounds cool. Now going back to PowerShell. My question for you is, so I need you to educate me a little bit. I have, honestly, not done a ton of PowerShell stuff in the last couple of years, beyond just leaning on my original experience from 2010, 2011, 2012 type era. So what was it like for you kind of transitioning to PowerShell core? Was there any big learning curves there? And I'd kind of also want to follow up on that and say, for somebody getting started, what would you recommend in terms of starting from scratch? Because we got a lot of people that are listening to the show, learning cloud, but may not be a PowerShell scripter at this point.

Brandon Olin:
So I've been using PowerShell, I guess, since 2008. Been a while. Not since the very beginning, but just after that, I think when-

Mike Pfeiffer:
Right after the monad days?

Brandon Olin:
Yeah. Yeah, right after the monad days. So when I transitioned to PowerShell core, I did it by... Because I switched to a Mac as my kind of personal machine at home, and I started... And you had to use PowerShell core at that point. And really kind of fumbling over kind of the growing pains of writing something in a cross-platform way, because as soon as you do that, you run into all kinds of assumptions that you made in your PowerShell code about, there should be a C drive. There's no C drive on Mac. The slashes were different. Sometimes there's case sensitivity issues and things like that. So those are the really common things that you would run into when you try to start something. [inaudible 00:18:35] assumes a certain environment, try to make it cross-platform. So that's writing kind of my own personal modules, my open source modules, that's the kind of stuff I ran into. Now I run PowerShell core exclusively on Windows and Mac.

Mike Pfeiffer:
That's got to be great, right? Because you get a real consistent experience at this point, they've done so much work. The tooling is good, experience is pretty solid between the platforms, right?

Brandon Olin:
Yeah. It's getting a lot better, especially the... When PowerShell core first came out, there was lack of support for a lot of standard PowerShell modules that administrators would use. It would only run on Windows, they weren't available on Mac or on PowerShell core. That's getting better all the time.

Mike Pfeiffer:
For the folks getting started, maybe from scratch... It's easy for us because we've been paying attention to this for so long, but is it just a matter of going back to the foundations, kind of learn PowerShell in a Month of Lunches, that kind of stuff?

Brandon Olin:
Yeah. PowerShell in a Month of Lunches is a really good starting point, I think. It helps to have a task that you need to accomplish. It depends on how you personally learn something. I like to learn... I need to solve a problem. I have some task to accomplish, I want to use a new tool and new technology to do it. It's very hard for me personally, just to learn a technology with no end goal in mind.

Mike Pfeiffer:
I'm the same way, man. That's why I never did well in school, and I always thought that I was the dumb kid. And then I realized, I just need to be self-directed learning. I have to take on my own challenges, and then I'm okay. So sometimes, sitting down to just read a book from beginning to end, I'm like, all right, what's the point of what I'm trying to do here?

Brandon Olin:
Well, that's a good motivator. It's like, I have a problem or an itch to scratch. This is how I'm going to resolve that.

Mike Pfeiffer:
Yep. Yeah, I'm the same way. I've learned a lot just by doing that. I think people listening should double down on that because a lot of people right now are looking for, what's my next move? Looking for the next opportunity. That might be a way. Takes something you're trying to solve, a problem, see if you could do a PowerShell.

Brandon Olin:
Yeah. I mean you can take it like, if I'm a person who's in a traditional data center environment and looking to figure out, what does this Power thing look like? Come up with a scenario, or even it could be a contrived example. I have this web app that's running on-prem, I need to move it to Azure. What are the tools that would allow me to do that? And that's my problem that I need to go solve.

Mike Pfeiffer:
Okay. Awesome. The other thing I was curious about, that you might be able to educate me on is, what's going on with PowerShell DSC? Is that still a thing that we should care about and is it still being worked on and what's status?

Brandon Olin:
PowerShell DSC is still being worked on. I personally don't do a whole lot with it in my job, but it's still alive and kicking. I have been, actually, using Azure automation in combination with DSC, to do some things. And even doing some kind of crazy things like running DSC on Linux to solve, particularly, a very specific problem that I have. But DSC is still a great technology that works. And the nice thing about it, is that it works with your larger configuration management tools, like Puppet and Chef. And historically, that's how I used DSC. I never used DSC as a standalone thing in my job. I used it as a tool to use another tool, like Chef. And I found it worked really nicely.

Mike Pfeiffer:
Yep. Yep. I mean, I've used it in the past in real projects. I've just used it as a bootstrapping solution to get servers configured the right way, with the first time they come up. But I do like that, what you said of, if you do have to manage traditional machines and you want to have a configuration management solution in Azure, that Azure automation is actually pretty cool because you can centrally, basically manage on-prem, cloud-based VMs and it's all based on DSC, right?

Brandon Olin:
Right. Right.

Mike Pfeiffer:
And yeah, I guess that was the question I was wondering, is are they still working on resources, customer resources and stuff? So it sounds like they are, and it sounds like, in PowerShell core, we could still use it.

Brandon Olin:
We can. And a lot of the... There's a lot of good community engagement on a lot of open source, PowerShell DSC resources. I don't know if Microsoft, the company, is making a lot of DSC resources, but the community is making a ton of them.

Mike Pfeiffer:
Right. Okay. Cool, man. So what's your game plan for 2021? You eyeballing any new languages that you're going to learn or anything new?

Brandon Olin:
Oh, let's see. I want to increase my skills in Go, I think is what is kind of a personal mission of mine. I mean, once you learn a language, it's pretty easy to start reading other languages. You can kind of... You can rock it and you can understand what the syntax is and what the logic is. But actually writing something from scratch, is a different thing. And that's what I want to try to get into, is really kind of getting into writing a tool in Go.

Mike Pfeiffer:
And then, because then you can start customizing Terraform and Kubernetes, because Go is everywhere at this point, and it's insanely fast.

Brandon Olin:
It's insanely fast, it's how Terraform resource providers are created. So if you ever want to tinker with the Azure provider, you need to learn some Go.

Mike Pfeiffer:
Right.

Brandon Olin:
That's kind of my motivator.

Mike Pfeiffer:
Yeah, exactly. Learn enough Go, so you can get in there and customize some solutions, solve some problems.

Brandon Olin:
Right.

Mike Pfeiffer:
Awesome. Cool, man. So where should we send everybody listening to this, to follow you, find your book and all this other stuff you're working on?

Brandon Olin:
Sure. So if you want to follow me online, you can find me at devblackops on Twitter. I blog occasionally, not so much anymore, but at devblackops.io is my blog. And you can find the Leanpub book, Building PowerShell Modules... Just Google Leanpub and Building PowerShell Modules, and it will get you right there.

Mike Pfeiffer:
Awesome. So if you guys are listening, go out to the show notes for this episode, go visit Brandon's site and get his book, follow him on Twitter, all that kind of stuff. Brandon, I got one last question for you. What's the best career advice you've ever gotten?

Brandon Olin:
Oh, best career advice. Don't be discouraged. You're going to run into problems left and right, and you don't know what they're going to be. Just accept that they're going to happen and roll with the punches.

Mike Pfeiffer:
I love it, man. So become process-driven and know that it's just not going to work out every time, and just keep showing up, right?

Brandon Olin:
Right.

Mike Pfeiffer:
That's awesome stuff.

Brandon Olin:
Yeah. 20 years ago, I was a United States Marine and improvise, adapt and overcome is one of the mottos, and that's what I live by.

Mike Pfeiffer:
I love it, man. Awesome. Well, Brandon Olin, thanks so much for being on the show and we'll see everybody on the next episode.

Brandon Olin:
Thank you for having me.