Today, Saron talks with Alice Goldfuss, Principal Software Engineer and Systems Programmer specializing in building resilient distributed systems at scale. Alice delivered industry-impacting talks on container platforms, infrastructure operations, and organizational best practices, as well as written on the SRE field, kernel crashes, and personal security. We hear about her coding journey and learn all about her take on various programs and the tech world as a whole.
[00:00:05] SY: Welcome to the CodeNewbie Podcast where we talk to people on their coding journey in hopes of helping you on yours. I’m your host, Saron, and today we’re talking about going from site reliability to principal engineer with Alice Goldfuss, Principal Software Engineer and Systems Programmer specializing in building resilient distributed systems at scale.
[00:00:25] AG: What I want to say is like it’s a job and you should have a life outside of it. But honestly, in my 20s, I was just constantly reading books, constantly coding, constantly figuring out Linux, prepping conference talks, traveling to give conference talks, just going all the time because I just felt like those dogs sniffing at my heels.
[00:00:42] SY: Alice talks about her coding journey and what an SRE is after this.
[00:00:54] SY: Thank you so much for being here.
[00:00:55] AG: Thanks for having me. Glad to be here.
[00:00:57] SY: So what initially drew you to code?
[00:00:59] AG: I grew up around code and computers. I grew up in a very rural, upstate New York town where we literally got our first traffic light when I was in high school and our only traffic light when I was in high school. But my parents were both programmers. And so we always had a computer in the house even before we had internet. And so it was just sort of something I was used to having around and seeing. And initially, what drew me to learn how to code was to create a website where I could talk about my favorite characters from different media properties. My father had left this like “Learn HTML 4 Book” hanging out. And I was like, “Oh yeah, that’s what I need.” And I taught myself how to write HTML 4 by hand, like inline CSS. At the time, I didn’t understand what it meant to like host a website. I knew that if I wrote code in, I think it was even Notepad++, and then I opened it in Firefox, or perhaps Netscape, I could see it, and so I had therefore made a website. That was my initial interest was, “Oh, I can now make a website to talk about things that I like.” [00:02:12] SY: And I understand that your dad actually worked for IBM and that meant that you didn’t want to get into tech. Is that fair to say?
[00:02:20] AG: It’s absolutely fair to say. Yeah. I grew up around computers and my dad worked at IBM and he worked a lot of late nights at IBM, and I actually did a few take-your-kid-to-work days. So I could see all of the cubicles back when cubicles were a thing. And I was like, “Well, this is fine, but I don’t want to do this.” This is the thing that keeps my dad from coming home until very late, and this seems incredibly throttling. I don’t want to do this. I want to do anything but computers. I’m going to go write books and make movies. So yeah, no, it came later.
[00:02:55] SY: So you rebelled. You rebelled and you got a film degree. Is that right?
[00:02:58] AG: I did get a film degree, yes, from a technical institute. So it was a hands-on film degree where we were literally shooting 16-millimeter by the second week, and I had to figure out how to load a camera and shoot it and send it off to get developed. It was a really useful education, honestly.
[00:03:12] SY: Yeah.
[00:03:13] AG: There are times when I ask myself like, “Oh, do I wish I had gotten a CS degree instead?” And while that would’ve been useful, I think kind of out the gate, I learned a lot of things in film school that have given me a leg up at work on the job that other people didn’t have, specifically around like working with teams and communication. So that has been really worthwhile. Plus, I made some lifelong friends at film school.
[00:03:38] SY: So when you graduated from film school, you started looking for jobs, and I know that they were a little bit hard to come by, and that’s kind of when you fell, I guess, back into the world of tech and then kind of got a little reacquainted there. Tell me a little bit more about that. What was it like trying to find that first film job out of school?
[00:03:57] AG: So I graduated in sort of the wake of the 2008 recession. And at least at the time, it was very typical for any kind of creative internship, like at an ad agency to be completely unpaid. Like you were paying them essentially for the privilege of apprenticing you. And no one would take me. No one even had room for unpaid interns. I moved from my little rural town to Austin, Texas because I had a friend there and there were more ad agencies and just more jobs, more people. And I crewed for an independent film and I tried to get ad agency roles and I tried to do freelance design work and like no one wanted anything. I was going to the library every day and checking Craigslist and trying to find jobs. And finally, my student loans came into repayment. And I thought, “Okay, I look at a part-time job to settle the student loan payments while I continue to try to be a designer.” And I answered this Craigslist ad for a part-time receptionist job. And I interviewed for the receptionist role. And while I was there, they happened to look at my resume and I just put HTML, CSS on the resume because it’s something I knew how to do. And when you’re fresh out of college, you’re trying to pad your resume.
[00:05:10] SY: Yeah.
[00:05:11] AG: And the manager was like, “Oh, you know HTML, CSS?” And I was like, “Yeah, I guess.” And he goes, “Do you want to interview for a tech support role?” And I was like, “No, but I need a job.”
[00:05:24] SY: So the answer is yes.
[00:05:25] AG: Yes. So the answer was, “Gosh, gosh, would I?” So there was like a test on a computer that I took and I passed enough. So instead of becoming a part-time receptionist, I became a full-time tech support person and I didn’t have a car. I was biking and/or walking in the heat to this job, working long hours and basically did not have time to do creative stuff. But I did have a full-time position at that point and I was doing tech support at a company that made marketing websites for golf courses. So very glamorous.
[00:05:56] SY: Fun.
[00:05:57] AG: Very glamorous.
[00:05:58] SY: Very glamorous.
[00:05:59] AG: You know, like use a lot of like templates for websites that you would then tweak according to the golf courses, desires, and a lot of phone tech support. But yeah, that was my first tech job technically.
[00:06:11] SY: So do you think that if you had gotten that film job that you wanted instead that you would have ended up in code?
[00:06:20] AG: Yes and no because I had colleagues, I had people I’d gone to film school with and it was just incredibly rough out there. It’s rough in general. If people aren’t familiar with the film industry, it’s honestly very similar to like blue-collar contract work. There’s no like real full-time studio jobs. Even animators are on contract. You have to provide your own kit and you’re getting up at all these odd hours and working for hourly pay and you don’t have health insurance. And I knew people sleeping in their cars to try and get work on film sets. It’s a very rough industry, and this was before the ACA passed, so I had no health insurance. I had student loans that needed to be repaid. I was renting like half a bedroom with doors that did not lock and cockroaches everywhere, and it was pretty miserable. I needed some kind of stable work anyway. And even after I got the tech support position, I ended up moving out of Texas into Portland and working remote for this company, and I kind of got a taste for it. I was like, “Oh, I could pay my bills.” Like, “I don’t have a lot of money, but I can pay my bills and I can afford this small studio apartment.” And so I wanted to learn how to code to further that because there were a couple of instances at work that gave me sort of a taste for like further complexity and problem solving. And I was like, “Okay, I can learn how to code, I can get better at this and maybe I can make more money and stabilize myself.” [00:07:42] SY: How did you feel about tech during this time when you got this tech support position? I’m assuming you’re getting good at your job, getting better at your job. How did you feel about tech in general? Did you feel like, “Huh, maybe this is the place that I want to be”? Or was this still like the pit stop on your way to Hollywood?
[00:08:00] AG: So back then, I didn’t have a concept of tech as like an industry. And if you had asked me what I worked in, I would say, “Oh, I do tech support, “not, “Oh, I work in tech,” because I was doing tech support in like a shared suite in like a repurposed mall in Texas. I wasn’t aware of West Coast Tech or like really tech with sparkles around it. So this was probably still a pit stop for me because when I moved to Portland, which is where I am now, I did try to get a job at Laika, which is an animation film studio here, which didn’t work out. So it was still very much like I’m working hourly and it’s kind of miserable pay. I was making like a little bit above minimum wage. It didn’t really have a lot of appeal to me. It wasn’t part of my identity or anything back then. It was a way to pay the bills, which I think there’s no shame in. If people are listening to this and they’re like, “Code pays well?” Yes, it does.
[00:08:49] SY: There’s no shame in that. Absolutely.
[00:08:53] AG: Not the job I had, but eventually, yes.
[00:08:57] SY: Eventually, yes. Did that attitude start to change at all? Was there a point where you started to go from I work in tech support to I’m in tech?
[00:09:07] AG: Yes. And I would say that was my second tech support job at New Relic. And I should probably talk a little bit about how I got there. So I was in this tech support job for this golf course company and that tech support job was like HTML, CSS, and there was a point where we were getting a lot of spam emails or our clients reading a lot of spam emails, and one of my coworkers was updating our spam server to block these emails. And I was like, “Oh, is that something I can do?” And he was like, “Yeah, sure. Of course, you can. You just need access to this server.” And so I asked my manager for access to the server so that I could handle spam email. And it just took forever, like I never heard back. I never heard back. I never heard back. Finally, I was on a conference call because I was the only remote person. And I was on this conference call and he was like, “Alice, what would you even do with access to that server?” And I was like, “Oh, well, I want to handle spam emails.” And he was like, “I guess.” And then he gave everyone access so then everyone could do it. And it wasn’t just for me who had been handling spam email. And that was the point where I was like, “You know what? I’m bored.” I had a glimpse of what it was to touch a server. I thought that was really cool. I wanted to learn more about servers and like command line prompts. And so I started to learn how to code at night. I would work my tech support and receptionist golf course job during the day, and I would learn how to code at night. And back then, the way I learned how to code was this very brand new website at the time called Coursera, which is very known and popular now. But back then it was just getting started and the classes were free. And there was a class called Intro to Programming where you learn Python to make video games. And so that’s how I initially learned how to code. And once I was done with that course, which was very difficult. Now it is like twice as long. It’s the same amount of material, but it’s twice as long because people were dropping like flies. It was a very rough landing into the world of code for me. But I passed and I learned enough and I was like, “I’m going to try and get a better job.” And I applied to a few positions that never got back to me. And one position that did, which was a tech support position at this company called New Relic, which I knew nothing about. I knew absolutely nothing about. I just knew that they were looking to hire someone. And I remember they sent me a follow-up questionnaire for the interview process where they were like, “Spin up an app and install New Relic on it.” And I just stared at that sentence, “Spin up an app,” for like a day. I was like, “What does that mean? What does it mean to spin up something? What is an app?” And I was even asking my other tech support coworkers. I was like, “If someone said to spin up an app, is that like start an app? Like what does that mean?” And they’re like, “Yeah, I think so.” And I really didn’t know what it meant because all the Python I was learning was within like a browser IDE and it was not like its own separate application. So I wasn’t really aware of how to do that. And I finally found the answers to the questions and the New Relic application from the New Relic documentation website. I was like, “Just Google searching for how to do this.” And then I was like, “Oh, here’s the answers.” And I got to a full day interview. And I will say that probably my first time of really being like, “Wow!” Tech was when I went to that interview at New Relic because I was expecting another like small suite in like a weird building and everyone crammed together and everyone looking miserable. And instead, it was in this big skyscraper. They had the entire floor and everything was just so luxurious to me. They had all these brand new Mac computers. People were like drinking fancy coffee. There were name brand Oreos on the snack table. I had like never had a name brand Oreo.
[00:13:00] SY: That’s how you know.
[00:13:01] AG: It was incredible.
[00:13:02] SY: That’s how you know. Yeah.
[00:13:03] AG: I was like, “Wow!” They took me out to lunch for the interview and like we had a lunch interview. I had no idea what was going on. I was like, “This is not what I expected. They’re never going to hire me.” And then they offered me a position and it was twice the amount of money I’d been making at the other job, like easily. And that’s when I, A, got very interested in tech, but also B, got very scared because I was like, “I almost didn’t take the job.” I was like, “They’re going to see through me any day now. They’re going to realize I don’t actually know what I’m doing and I’m going to have left a stable sure thing job that I’m miserable at for this new job and then I’m going to be unemployed and no one will ever take me again.” But I gave it a try because I was not making a lot of money as a quote-unquote “freelancer”. And then I go to this new place and the people are nice to me and there’s catering. And my first day at work, I just went to my desk and there was a brand new monitor and laptop and keyboard and all I had to do that day was unwrap it. It was just mind boggling to me. And I really wanted to stay there. And so that is when I got really into computers and really into the community. I guess it was partially like a survival reflex, but also, I was like, “Wow! This doesn’t have to be miserable. I don’t have to be working somewhere where I am being paid like a couple dollars over minimum wage to be harassed by my manager and my coworkers and clock in and clock out and not have health insurance. Like it could be something else. There could be room to explore.” And plus, there were just so many more learning opportunities at New Relic. I really lucked out because New Relic is software that monitors software. And so to do my tech support job there, I had to learn how software works, and that was just really useful in my education and my growth. I really think that that was just a cherry landing for me to end up there. That really accelerated my education.
[00:15:02] SY: And how did you start that? I mean, learning about software, the monitor software feels very big and a little meta. And how did you kind of start to break that apart so that you could tackle it and learn about software to monitor software?
[00:15:17] AG: The good thing was that the initial support team at New Relic, it was a small group of people, but very dedicated and they were very good at training each other. And some of them had software engineering or operations backgrounds and they were very kind and they were very good at just explaining things. And I got good, not quite faking it till you make it, but I got good at understanding things in pieces. Like I could analyze logs and I could understand what was happening on charts. And the support team had example apps and they had really well-written docs and it was like I had gotten past. the interview stage. It was interesting. I later found out that the fact that I had Googled how to answer those interview questions instead of spinning up an app myself actually worked in my favor because they were like, “Well, that’s what tech support is. It’s trying to figure things out and look for answers to things.” And so the fact that I had that initial curiosity, and I don’t know, just stubbornness worked out. And they had example apps for me to use and I learned how to set up an app and I learned how to run New Relic on it because we had our own internal documentation and it was just, I’m not going to lie, I worked so many hours. I just worked all the time trying to learn that job and learn what I was doing so that I wouldn’t get fired because I was constantly terrified of being fired.
[00:16:38] SY: So once you started picking it apart and kind of learning step by step, what was your tool set like? What tools did you use to level up and get comfortable learning about software?
[00:16:49] AG: I mean, my initial tools were like I knew Python. Eventually, I learned IDEs. I think I used Sublime Text back in the day because that’s what people were using.
[00:16:56] SY: Yeah, I remember Sublime. Yeah.
[00:16:57] AG: Yeah. Sublime Text. I was using Sublime Text well long after it was cool. People were like, “No, use VS Code down.” I was like, “But I like Sublime Text.” Because we had to understand different customers’ setups. I became really familiar with different little service setups. So my position was I was one of the tech support people for the Python agent, the Ruby agent, eventually the Node agent, and I think the Go agent before I moved over to software engineering there. So I had to learn a variety of languages, at least surface level deep, and also like their frameworks and just constantly spinning up different environments on my laptop. So I got pretty good at using VMs in installing Linux. I really liked poking around Linux. And also just New Relic, the product. Not to turn into an ad or anything, but it was very interesting and very clear to see how applications were performing on New Relic. Like I could start up an example application locally and have it report to New Relic and I could look at the dashboards for my account and see it reporting. And like that really helped me understand what was going on. And like it also helped me understand when I was looking at a customer’s page, what they were doing, and because I could replicate it on my side. And I read a bunch of books. I watched a bunch of conference talks, and also the senior engineer for the Python agent was a man named Graham Dumpleton. He is still alive. He just doesn’t work there anymore. And he lived in Australia. He started his day later than I did. So I would work my support job during the day and then in the evenings, again, I worked all the time. I would bother him. And I would be like, “You give me harder tickets.” Like I would ask him Python questions and I would just pester him. And that also helped me level up. He put up with me, let’s say, he tolerated me for a good long while.
[00:18:59] SY: And how did you ultimately make that switch from tech support to joining the engineering team?
[00:19:04] AG: Oh, that was a hurdle. So I had been in tech support for only about six months, when a bunch of engineering teams began expressing interest in having me come over. And because I worked all the time and networked so much and had been there at like kind of a pivotal turning point in New Relic, like within the company, I was well-known and really trying to put myself out there and people liked working with me. And so there was interest from a lot of the agent teams and I started interviewing with different agent teams trying to move over. And this actually caused some issues internally because I was sort of the first support person to move over. There had been someone previous to me, but it was before the different support versus engineering roles were really hammered out. Now that support was like a separate department. There hadn’t been someone who had moved over and I was trying to move over. And even though technically that was allowed, it had never been done before. And support put up some resistance because they wanted me to stay with them. And so there was some internal politics going on between support and engineering. And I remember there was an internal tools team that I had interviewed for and wanted to hire me. And the support team was like, “Well, you should. You have to interview the external candidates as well.” And so the director for the internal tools team put up a job description that was literally one paragraph and just described me. So eventually, I got to move over to that internal tools team. We own the internal tools like the Jenkins environments, the QA environments, and that was how I officially moved from tech support to software engineering. I was doing internal tools work at New Relic. That was my first big jump. But again, it’s still, I always had this vision of me being like on a ladder hung in space over like, I don’t know, a deep, frigid ocean. And I was just like, “I had moved up from the bottom rung to the second to last rung, the penultimate rung.” And I was like, “Okay, I just have to keep climbing so eventually I won’t fall into the ocean.” So now I was on a little bit of a higher rung before the abyss.
[00:21:14] SY: Before the abyss. I have that image in my head right now. It’s a little scary.
[00:21:20] AG: It is, but I mean, when you come from like… back when I was in Texas and I was so miserable and when I didn’t have health insurance, I was just like, “Oh man, like any one thing could just send me down.” I was just like, “Oh, at any moment I’m just going to be smacked down again to going into the library and checking Craigslist.” [00:21:37] SY: how did that fear impact the decisions that you made? Because on the one hand, I can see that being a very powerful motivator, right? Of something that makes you kind of work a little bit harder, push a little bit more. But on the other hand, I can see that also being a detractor and really stressing you out and making you potentially make mistakes. Or how did that fear, how did that affect the way you were operating and the way you were making these career decisions?
[00:22:02] AG: Yeah, that’s a great question. I don’t think I made mistakes on the job, but I think it drastically impacted like the rest of my life. And I find it very difficult sometimes to give advice to people looking to get into tech because what I want to say is like, “It’s a job and you should have a life outside of it and take breaks and stuff.” But honestly, in my 20s, this was in like the mid 20 teens, I just worked constantly. And I always described it as I had the job I was doing during the day and then the job I was studying for at night, because I was always trying to move up that ladder, that literal over the abyss rungs. And so I just stopped pursuing other things. I was trying to write short stories at the time because I had a film degree, I wanted to be creative. And there was a point in time where I was looking at kind of like two possible paths my life could take. I think that was when I was an early SRE, when I’d just become an SRE, Site Reliability Engineer.
[00:22:58] SY: Yup.
[00:22:58] AG: I was like, “Okay, I could do the site reliability engineer work and also try to write fiction at night in the basement of my terrible apartment building. But nothing is selling and also like there’s no health insurance in writing fiction or I could double up. I could stop writing and I could just do more coding work at night and do conference talks and just really try to secure my place on that ladder.” And that’s what I ended up doing because that was the job that was giving me money and health insurance and a community. And so yeah, I definitely prioritized things and really stacked the cards there. I was just constantly reading books, constantly coding, constantly figuring out Linux, prepping conference talks, traveling to give conference talks, just going all the time because I just felt like those dogs nipping at my heels.
[00:23:44] SY: So I want to talk about the site reliability engineer role a little bit because I think that that makes so much sense given your background in tech support and kind of going straight into, “Let’s keep the website up, but let’s first define what site reliability engineer is. What is an SRE? What are your responsibilities?
[00:24:03] AG: I actually have an entire blog post about this.
[00:24:06] SY: Nice!
[00:24:07] AG: How to get into SRE because people would ask me all the time, “How do you get into SRE?” And I wrote a big blog post about it. It was very popular. It actually got translated into Chinese, which I thought was very cool.
[00:24:17] SY: Whoa! That’s cool.
[00:24:19] AG: It was super cool. Yeah. I made like a bunch of graphics for it. It’s a very popular blog post and I think it’s very useful. So site reliability engineering means a lot of different things. Site reliability engineering started to become kind of cool about 10 years ago now, because Google popularized the term. They’re like, “We have site reliability engineers,” and that’s what happens if software engineers do operations work. Traditionally, you have the people who write code, the programmers, and you have the people who run and maintain the servers, the code runs on, and that’s operations or that’s sysadmins, systems administrators. And when you have a very large distributed system like a Google or an Amazon, or even Twitter, well, less so these days, but a Twitter, when you have like a large site like that, it gets to the point where you want to start making more automated decisions about how your platform is being run. And so this idea of these software engineer who do operations work, they write software to operate the servers and the systems came up. And Google popularized this term, like, “Oh, we have SREs.” They’re an elite force. And then everyone else in the tech industry was like, “Oh man, we need some of that.” But most places aren’t Google. In fact, only Google is Google. And very few places actually use and utilize like a Google level of SRE. But everyone was like, “SRE, such a hot term. Well, we have an operations team that owns our servers and our network, so we will just call them SREs and change nothing about their job or their authority or how they operate, but they’re SREs now. What a cool term.” [00:26:00] SY: It does sound pretty cool.
[00:26:01] AG: Yeah, it does sound pretty cool, but, oh man. Google sold it as this very elite role. Like, “Ah, they’re good at software and systems.” Like, “They know hardware and Java or whatever.” And so they get leather jackets and they carry the pager for Google. But if your code messes up too many times, they get to hand the pager back to you. They give you back the pager. That’s right, devs. Ops is in control. And it was this whole thing. But SRE in other places wasn’t like that at all. There was no handing back the pager. It was just, “Here’s more code running on your servers also. You’re an SRE now. And no, that means nothing different. Goodbye.” So SRE became this diluted term where at some places you had the Google SRE and at some place, it was like, “You’re on call for the databases and you write Bash.” And so I like to think of SRE, Site Reliability Engineering, you are responsible for keeping a large distributed platform available and reliable and accessible to people. And so if you are at a company where you have like, I don’t know, five servers running, you probably don’t need SRE. You probably need maybe like an ops person, operations, someone writing automation work, but you don’t need an SRE team. And I don’t think there’s anything wrong with being an operations engineer or doing automation. I think those are great and necessary things, but it’s not SRE. When I was an SRE at New Relic, it was one of those kind of hybrid operations roles trying to be better, and there was constantly this sort of friction within the group because people were trying to write code because they were told, “Hey, you’re SREs now.” But there was so much operations and appliance level technical debt that it was a lot of fighting fires, a lot of heavy pages, and so there was sort of that struggle going on. But I did get to write a program. So I was very close to like the networking side of things because I was constantly just like, “I want to know how things work. I want to get lower level systems.” Like, okay, I can write Python, I can write Ruby. I want to learn about the kernel. I want to learn about operating systems. I want to learn about hardware. I want to learn about networks. I just want to know how things work under the hood. And so I got very close to our networking team and we’ve ran F5s, which is a load balancing software appliance. And they could be very difficult and finicky things to configure. And a lot of operations are our SREs didn’t want to mess with them. We would have to wait for basically one guy to be available to put in a request to change the load balancers. And so I wrote an application that fronted the load balancers and basically had a form that you filled out that handled all of the kind of routine changes that people needed to make. And it would flush that and talk to the F5s. And so I got to write like network automation software. And I would argue that that’s a pretty SRE thing to do. So that’s kind of an idea of the role. It can mean so many different things at many different companies. Sometimes there are SREs who only manage internal platforms, like a giant database that only has internal use could possibly use SREs if the company and the need is big enough.
[00:29:16] SY: Right. Right. And I’ve also heard a relatively new, at least it feels new, term popping up over the years, platform engineering. Is that related? Is that basically just SRE 2.0? How do we understand platform engineering?
[00:29:32] AG: Platform engineering is the new hotness. What happens is you have a company and you’re like, “Oh, we have a website. We made an application.” So we’ve got application developers or software engineers working on that application, and then we have an operations team or SREs that own the services that this runs on. But eventually the company and the architecture gets big enough that you’re like, “Actually we need some sort of middle layer here. We have all these databases and these backend services and like networking things and we’re trying to connect all these lines from our front end to our back end. And what we should really do is build some sort of abstraction layer. We should build our own internal platform to run the front-end application on.” And that is sort of where platform engineering comes in. You hear a lot of like, “Okay, when you have like a platform engineer, it’s sort of like you’re writing software, but you’re writing back-end software with the intent of running other internal services on top of the platform you are making.” So you could argue that some of the SRE work I did at GitHub was actually platform engineering work because we were building applications to abstract away Kubernetes so that we could run GitHub applications from GitHub programmers on top of that platform. So my title there was SRE and I think it eventually was production engineer even. So platform engineer is kind of like, “Oh, we have a bunch of databases, but we don’t want our front-end engineers to constantly be making direct calls to the databases. So we’re going to abstract that away behind a platform and they will just have a single API endpoint. Like that’s a platform we are making internally. And so that’s platform engineering.” [00:31:16] SY: Very cool. Okay. Makes sense. Coming up next, Alice talks about her transition from SRE to principal software engineer after this.
[00:31:41] SY: So you left New Relic in 2017 and you are now a principal software engineer. What happened between now and then? How do we get from SRE to principal software engineer?
[00:31:52] AG: So I left New Relic to go to GitHub in 2017. And I was an SRE there first on the networking side, which was very cool, by the way, to be on the team that owned all of the edge for GitHub. GitHub gets quite a bit of attention from all sorts of places.
[00:32:06] SY: Just a little.
[00:32:07] AG: Just a little bit. And then I worked on the container Kubernetes team there also as an SRE, and that was where I got to write my first production C code. I actually specifically joined GitHub because I knew that they ran C in production and I wanted to write more C and constantly wanting to get lower and lower and closer to the metal is what we say, closer to the metal. And I didn’t make a full transition to software engineering at GitHub. I did software work at GitHub under the SRE title because that was the expectation. You own the systems and you write software for the systems. But I was still an SRE, production engineer, when I left GitHub at the end of 2019, shortly in the wake of the ICE business at GitHub, and I didn’t have another job lined up. I left GitHub in November of 2019 and I was just unemployed for months. Honestly, I was very burned out from working all the time and just constantly trying to race up that rung. I was very like disillusioned and disappointed in tech and just how much of like my attention and passion it had taken up and I just needed a break. And so I’ve been wanting to learn Rust for some time because at that point I knew C and I had been studying Linux kernel, but there’s no community for C programmers. It’s very odd. The community is the Linux kernel mailing list. There’s no conferences for writing C. But Rust has a very vibrant and wonderful community, and it’s a systems programming language. So I spent my unemployment time learning Rust from a book and teaching myself Rust. And then like February, March of 2020, something changed in the world. COVID started and I was like, “Oh, I should get a job.” I was like, “Everything is very uncertain.” I was like, “I should get a job right now.” And that’s when I started applying to places. And I put up actually kind of like a reverse job posting where I made a blog post where I was like, “This is the stuff I’m good at. This is the kind of job I’m looking for. Contact me if you want to hire me.” And I got a lot of interest. It was cool. I think it worked out very well and I got a lot of interest, including my current employer, who basically made a role for me when they saw that I was on the job market and they reached out. I had interviews with them. I loved the description of the work it was going to be as a software engineer, but the role I was hired for was to do kernel efficiency work, which I was like, “Oh yes, please. That’s exactly the kind of stuff I want to do.” So they hired me as a senior software engineer. But shortly after I started, they introduced a staff software engineer role, which hadn’t existed and then I was like back promoted into being a staff software engineer like immediately, which was very cool. And then after, so it was a year ago, so early 2022, I was informed that they thought that I had been under leveled when I was hired. And so I was promoted to principal software engineer.
[00:35:11] SY: Wow! That’s nice.
[00:35:12] AG: It’s very odd, I have to say, because I’ve basically spent most of my engineering career just being like a baseline engineer, site reliability engineer, no senior, no nothing, and then I’m at this place, and now I’m a principal software engineer within just a couple of years. And it just really feels like I was like a rubber band that had been pulled back and there was all this tension gathering and then suddenly this company is just like, boom, there you go. Let you go. Boom. Just propel you forward. Like this is all the stuff you’ve been working for.
[00:35:38] SY: Now you’re on top of the ladder. Look at you.
[00:35:40] AG: Yeah.
[00:35:41] SY: No more hanging over the abyss.
[00:35:43] AG: It does feel very nice and I love this job. I feel like I have been working towards this job my entire career because it’s a combination of systems programming work, which is like lower level languages and also distributed systems resiliency work, and I just feel like I’m in a unique position to do it. I love my job. I love the stuff I get to work on. And what’s really cool is that now that I am getting paid during the day to do stuff that I want to do and I get to learn the stuff I want to do, that means I have free time in the evenings and on weekends. I am not constantly working one job and trying to work towards another. It’s been really great.
[00:36:25] SY: That’s awesome. I’m curious, what does your dad make of all this, the IBM guy seeing his daughter kind of make her way back into tech and kicking butt, doing so well in it? What does he think?
[00:36:36] AG: I like to think he’s proud. Occasionally, he’s no longer with IBM. He’ll be like, “You know, you can come back to the East Coast and work here.” And I’m like, “No, thanks. No, thank you. I'm good.” We do talk about computers on the phone, and I think it irritates my mom, but he does a lot of like Java stuff. I’ve taken care of JVMs, but I have not really written much Java, but it’s nice to talk to him about stuff.
[00:37:01] SY: Very cool. Very cool. Now at the end of every episode, we ask our guests to fill in the blanks of some very important questions. Alice, are you ready to fill in the blanks?
[00:37:16] AG: I love filling in blanks, Mad Libs. Let’s go.
[00:37:20] SY: Number one, worst advice I’ve ever received is?
[00:37:23] AG: Okay. So worst advice, well, first of all, I want to say my mother once told me that I would never be allowed to take naps in college. So like already, that’s my favorite advice. Worst advice specific to tech, I actually think this advice is good advice, but it was bad for me. I was at a conference and I was talking to someone after hours about where I wanted to go with my career, and I was like, “Man, I really like kernel stuff and lower level systems things. I think that stuff is fascinating.” And this person said, “No, no, no, no, no. That stuff is not going to matter in the future. It’s going to be abstracted away. What you should learn is cloud providers and cloud systems and like distributed databases. That’ll get you a job.” And I agree. I agree with that advice. I think there are way more jobs managing databases in AWS than there are doing kernel work, but I was just not interested in it. I tried to learn it and I didn’t care. It was too much abstraction for me and I really like to know how things work. And so instead, I continued learning about kernels and doing capture the flag exercises because I thought assembly was just super cool and that got me to the role I’m in today where I’m just so happy. And if I had taken that advice, I would be in a different kind of role where I would be less happy. And yeah. So for me, that was terrible advice.
[00:38:45] SY: Got you. Number two, best advice I’ve ever received is?
[00:38:50] AG: Don’t work somewhere that gives you free dinner because you’ll be expected to work during that. But also further than that, have some sort of, you need external validation outside of your job. I don’t know if this is the case today with remote work being more popular, but back in the day, these tech companies, they wanted your entire life. The large campuses like Google, they would do your laundry. They would feed you three meals a day and a lot of the startups tried to capture that as well. And you wanted to go into it, especially if you were young, just out of college, like you were kind of missing that community experience. You wanted to just give your whole life away to these companies and that meant that if you had a bad day at work, if your project was canceled, if you had a bad performance review, if you were laid off, you were just crushed and you had nothing. Like all of your sense of personality and validation were tied up in this job that at the end of the day is just a job. And so I urge people to have some source of validation outside of work, whether it’s hobbies, whether it’s like sports, like running marathons, whether it’s like a really strong like romantic or familial or friendship relationship, you have to have something else. So when you have a bad day at work, you can be like, “Yeah, but you know what? I can make a mean loaf of bread and my friends all love me and my cat loves me and I’m running a half marathon this weekend.” You have to have something else to balance yourself with.
[00:40:18] SY: Yeah, I love that. Number three, my first coding project was about?
[00:40:23] AG: If you count HTML, CSS, then my first coding project was a little website where I just hosted it locally because I didn’t know better and it was talking about characters that I had crushes on, like Night Crawler from X-Men.
[00:40:38] SY: Nice.
[00:40:39] AG: I thought the world needed to know about it.
[00:40:42] SY: They did. We did need to know. Thank you for sharing. Number four, one thing I wish I knew when I first started to code is?
[00:40:49] AG: So this will probably sound silly to people listening to a podcast about coding, but I wish I had known that there was a community. I did not know that like meetups were a thing. I didn’t know that Stack Overflow was a thing. When I was doing that Coursera course to learn Python, my only sense of information was the course material and the course forums. And when I was stuck on something, I was just bashing my head against the walls of my studio apartment thinking I would never, ever learn this. And there was no one to help me but myself. And I wish I had known that I could go to a meetup and have someone explain things to.
[00:41:25] SY: So powerful. And I feel like especially if you’re learning to code nowadays, there’s so many different communities online, in person, locally all over the world that give you that opportunity to learn from people and meet some great people. So we’re very lucky to… [00:41:37] AG: Yes, and you level a lot so much faster just being around people because you hear what they’re talking about and then you can go research it. But you have to know that something exists first.
[00:41:46] SY: Yup.
[00:41:46] AG: And I wasn’t finding out this stuff on my own.
[00:41:50] SY: Yup. Absolutely. Absolutely. Well, thank you so much, Alice, for being on the show.
[00:41:54] AG: Thank you for having me. A pleasure.
[00:41:59] SY: You can reach out to us on Twitter at CodeNewbies or send me an email, email@example.com. For more info on the podcast, check out www.codenewbie.org/podcast. Thanks for listening. See you next week.
Thank you to these sponsors for supporting the show!