00:47
This is the Art of Network Engineering podcast.

00:58
In this podcast, we'll explore tools, technologies, and talented people. We aim to bring you information that will expand your skill sets and toolbox, and share the stories of fellow network engineers.

01:11
Welcome to the Art of Network Engineering. I am Tim Bertino and I am doing fantastic this evening because I am joined by two of my favorite people in the entire world, Lexi Cooper at Track and Pacer. What's happening, Lexi? Hey, what's up? You don't seem very excited to hear that you're one of my favorite people in the entire world. No, you didn't see my head bobs, my excited head bobs. Oh, sorry, I wasn't paying attention. I did them. You're one of my favorite people too, Tim. Thanks.

01:41
I had a productive day to day, so I'm happy. Daylight savings happened, which kind of sucks, but I'm also happy because there is more sun in my life now as a result. So that is awesome. So today is a net win for me. I probably need to pay more attention to voting in this country, but aren't we supposed to vote that out at some point? I thought we were getting rid of it. I guess I should pay more attention to it. Because I'm tired of it. I hope we get rid of it. I keep hearing it's coming, but yeah, I don't know. I mean, in the grand scheme of the problems that we're looking at.

02:10
I don't know where daylight savings time fits in the prioritization of. When did that ever stop anyone from doing anything, Andy? Totally. I'm totally for getting rid of it, but I think we might have bigger fish to fry at the moment. And that my friends is another awesome person. Andy Lapteff. Andy, so good to see you. Hey Timothy. Good to be seeing buddy. Great to be here. I changed the batteries in my goat this week.

02:39
So the winning goat is back. I am fresh goat if I ever heard one. Yeah, I'm lobbying for you to start reading the wins from our discord channel on the show again. I don't care if they're a month old by the time the show comes out. I miss the wins. Are you still doing the Twitter thing? Where you shout them out? No, I got lazy. So let's do it on the show. Yeah. So anyway, I'm hoping the wins come back, but no, work is great. I'm having the most fun that I've had since I've started there.

03:06
I'm starting to do things that are aligning with my strengths. Shout out to Mike Bushong, strengths episode, and I feel really good about what I'm doing. So, yeah, man. It's shitposting, right? On the Juniper Twitter account, right? Yeah. I'm doing some content creation around some cool stuff that, you know, I can't talk about like Lexi, but creating some content, which is something I like to do, I'm communicating, I'm doing some storytelling and I'm getting some good feedback with my initial stuff. So I'm super excited. Now I have like a pipeline of work that I'm.

03:36
working on. So work is awesome after, you know, eight or nine months of feeling like a lost little puppy dog. I finally have something to focus on, which I love. And, uh, hey man, everything else is good. I'm super excited for the Cables to Clouds show we just launched. We just became like a podcast network. That's even a thing. That's what AJ called it. So I guess it's a thing. No, no. Andy, it's an empire. That's what I'm choosing to call it. I'll call it an empire when I get my first paycheck, Lex.

04:06
So who do we got Tim? What's going on brother? Oh wait, how are you? I have to ask you how's Tim? I'm good. Yeah, you gotta do better than that buddy You're like my son. How was your day buddy? Fine. Fine school was fine. Everything's fine. Who'd you sit with at lunch John? No, it's good. I'm still I'm with you. I'm riding on the high of releasing the cables to cloud podcast, that's a

04:33
Great group of folks. I think it really helps that they have known each other for a while and they got a good chemistry and, and it's going to be really interesting when they start diving deep into technical aspects of cloud. And I think they're going to get a big following. But, uh, as far as me, I've had, uh, I think last week I had, uh, some college buddies in town and we just got to hang out and catch up. We try to do that a few times a year. So I am good and refreshed. And what are you laughing? I'm just, I can't imagine the debauchery.

05:02
You and your college buddies. Although you were like computer science guys, so you might've just sat around and played Dungeons and Dragons and nerdy shit like that. I take one beer selfie outside of Andy's clothes door in an Airbnb and everything I do now is debauchery. All right. So let's talk to our guests. Let's talk to our guests. What's going on? Lexi, Andy, and I are very excited to invite on Jeremy Stretch. Jeremy's a distinguished engineer at NS1. Jeremy, welcome to the show. How are you?

05:31
Doing very well. Thanks for having me. We appreciate you coming on. You've got a very storied past. You've done a lot of cool things. You are, I'm still gonna call you a network engineer. I know you've been, you've done quite a few things over the years, but I know network engineering's near and dear to your heart. Can you kind of take us back to the beginning? How did you, what got you excited about networking to begin with? Absolutely. It's funny you mentioned the network engineering as a title, because I think it was,

06:01
I've been doing development for years now, but I think it was just about a year or so ago, I introduced myself as, hi, I'm Jeremy. I'm a software developer. Wait, is that right? Like it was a question? It very much kind of like, I woke up one day and I realized I haven't touched a router or a switch and I don't know how long. And it makes me sad a little bit, but it happens. Yeah, so I got my start. I think most people probably know me from, well, from my last name. It's a stupid last name. Who has the last name, Strudge?

06:30
But secondary to that is probably PacketLife, right? That's the blog I started back in, back some time ago, 15 years now, I guess, back in 2008. I was a defense contractor out in Iraq and shortly after having completed my military service in the Air Force and found myself with ample time and not many ways to fill it in the middle of Al Anbar province, Iraq. And so I set about studying for CCNP and I said, well, I'll do something productive, you know, alongside that, which is

06:59
set up a blog and everything. And interestingly, a lot of people might not know this. That's actually when I got my first, I guess, experience in development, because the website itself was built on the Django platform, which back then, I believe, was version 0.96, something like that. And that's when I first started getting into Python and development. So that actually predated anything I've done around like network automation or anything in that arena. But yeah, I started doing the blog, and kept getting more and more.

07:26
our readers that kept going for years and years. And of course the cheat sheets, right? Everyone knows that the cheat sheets. All hell the cheat sheets. They're good. Oh my God. I still use them to subnet Jeremy. I still use them. Good. I mean, that one at least is still up to date as far as IPv4. I don't think that's changed any. So good. This is last I checked. Although a lot of them have fallen out of date. I still get emails every once in a while. Hey, can you update the cheat sheet? Like, no.

07:54
Not really. I would love to, I'd love to revisit them. I'd love to redo them, but with everything else going on, it's just, hasn't been, haven't had the time budget for it. Were you doing networking? I, so when you started it, I got to know what a defense contractor does. Were you doing networking? Are you even allowed to tell us what you were doing? Yeah, no, I was, I was just a regular old contractor out there in Iraq working with the Marine Corps at, I was at fortunately a very small base out of the way, which is where you want to be ideally in Iraq.

08:22
right far away from the action. And I had maybe four to six hours of work to do per week. But I got paid for the full week, your tax dollars at work. And in the ample off time, I would train the Marines and things like that and networking because those guys would rotate out every six months and they would come in and they're all different specialties. So they have different backgrounds and everything. So it was really cool to see and work with those guys. And in doing that, it was a great experience.

08:49
basically just running the local network for the base, had a couple satellite uplinks and some Cisco switches, and that was pretty much it. Did you learn your networking in the military? I actually learned. I got started with Cisco through a vocational class in high school, believe it or not. I was very fortunate to land one of those. So I spent half the day in a community college. I didn't even realize it at the time, what a step up it would be for my career. I was more like, oh, this is kind of cool. Networks sound interesting.

09:18
So I did that, I got my CCNA through high school, and then I became lucked out a second time when I landed a tech control AFSC, which is an Air Force specialty code. So it's like your job code in the Air Force, which is 3C2X1, which sadly has been retired in recent years, it's no longer exists. I think it's been replaced with like 3X and 3D stuff for anyone who knows the vernacular.

09:42
But I was a tech control specialist and I landed a position in Nellis Air Force base outside Las Vegas doing primarily Cisco networking. So that was armed with the CCNA and this giant network that only the government would be stupid enough to let an 18, 19 year old kid go play with. It was fun times. So you jumped right from high school to the Air Force? I did. You were managing a network. Wow. I wanted to ask that. So was that something that was in the plans from a young age?

10:11
Did you hit high school and just trying to figure out what you want to do and landed on the Air Force? I think I was, for better or worse, I think by my senior year of high school, I decided I was done with schooling for a while. Sure. And I wanted to do something different. Yeah. So yeah, it's an interesting career path and I, especially now. So it's been, I enlisted at 17, I actually turned 18 in basic training of summer birthday. But

10:35
I'm 37 now, turning 38 soon. And that would be about my 20 year mark. Had I stayed in, I'd be retiring next year, probably. I'm sure that's crazy to think about. Yeah, how does that feel? Didn't quite pan out that way, but all for the better. All for the better, I'm sure. Did you have the idea that you wanted to get into technology when you signed up for the Air Force? Or did it happen once you were in? OK.

11:01
No, I did. I definitely wanted to go into computer science. I remember there were a few different specialty codes you could select. So they were all full under 3C. So it was basically like sysadmin, network stuff, different things of that nature. Ironically, my specialty code was actually for long haul circuits. So I went to tech school in Biloxi, Mississippi, Keesler Air Force Base, and I learned how to, I still remember

11:25
We learned the old, old stuff because it was still in the books for some reason, like wave guides for microwave radio systems, which still exist, certainly, but it's not something E3s touch on a regular basis. And I remember the long-haul circuit frames. So we used the vernacular distribution frame when we talk about racks and switches and stuff. No, this was an actual steel metal frame. There was no active components. And you were terminating individual wires onto the little pegs using a little twisty tool.

11:54
And that was the only time in my life I've ever done that. I think that's probably the same for anyone who's gone through that school in the last, I don't know how long. That was my first job in tech at a central office, punching down wires on. Yeah. On a clove frame. I found out later it's a clove network, by the way, there it's clove architecture. All those metal crossbars. Is that how you pronounce it? Clo. I'm going to. It's French. So you could say closs, but it's clove. Everybody says closs, but it's clove. A lot of fighting about that here and there. Yeah. But yeah, that was.

12:22
That's how I got my start. And I think it's a great on-ramp for a lot of people because once you start in the military, certainly a lot of it's luck with a draw. No doubt about that. It depends on where you get stationed, what opportunities are available to you at the time, the priorities of your command and things like that. But for me, it worked out very well. And I got tons of hands-on experience with like enterprise networking very early on. This is back in the days of catalyst 2900XLs and that stuff. So.

12:49
simpler times. I guess you did good in your entrance exam, right? Because don't they place you in the Air Force? I was just going to ask about the ASVAB. You must have had a pretty good score. I do. I remember I could tell, like, they would show you a diagram of, like, four gears. And it's like, gear A is turning clockwise. What direction is gear D turning? This is kind of easy. But yeah, I remember that the test was interesting. But you did well on it, I guess. I guess. I think I got like a 96 or something on the ASVAB. So yay me. But what points did you start?

13:19
packet life. Yeah, so that was after my Air Force days. After I got out of the Air Force and went, ironically, I was all stateside during the Air Force and went to Iraq as a contractor, as a civilian technically out there. And that's when I started. I had all my downtime. It was just day after day of trudging through the Iraq sun and trying to spend as much time indoors as possible. What was the draw to being a defense contractor? A lot of money and a lot of it tax free. So going from making like, I don't know, it was like 25 grand a year or something as an E3 or E4, I guess.

13:49
to making a defense contract or salary. They were throwing money at anyone with a pulse. I remember my interview. I seriously was, it was like two questions. It was like, are you something about like, you have a clearance? Yeah. You healthy? Yeah. Okay. You know what Cisco is? Yes. All right. When can you start? Oh man. You get the hazard pay. So why did you start a blog?

14:12
Cause I started a blog to try to make a name for myself in the industry because I was trying to break in as an outsider. You were established in your career. It sounds like by the time you started a blog, it was, I don't know about established in my career. I mean, had I gone on to do other things without the blog, I certainly would've struggled to find a foothold, but I started it primarily as a way to enhance my own studying. So I kind of set out studying for the CCNP, which I guess if you go back to the very early articles, 08, 09.

14:40
You can kind of follow that theme as to what the four major exams were back then. And I've always been a strong believer, at least for myself personally, I learned better if I can read something, process it, and then write about it. Like go explain it once, now that you understand it, go explain it to someone else. Right. Cause that's the true test of whether or not you really, you know, grok what you just read. And in doing that, it's repetition, it's forcing you to, you know, before you go write something, oh wait, was that correct or not? Right.

15:10
And just the repetition and getting it out there helped me tremendously. I think I eventually ended up getting the CCMP at some point. But in doing that, a lot of other people saw that and that's where the cheat sheets initially started, right? It was like these nice little like segmented areas of knowledge, all crammed onto an 8.5 by 11 sheet of paper turned out to be another great way to do it. Because when you go to make a cheat sheet on IPsec, for example, you have to figure out, okay, what do I need to know about IPsec?

15:40
Because you're limited in the amount of physical space and what you can actually cover. And it's really interesting because then you have to go back and figure out, how succinctly can I explain things like AH versus ESP? I'm probably mixing up the acronyms at this point. It's been a while. But things like that. And it's an interesting challenge. I'll definitely say, if you're struggling to get around a concept, definitely try it yourself. Sit down. It doesn't have to be anything fancy, but get a normal blank sheet of paper.

16:08
And write down everything you can about it and fit it on that paper. And by the end of it, you'll have like everything memorized. That's crazy. I like the idea of constraining yourself to a, to a space. I took a short story writing class in college and it was one of the hardest classes I ever took. Cause I, I don't know, it was a thousand words, 3000 words, but you had to tell a story in a very small space and it was so hard. It is, it really is. And we all think like growing up, you know, as kids in school, like, oh gosh, I got to write five pages. And then as a professional, like I only have five pages to fill.

16:35
We were, a buddy of mine and I were laughing because he's in a master's program and he is like a very old school professor for one of his classes. And she was giving assignments that was like five pages in length, seven pages in length to talk about a topic. We're joking saying like, if I give my boss five pages, he says, that's great. Tell me what you just wrote about in like a paragraph or less, right? How accurately and succinctly can you convey a complex idea that's a real skill? Be concise, right? It's so hard.

17:05
I know we just talked about this, but let's, I want to touch on it a little bit more. So you made the cheat sheets as a way to enhance your own study, right? What about like blogging about things? Did that do something similar for you or is that more a way to get your thoughts out and like sort of, I don't know, vent in a way about networking and the things you were learning? How did that enhance your studying? Just like the blog portion of it.

17:31
Definitely depend on the content. I mean, certainly some of my blog articles are straightforward. Here's how you configure protocol X on a router. Some of them are a complete waste of time. A lot of them fall in between. So a lot of it is just it began as a study tool. And then as it gained a following, I said, hey, this is a cool way to get information. I could share other things out there. One of my favorite things to do is to highlight other people's work and say, hey, this person wrote something cool. Go check it out.

17:59
And I noticed you have a blog role on the side, and you have a featured blog. Yeah, you definitely support others as well. That's great. I've tried to. I've not done as great a job with time demands and everything these days. But early on, especially as a game to follow, and I love to use that as a launching platform. Because so much of what we see today, I feel like, is you have a lot of artificial promotion, more so now than ever, even 10 years ago. And that's great to be able to kind

18:28
really organically have things grow. Back before the internet was just Facebook and Twitter, people actually had their own websites, believe it or not. I did. So I'm seeing the blog and the cheat sheets is all networking related. And you mentioned at some point you became a software developer. How and why did that happen? So then we jumped forward a little bit to my days at DigitalOcean. So I spent about five years there after hopping around. So that was about 20,

18:57
14, I think I started there. So yeah, in the latter half of that decade. And I came in as a traditional network engineer. I had some good amount of experience under my belt at that point. DigitalOcean was still a small but very quickly growing cloud hosting company. And when I started there, we were still doing things like, we had configuration templates, but it was mostly like kind of a copy and paste by hand deal, not anything you'd consider automated in any fashion.

19:26
And we were using, like most people, spreadsheets for IPAM, as you do. And over time, as we kept growing, as the company was growing, we realized we can't scale doing what we're doing. We need to do better. And a lot of my work slowly shifted from the traditional network design and engineering work to software development. And I joke, but I basically fell into it by accident. Me trying to always pursue a more efficient way of doing something

19:56
Obviously, the way to do that is to program. So you develop tools to do that. And like I said, so I got a very early start in development through my work with PacketLife, fortunately. So that set me up to work with the Django Python framework. And that's where I started working on Netbox. So my manager at the time, Lucas Elvatory, I went to him and I said, look, we got to do better. We've looked at some different IPM products. None of them really fit what we need. And I'm happy to go into that if you're interested. But...

20:24
I said, give me a week, let me see what I can put together as like kind of a proof of concept for an internal tool. Now, the huge caveat here is I don't recommend doing this. It usually doesn't work out well. I've done it, I've tried it before and it's failed. But for whatever reason, whatever stroke of luck, it worked and it's what became Netbox. So that's kind of what took me in the transition. And as our team grew, I became not just a network engineer who does some development on the side,

20:54
over the course of a year or two, a software developer who does some network engineering on the side. This is big for me, Jeremy, because I'm a network engineer who has really struggled with any kind of dev stuff, any kind of coding. So I just wanted to start real quick for myself and back up. So you, when you went to DigitalOcean and you were going to do dev work to scale, you had experience at that point with Python? I did. Right. Because I had written, so the PacketLife.net blog, it wasn't just like a WordPress blog.

21:23
Because as mentioned, I had an ample amount of time, I decided to why not write the website too. Right. But that's what gave me a kind of an early foothold, I guess, into Python. And it was all strictly web development, but it was still a development experience. Was that just self study? Like, how do I do this? Oh, there's Python. Let me learn syntax and language and I think I kind of learned Django and Python together. Okay. And you taught yourself? Yeah. Okay. And it was easy. Like you got it.

21:50
I managed, now it seems easy. Well, when I hear somebody jump from network engineer to dev, my ears perk up because that's, I mean, Lexi and I were joking on the side the other day too. Like this is, you know, how do you succeed as a network engineer? Something Lexi just put out the other day, become a software developer. That's how you succeed. That was satire. I do not believe. No, no, no, no, it was, but.

22:15
Every year, every hour, every minute, it's like, okay guys, yeah, the job is, it's changing in huge ways. So, um, I'm amazed that you were able to jump into dev. It sounds like relatively easily and there's different skill levels, right? Like you're good with coding. I struggle with it. So that's, that's awesome. You were able to jump right in. So you created Netbox at DigitalOcean because you needed a better IPAM. Yeah, that's exactly it. Yeah. And it initially was a, um,

22:42
just an IPM application, it very quickly grew once it caught on. And it was because we were using the Django framework, which is sort of, I guess a good analog would be like view.js or Angular in the JavaScript world. So it's a framework that allows you to iterate and prototype things much, much more quickly than doing it yourself, right? You're not writing code that actually has to handle raw HTTP requests and all the authentication and all the database backend. That's all there for you. So you can just concentrate on the business logic.

23:11
And that allowed us to very, very quickly scale out and start adding things like racks and devices and sites and all the other stuff that we needed to manage. And turns out we're still doing that to this day, seven years later. How did it turn into, so it started out as an internal tool that you created for your company to be more efficient. How did it turn into what it is today? How did you leave with that tool? You created it there.

23:36
So I always have to have to give kudos to DigitalOcean because they've always been, from the early days, they've been super supportive of individual developers. They've been very active in the community. They have some of the best technical documentation out there as far as, you know, how to do X from a developer standpoint. And they've always been super open source friendly. So I basically went to the leadership and I realized we were onto something and I said, Hey, can we release this as open source? And they said, what's in that box? And so there were some conversations around that.

24:05
around licensing, you know, it's licensed under Apache 2, just as it was released under Apache 2. But there were still considerations around what does this mean for the company, you know, what are our obligations to shareholders and things like that before we go through a process like this. And we were kind of making up the process as we went along because they had released open source things in the past, but it was all or mostly projects specific and pertinent to DigitalOcean's product. So like their...

24:34
API client, things like that. This is the first standalone project that really had nothing to do with DigitalOcean itself. It just kind of was born out of the company. But we did eventually release it. It was June 2016. Yeah, that's right. That seems really long ago, but that's accurate. So because it's open source, there's no fight over like, who's is it, who owns it, who's gonna get what money when we sell it. Like it's an open source project, right?

24:58
Right. It was basically the company agreed to release under Apache tool in terms of the Apache tool license basically say you can do more or less what you want with it. There's still a copyright. There's still some limitations on what you can't do with it. But for the most part, yeah, it's open and out there and it's one of the more liberal open source licenses. Wow. It's great culture that they didn't say this is ours now. Yeah, it was. I mean, they weren't, DigitalOcean wasn't in the business of wanting to sell an

25:28
One of the solid arguments I made was, solid in my opinion, one of my primary arguments was, look, if we release this, it's gonna become a more mature, more robust tool because we're going to have outside parties looking at it, evaluating the code, telling us what doesn't work. And we had a ton of feature requests and bug reports just in the first few months that would have far surpassed anything we'd be able to do internally, right? It's one thing, I think that's part of the problem with internal tools is someone says,

25:57
Oh, there's a bug here. Oh, we know about that. We'll fix it eventually. But when it's a public tool, you kind of have to. You're kind of more obligated because you have 17 people beating down your door instead of one. So it does lead to a more mature product, I guess, in most cases. And the feature requests that we've gotten as well and the contributions from the community certainly started to accumulate at a later date as well. And people are saying not just it would be cool if, but also, hey, I would like to help with.

26:27
And that got a lot further, a lot faster than it would have been as a closed source or proprietary tool. What kind of advice would you give to someone else out there who might create, you know, go along the same path, create an internal tool that's very useful for them at their company and then eventually decide they want to open source it and, you know, present it to the community? What would you tell someone who's looking to do that? One of the biggest challenges that I think probably the biggest challenge was getting people.

26:55
to understand open source because as prevalent as open source is today, a lot of people don't really understand the implications of it, especially on the business side. It's, wait, we paid you to build this. Why would we give it away for free? But for like the benefits that I covered, there's a solid argument to be made. There's goodwill, it's promotion for the company and the community as well. There's a lot of reasons to do that. But

27:17
but you need to be able to express that in terms of business value. It can't just be, well, it makes me feel warm and fuzzy, right? It's no, this is going to help the business because, and then listen, you know, iterate on those points. Another point that I'll cover, because I see this pretty frequently is, a lot of people say, well, I don't want to release it as open source because my code's not that good. Neither is mine. Most code isn't great, right? Owing to the time constraints that we all have to answer to. It doesn't matter, right?

27:47
Bad code is tolerable to a point, certainly, in exchange for delivering functionality and features, which is probably not the thing I'm supposed to say with my maintainer hat on, but there's a truth to it, right? Obviously, there's ways to clean things up too, right? It doesn't have to be perfect. Just because you put something out there doesn't mean it has to be perfect from day one. Say, hey, I'm working on something. I think I'm onto something. Other people tell me if this looks useful, right? And inevitably you'll get feedback because it turns out that we all have opinions on things.

28:16
that we'd like to share on the internet anonymously. No. What? But for the most part, I mean, people don't succumb to imposter syndrome and thinking, oh, I'm not really a developer. That was me for a number of years. I'm not a real developer. I'm a network engineer. I'm putting my software developer t-shirt on. But it turns out there's no licensing for becoming a developer. It's not like being a doctor or a lawyer. You just kind of show up and start doing it.

28:45
Um, but those are probably the biggest two pieces of advice I'd give. Yeah. I like that. I like that a lot. And I like that you tied it into imposter syndrome because that's definitely true, right? You definitely a thing. Yeah. And if, you know, you might put out not perfect code or a not perfect product, but at the end of the day, you know, you're going to improve it. You're going to learn through improving it. So just go ahead and do it. Right. I love that message.

29:11
That's exactly it. I mean, there's definitely worse code than yours on GitHub. I can tell you that much. Whoever you are, that doesn't. You define bad code. Do you just mean ugly or do you mean buggy? Both? I'm speaking more to like, um, inefficiencies, incomplete, like things that things of that nature. Certainly bugs need to be fixed security vulnerabilities and things like that. But more of like people worry, Oh, I need to, I need to refactor all this before I go put it like, well, you can put it out there and then refactor it. Like there's nothing saying, you know, you can't disclose it.

29:40
Um, very, very seldom will you see anyone attack code saying, Oh, this is embarrassing. Like you shouldn't have put this out. No, no, no, say that. Right. Some people might, but they're, you can ignore those people. Right. Code bros. For the most part. Yeah. For the most part, people want to help, right? They want to see you succeed because they are interested in what you've written. They want to use your tools. So they want the tool to be good. So they'll offer a feedback where they can. I like that advice. Cause I think it goes back to the.

30:10
the quote that I like that I always get wrong. That's essentially perfect is the enemy of good. Like if all you try to do is put out this perfect pristine thing, then there's a good chance that you may never do it. I mean, that's one of the best, that's some of the best advice that I got early on when I was gonna start a blog. Because I know me, I would have had to get the website just right and I'm not a.

30:37
I'm not a web designer or anything like that. So I would have sat on that forever. And then I got the advice that said, no, just, just write. You got to find, you got to understand what your purpose is for what you're trying to do. And for me, it wasn't trying to design a website. It was trying to write content and get it published. So that, that's what I did. Otherwise, if I wouldn't have had that advice, I probably still wouldn't have written anything on the blog. Yep. It's a solid strategy for sure. Um, and, and moving forward, like it.

31:06
Sorry, I lost my train of thought. I do want to pick up, I want to do some more justice to like the core of Netbox. So I found a quote on one of the web pages that kind of frames up what Netbox is at a high level. And it goes, Netbox is the source of truth for everything on your network, from physical components like power systems and cabling, to virtual assets like IP addresses and VLANs. Can you kind of go into...

31:30
what it means for a system like Netbox to be a source of truth and how does it become a source of truth? So I think one of Netbox's strengths early on was that it was able to marry systems that traditionally had existed in separate applications and separate databases. So you know, we've all probably used like legacy IPAM and DSIM tools. So on one tool, we've got over here is all my subnets and my IP addresses and probably DHCP and DNS as well.

32:00
And then over here, I have all my separate application that's doing all of my physical stuff. And then maybe I have my circuits tracked somewhere else and I've got a spreadsheet over here that has all my wireless stuff. And what we try to do in that box is put all that in one place, right? So you have one cohesive data model. Everything's interrelated. When you build a network, you model a network in Netbox, you start by creating a site and then you create a rack that's assigned to that site. And then you create a device type and you create a device of that type and you mount that in a rack.

32:29
all virtually in Netbox, at the devices interfaces, and you assign IP addresses to those interfaces. And it's all designed to mimic the real world as closely as possible. Now, there's certainly challenges and caveats to that statement, but that's been one of our design tenants from day one, just trying to have the model as close to the real world as possible. And the reason I say that is a lot of applications don't do that. They skip steps. They abstract things, which turn out to be critical.

32:57
network engineers that other folks might not appreciate. For example, in a lot of applications, you might assign an IP address to a device. Well, we know we don't assign IPs to devices. We assign them to interfaces, and interfaces go into devices. It's a pretty big distinction when you have to work at the network layer. But a lot of those tools weren't really built by network engineers. They didn't have an appreciation for that kind of detail. I think that's where Netbox gets a lot of its early foothold is people realized, oh, this is actually I can model my network and not just kind of have to pretend.

33:27
How did it evolve then? So you released it open source and it was just sort of a, sorry, remind me. So at the very first outset, what did Netbox look like? And then how did it sort of evolve? Was it just feature requests from people, feedback? How did that go? Yeah, when we released version 1.0, it was pretty sparse. I don't remember exactly what was in there, but it was basically just IPAM and some sites. And I think maybe, I think we had devices and things at that point.

33:54
And then yeah, it was a ton of feature requests. So we had a backlog of features at that point, because by that point when we released it, it was probably had been under development for about eight or nine months. And so we had a pretty good backlog, but then we had people from the community. Initially, there's like a deluge of just feature requests. This is great, can we do this? Can we do that? And that's something else I should point out real quick as well is I had the unique advantage again, because I was fairly well known and I was still doing my Packet Life blog at that point. So I had a pretty good following.

34:24
So I had an audience already primed to release Netbox too. And this was thousands of network engineers, people who, of course, would be the ideal user for this kind of application. A lot of people don't have that luxury, so they'll struggle to, not struggle, but it'll be more challenging, certainly, to get your project out there, to get people interested in it. But that's where you try to ask for feedback as much as you can. So I had that.

34:51
going for me, certainly. And so we had the initial deluge of feature requests and bug reports and feedback. And that really helped shape the early stages of the application. And then we realized quickly, well, we're onto something here. A lot of people are using this, are interested in this. And it's challenging at times, too, because given the nature of Netbox, it's inherently prone to scope creep, meaning there's always going to be more that people will want added to the model.

35:17
We've certainly built it out over time and we'll continue to do so, but it's very easy to fall into the trap of saying, let's bolt on maintenance windows and over here, let's track circuit contracts and things like that. The thing is, there's very valid arguments to add those things to NetBox. Those are things we as network engineers need. But you can only do so much at once and you have to be cognizant of everything you add increases your perimeter of development and the burden attached to that.

35:44
That kind of reminds me of something I've been wondering about here and there. Like how do you, when you have a product like this, how do you sort of draw the line or set a boundary between like, okay, we're going to implement a feature that does the thing versus we're going to make it possible to integrate with our product, you know, from somewhere else so that that thing, that external thing can do the thing you want instead of our product doing it, right? Like where do you draw that line?

36:11
I don't know if this is an appropriate question, like right at this moment, but I'm curious for Netbox. Yeah. 100%. It's a great question and I wish I had an answer to it because that's what I struggle with every day, honestly, is where do we draw the... And the thing is like drawing the line today doesn't mean it stays there. There's a great saying in open source, I forget who the quotes attributed to, but it was basically, no is temporary, yes is forever. The implication being if I tell you no, we're not doing that today, that doesn't mean we won't do it next year or somewhere down the road.

36:41
But if I tell you, yes, we're doing that today, I have an obligation as a maintainer to the end user to support that functionality indefinitely. And that's the hard part, right? And that's why APIs change over time. We try to limit how much they change and how much we have to rip out of the application. But it does happen over time. We just try to avoid it as best we can. But to your point, that's one of the reasons we have a plugins framework for Netbox. So that's been great to kind of offload some of the more niche or.

37:09
lower priority feature requests and saying, listen, this isn't something we can take on a core, at least not right now, right now being temporary, but maybe you'd like to build a plugin for it. And most people kind of say, no, not really. Turns out they didn't want it that much. But some people do take us up on it and say, that's a great idea. And we have a plugin tutorial and everything that takes them through this stage. And of course people have wildly varying degrees of development experience and skillsets.

37:34
which is wholly understandable. But fortunately there's a great community around it too. It's very supportive. So if you do need help, there's plenty of people to ask. But that's, that's been a great proving ground as well, because we can have people develop plugins on their own, unencumbered from the release cycle and the restraints on the core project. Right. So they have their own release. They can crank out a new release every week. They're not waiting for us and figure out what works and what doesn't.

37:58
And then if it gets mature enough, we can look at merging that into core. If the, if the maintainer is agreeable, if they prefer to keep it on their own, that's perfectly fine as well. I've always wanted to ask somebody who created an open source project. It sounds like this thing took over your life. It sounds like you spend an enormous amount of time porting it and open source is free, correct? Yes. To all of that. So the obvious question is you've dedicated your life to this amazing product that is free. Is there a paid version? How do open source people?

38:27
pay their bills if 85% of your cycles, probably more, like your whole life is Netbox and it's a free product. I've never had the opportunity to ask a guy like yourself that has done this. Like how do you, I mean, the why is obvious probably. It's a passion project. I guess you love it. It necessity bred the need to do it and took off, which is great. You know, but it'd be nice to be able to, you know, pay your mortgage with it too. And.

38:53
I guess you can't, right? No, it's a right. So the question you're getting at is, how do you monetize open source? And if you Google that, you'll find a Wikipedia page. It's got about 50 different ideas listed. There are three core components from what I've seen. Now, I'm going to caveat all this with saying, this is my personal experience as one person with one project. Certainly, other people will have other opinions and probably offer other ideas that I'm completely blind to. But my understanding is there's basically three core

39:21
business models to it. One is open core. So you release the open source project itself, but all of these added features you're gonna pay for, you're gonna license them separately, be they plugins or additional functionality or whatever. One is the hosted or managed or cloud model, however you wanna look at it, just basically saying, I'm gonna release this, the product itself I'm going to release, but the much more polished version of that product is available as a SaaS platform and you're gonna pay us for that.

39:49
Uh, and the third is offering a support. And I think that's where a lot of people, especially individual developers really start out is I'm going to produce this, but if you want support, I'm going to, I mean, I'm going to ask you to pay me money shocker, uh, for my time and effort to actually provide you support. Otherwise it's, it's, um, you know, the best effort kind of thing. So you buy a support contract, right? I mean, that's kind of like what Red Hat does, right? Like it's, yeah, exactly. Then you, you buy support. Cause I guess if you're running this thing in prod and your whole.

40:14
Infrastructure is dependent on this thing working. You're going to need support at some point, I guess, right? You can't just sit around and wait for a bug fix while your automation can't talk to your source of truth. Like that's not good. So I guess big places that leverage an open source solution by support. That's probably the model I'm guessing. That's, that's one of it. Yeah. So like what we're doing with Netbox Labs is offering Netbox as a managed service, as a cloud platform and at least to start and the end goal there is working toward, um,

40:42
automation as an automation platform in the cloud. Because as I'm sure you all know, at least from my perspective, I love using network tools. I don't like so much the care and feeding of them. I don't enjoy being a systems administrator. I'd much rather just have the tool work every day when I show up. So trying to fill that gap and that need on the network engineering side and let network engineers get to do what they do best. So you won't have to maintain this cloud version of what you call it, Netbox Labs?

41:08
Yeah. So Netbox Cloud is the product. Netbox Labs is the company. Somebody's going to maintain it, right? I don't want to care and feed for it either, but if you're putting your product in a cloud platform, will you have to care and feed for it there? No, it's operated more like a SaaS product. So backups, upgrades, maintenance, all that stuff is managed for you. You just have access to the application. See, Andy, that whole support model...

41:35
interest me because you have something that's an open source product and then a company will take it on as a support model. But I guess I don't, maybe you can help us Jeremy. I guess I don't understand how that works because you've got a company that's providing support, but at the end of the day, it's still an open source product, right? That can change with different developers adding in features and that kind of thing. So how is the support arm supposed to keep up?

42:02
if you have a bunch of different developers that aren't necessarily affiliated with that support organization changing the product? Well, typically you'll find that most projects answer to a few maintainers, so a very small number of people. You'll have community contributors, certainly, but those folks are more concentrated on the one-off to two-off contributions to the project, which is certainly valuable, don't get me wrong, but you have a few stewards of the long-term roadmap of the project and everything. Those are typically the people either providing support directly

42:31
Or in the case of a larger company working for a company that's providing that support and they're on that company's payroll. So if you look at like Red Hat, for example, right, Red Hat controls, Red Hat Linux, it is open source for the most part, at least as far as CentOS goes. Or I think that changed recently, right? It's something different now, but they follow that model where they're basically licensing and, you know, a number of hosts for which they provide support. And in turn, that money gets funneled in part to producing the product. So you created Netbox.

43:00
It became a large success. Have you ever gotten the itch? Has anything else come up? Uh, any other type of potential product open source that you have thought about jumping into, or do you, do you have anything cooking now? Or is it, is it all in that box? I'm still very much focused on that box. And I hope to be able to, um, you know, through net box labs and building out a team to help me with that, um, I hope to be able to.

43:24
At least divvy up some of my attention to other initiatives as well, because I mean, the network automation space is amazing. There's, there's so much going on right now, uh, in terms of different tooling and, uh, you'll find different tools at various stages of life. And it seems like every other week, someone's coming out with a new tool that says, Hey, look, look, I made this. And, um, you know, those are the people who, who fortunately are, uh, they might be coming out with something half baked, but a year from now might be the go-to tool. You know, you don't know, right? Um, you don't know until you do put it out there to our earlier, you know, earlier conversation. Um.

43:54
And there's so much else going on and so many integrations. And unfortunately, you do find a lot of these open source tools struggle for maintainers. You'll see that on GitHub constantly. We see that in networking. We, I see that on the Django side. Some people have come out with like amazing Django libraries to enhance the framework and, but you'll see, oh, maintainers wanted, or a project is in like life support mode because the person who started it back when they were at college, well, it turns out now they have.

44:18
A job and kids to feed and things like that. So other demands on your time, right? I have a question. What's a maintainer? Oh, sorry. That's merging like the requests into the main branch or like, are you in charge of the changes? Yeah. So I don't know if there's a formal definition, but I think the common definition is basically someone who's charged with the responsibility of ensuring the, the integrity of the project. And, um, most open source projects, smaller ones to only have one maintainer. It's the per typically the person who created it.

44:47
As they get bigger, you'll get more and more maintainers. People will join as maintainers, they'll leave over time. Netbox currently, I think we have five or six folks designated as maintainers, two of whom are Netbox Labs employees. So it was myself and Arthur Hansen are both full-time employees, which lends a little bit of backing to the project, right? The more, the greater the number of full-time maintainers you have that are actually being paid to maintain a project, inherently the health of the project is ostensibly going to get better.

45:17
And then we have community contributors as well, or community maintainers who volunteer their time. So not being financially compensated, at least not directly for contributing, but are maintainers nonetheless. Those are typically the people who are approving pull requests or merge requests. Okay. That was my next question because you lost me with maintain the integrity of the project. I'm like, so what happens if there isn't a maintainer? People are mining Bitcoin on that box. What is this integrity you speak of, of the open source project? But I guess...

45:46
Somebody has to be at the wheel, right? Somebody has to be in charge of the decisions that are being made. Yeah, I think a maintainer has many hats, especially as a one-person project. They are the lead developer. They are the product head. They are the community head. There's every different aspect of it. I've tried my hand at all of them. Some I'm better at others. I'm better at than others, certainly. But.

46:14
You know, with a community like net, the size of the net boxes, it's definitely, you know, the stage where we're growing. Um, and, and fortunately being able to fill those roles with additional people who are better suited to it than, than myself. And how do you pay maintainers? That's through support, right? That's where the revenue comes from to pay people. Uh, again, depends on the, on the project, right? Uh, some of it's support, a lot of maintainers to be clear, like are not compensated at all for smaller projects. Even shockingly large projects often don't have.

46:39
anyone behind them, especially if you look at the... So with Netbox, we're kind of spoiled because we are a user-facing application. People know what Netbox is. They recognize the logo. They use it. They have opinions about it, good or bad, but they know what it is. Compare that to a library like OpenSSL. Most people don't know what OpenSSL is. They just know that it's there, and if they need it, they install it. And presumably it comes from someone. Well, it turns out that someone is a shockingly small number of people. I'm just using them as

47:09
that do need to do something eventually with that. And we've had, there've been issues lately where, I forget the exact thing, I think it was in the JavaScript world, but basically one of these small JavaScript libraries became very widely used and it was becoming unmaintained. And someone reached out to the maintainers that, hey, I would like to take over your project. And they said, sure, that would be great because I would love help. Turns out that someone had nefarious intent and slipped malicious code.

47:39
into these packages, which then were released. And then people were raising hell saying, how could you do this? Why would you give that someone access? And they said, because they offered to help. Wow. Yeah, that's unfortunate. So it's the nature of open source. You know, it's kind of, it comes with a territory. I mean, ultimately at the end of the day is you can either opt to use it for free as it's been provided and contribute to it, or you can opt not to. And you can pay for closed source software. That's an option as well.

48:05
interesting life cycle of Netbox. I want to go back really quick because I feel like some of our audience might have this question, if you don't mind. How did you get such an audience for your blog? You started it just sort of on a whim. How did Packet Life become so popular? I can see why. We can all see why. It's incredibly helpful. It's got useful things. It's very informative.

48:33
But how did people start to see it in the beginning? Do you know? To find the popularity first, because I thought I saw something like 100,000 views a month or something insane. Like, where did I see that? Did you say that? 100,000, it's in the about section. You actually laid out in there, right? Okay, yeah, it must have been the amount of, like, oh my God, that's amazing. 100,000 a month. Back in the day, maybe not so much recently. It's been pretty stagnant for years, unfortunately. But yeah, I don't know why it caught on initially. I think it became...

49:00
I know some of the most popular blog articles were my early ones where it was basically, here's how you do X. It was very unintentionally very search engine friendly because it was people searching for how to do X found a blog post titled How to do X and it turned out. And then of course the cheat sheets. Those were surprisingly good at just...

49:21
getting the site out there and people would see them on other people's desks and say, oh, pack a lot of stuff. They're so incredibly valuable. Yeah. Because like you said, there is a lot of effort actually involved in making those, right? And a lot of people probably don't have the time or don't want to put in the effort, right? So to have one to study off of that you don't have to make, it might not make you, you know, it might not give you as much information, have you studied better as if you were going to do it yourself, but it's still, you know, it's a good study aid. So I can see why. Certainly.

49:51
I got a question. You're a network engineer who successfully pivoted to software dev. So, you know, a lot of what we've done in a lot of these episodes is, you know, what would you say to like a newcomer or, you know, how do you, like, you know, how do you help somebody come in? And so like, I'm a network engineer who really struggles with everything dev. Just, just all of it. It just, I came up, you know, I, Python didn't come easy to me like it did to you. And like, I just, it's, I have a block with it. It's hard. I've tried, right? I mean, do you have any advice for.

50:19
Because now, you know, network engineers are surrounded by automation on all sides, everywhere. And basically I feel like it's, you know, you better learn this stuff or you're going to be a dinosaur or, you know, there's the dev skillset and coding and automation, all that stuff. Like you got to learn it soon-ish. You know, if you're coming out of school or if you're new and you're trying to get in, like I don't, you know, maybe it's not now, I don't know if it's in five years, but the industry is changing. And

50:46
Is there any advice for a network person who was like, Oh man, this shit's hard. How do you, or maybe you don't cause it came so easy to you, but it's, it's been hard for me and I'd love to be a better at it. Yeah. First, let me, let me say like, take a step back and look at what we ask of network engineers today, uh, go memorize how this protocol that was developed in the 1970s, uh, how that works inside and out, figure that out, memorize that.

51:13
Also know what Cisco's coming out with next year and plan your budget for that. And everything in between, and it's like, it's a ton, right? We have specialties within networking, right? You don't see someone who excels at wireless and data center and voice, right? These are all different disciplines. There's different CCIE certifications for a reason. And I think this is, the way to view it is this is an extension of that, right? I don't think everyone needs to be a developer. I think it helps. I don't think you need to transition to the software developer the way I have.

51:41
You know, familiarize yourself with what's out there and the opportunities that you have to learn more, certainly. Right. But like voice, I learned a little bit about voice, didn't have a passion for it. So I didn't pursue it any further. I've still got employed, right. Even before Netbox, right. But learn enough to know what your options are, to know, to have the presence of mind to say, when you find yourself stuck in something, to say, hey, I know how I can make this faster. If I can spend an hour writing a Python script to save myself a couple hours, that's a win.

52:11
Right. And own that and just say, Oh, well, I'm not a real software development. Doesn't matter. Right. There's no real software development. Are you a real network engineer? Who, what they did someone with a, you know, an official hat show up and hand you a certificate saying you're a real network engineer now. It didn't happen. It's not to me. I maybe might get lost. I don't know. But, um, I, I, I laugh at your explanation of the expectations of network engineers and, and how we need to know this, that, and the other, because that explanation sounded like you just wrote the script of.

52:41
one of Lexi's TikTok videos. I feel so validated right now. You have no idea. Which is all my... And that's the source of all my whining and crying with the automation stuff when it hit because I'm intellectually and emotionally overwhelmed at the rate of change in this industry. And it all started hitting at once. You had to learn cloud, you had to learn automation, you better learn Python, Ansible's there, but then there's this other thing and we're doing Netbox. Like, I mean, you know, just name it all, right? And it just...

53:09
I got overwhelmed and kind of just like my head exploded and you know, maybe it's just eat the elephant one bite at a time, like you said, like, do you have a use case, you know, I, I remember my last job right at the end when I was in prod, we had like 500 devices that we had to update an SNMP community string on. Oh my God. Do you have any idea how long it's going to take me to log in and do that damn thing 500 times? So I had a senior guy showing me how to do it in Python and it was like, okay, this is, I get it, I see the value in it, but then I get.

53:38
I look at Python and I'm just like, Oh my God. And, and, and another buddy said, you know, if you can, if you can manage the Cisco CLI, it's it's programmatic thinking. It's like, so, but for some reason I just get like, Oh my God, is it a space? Is it a tab? Where's that thing? Or if it, you know, was it a bracket? Was it a curly thing? Like I just, I get all freaked out by it and, and that's, that's unfortunate. Yeah. You know, it's just a block that I have. Yeah. I got a lot of it's just, just experience right until you have the.

54:04
It's more natural for some than others, certainly. We all learn in different ways and have different styles of thinking. But yeah, a lot of it's just repetition. I'll say as far as eating the elephant one bite at a time. And one of the things I see people kind of fail at all the time is we call it premature optimization. And they say, oh, like your example, I need to change the SNP string on 500 devices.

54:31
Well, clearly that means I need to set up Kubernetes and install a few Docker containers that I'll have running microservices so that they can be up. You're not building the next Google, right? If it takes an hour to do it, but you don't have to be at the terminal, that's a win. That's automation. It doesn't need to happen in two minutes. It just needs to happen without you pulling your hair out, right? Yeah. So get those easy wins. It's the Pareto principle is everywhere, right? It's the 80-20 split. It's 80%. You can accomplish 80% of what you need to do with 20% of the time.

55:00
And then it's like a sharp curve up for the rest of it. Stop before that sharp curve and you'll be a lot happier. Great advice. Thank you. I like that. I will try again. A lot of, I mean, it's, I think the best advice I could offer for that specifically is like focus on one thing, make it an attainable goal. Just focus on getting that.

55:18
And then once you've got that, you start that feedback loop where you get the endorphin rush. Oh, I made it work. If the thing works, then you add onto that slowly, you iterate slowly over time. Oh, can I add a command line argument to do this? Can I take the JSON and put it in a different format or something like that? And don't, don't look, people trip over their own feet, trying to run down the road, say, oh, I'm going to go build all this stuff. And then they get burnt out. And then it just leaves such a sour taste in their mind. They don't want to do it anymore. So it should be fun, right? At least at the early stages, it should be fun.

55:47
Jeremy, this has been a fantastic conversation about your career, about Netbox. And, and at the end, there's some great advice for people trying to break in or changing what they're doing today, or maybe think about things like software development, or at least the basics of, of just find it seems to be something that, that constantly comes up, um, with the different developers we've had on yourself, John Capobianco.

56:11
have a use case, have a goal in mind. Otherwise you may be setting yourself up for failure. So thanks for all that. Was there anything we missed that you wanted to cover in this? I'll just do a few quick plugs. For anyone learning Python, No Starch Press has some excellent books on Python. They do a great job, the authors that they have, of breaking down small tasks, like automate the boring stuff.

56:35
There's probably a dozen different Python books. Those are great because they break things down to easily achievable pieces. They walk you through every step of the way. That's a great way to get started. The other one I'll plug, which I love, is anyone interested in open source, there's an excellent book called Working in Public by Nadia Engball at GitHub, who I wish I had read that before I even started Netbox, because it lays out the current landscape so well. There's older books like Cathedral in the Bazaar, but those are

57:03
pretty far out of date, the open source community and ecosystem has shifted pretty substantially since that was written. But Nadia does a great job of kind of like, I remember just feeling so validated as a maintainer realizing like, oh, it's not that I'm doing everything wrong. It's just that these are challenges of open source. So it's a really, really great read. And I almost forgot, I almost forgot the most important question of the episode. How do you really feel about Jordan Villarreal? Jordan's awesome.

57:31
I mean, Jordan has been a phenomenal value. Uh, he's, we, we joke that like, we need to hire another Jordan all the time because, um, yeah, they get the guys awesome. Really? Cause we hate him. I was waiting for an answer like that out of Jeremy. No, no, I can't even. Well, yeah, no Jordan, Jordan's get that rare combination of, of personable and professional and just like.

57:59
That we struggle, but yeah, if you find any more of him, let me know. Sure thing. Well, where can people find you on the internet if they want to learn more? Is, is packet life still the best way to find Twitter? What would you say? Yeah, I'm J stretch 85 on Twitter or of course on GitHub. Most fittingly. If you pop on over to the net box project on GitHub, you'll definitely find me there or the net dev slack. If you go to netdev.chat, we've got a nice community slack. It's home to net box and a few other.

58:29
open source projects focused on networking, automation, and monitoring. Excellent. Well, Jeremy, thank you for joining us. Thank you all to all of the listeners for joining us on the art of network engineering. You can find us on Twitter at art of net Inge on our website at art of networkengineering.com. Also check out on Twitter at cables to clouds and check out www.cablestoclouds.com for all their new episodes coming up.

58:56
Lexi, Andy, thank you both very much for joining me again. It's always a pleasure to see you and we'll see you all next time on the art of network engineering. Hey y'all, this is Lexi. If you vibe with what you heard us talking about today, we'd love for you to subscribe to our podcast in your favorite pod capture. Also go ahead and hit that bell icon to make sure you're notified of all our future episodes right when they come out.

59:20
If you want to hear what we're talking about when we're not on the podcast, you can totally follow us on Twitter and Instagram at Art of NetEng. That's Art of N-E-T-E-N-G. You can also find a bunch more info about us and the podcast at ArtOfNetworkEngineering.com. Thanks for listening.