Intro

andy lapteff 0:16

This is the Art of Network Engineering. Where technology meets the human side of IT. Whether you're scaling networks, solving problems, or shaping your career, you've got the insights, stories, and tips to keep you ahead in the ever-evolving world of networking. Welcome to the Art of Network Engineering podcast. My name is Andy Leptev, and I am joining you once again from Munich, Germany. And I have not one Bart, but two Bart. Hi, Bart. Hello. Even in Munich, this is amazing.

Two Barts, one Munich hotel

andy lapteff 0:48

So if I don't tell which one to go first, they're probably gonna talk at the same time. So, Bart, would you like to tell the people who you are and what you do?

Bart Dorlandt 0:55

Yeah, sure. I'm um Bart Dorland, um from the Netherlands, and I've reached out to you to have this uh this chat while we're both in Munich. Bart number two.

Bart Smeding 1:04

Yeah, I'm Bart. Bart's me. I'm also from the Netherlands, uh network automation consultant. And uh yeah, we're uh mostly traveling uh with both of us.

andy lapteff 1:14

So you are network automation professionals. You are going to be my um experts. I can barely spell Python, so I am going to be uh picking your brains. Thank you for being here. It's awesome to see you again. As you may or may not know, I have, I don't know, roughly 15 years experience as a network engineer, and I struggled with automation. I struggled with learning coding and dev, and I took a bunch of classes and read a bunch of books and just really struggled to internalize, I guess, all the concepts. And I was mentioning it to you before the recording started, like the levels of abstraction. It seems like I just get to a point in the curriculum where something in my brain breaks, but there doesn't seem to be enough time in these boot camps to like just slow down and like wait, my brain just exploded. Can we just review where we are so I can get re-acclimated? Now we're at Autocon 5, and these are people interested in automation, right? And I think they're the minority. I know at Autocon 4 they do that survey every year of who is, who isn't. And I thought the survey results were something like 60 or 70 percent of networks aren't automated at all or in any meaningful way. So I take that to mean that most traditional network op, you know, engineers like myself aren't automating at all. That's weird, right? Because I think everything else is automated, like Linux server guys have been and people have been doing it forever. And like I go to the supermarket to get food and like I can do self-check, like everything in our lives is automated. There are cars that drive themselves and don't run over people, but for some reason in networking, we're not doing it, right? And we have all these reasons. And for me, I've said it on the show a million times, so I won't bore you, but I failed out of computer science. I thought I wasn't smart enough, I couldn't figure this out. And now, years later, I'm again about to sit through a two-day Python dev class that I'm really interested in. I really want to learn this stuff. And I've been able to code a bit with Claude Code's help. So I don't even know if you guys would call that coding or if I'm just like some vibe imposta, right? But I thought what we could talk about here is your experiences with being the um network automation experts, and it seems like in your jobs you interact with companies and network engineers who want to automate, are told they need to automate. And then sometimes when you go in, the practitioners don't want to automate. So I think you have an interesting perspective on like you get to go inside, right? A company calls you and says, Hey, we'd like to automate, come on in, and you talk to them and they buy in and you get a contract and you go. And then I guess were you saying that like you'll go in and the network engineers aren't

The easy button nobody gets

andy lapteff 3:53

interested?

Bart Smeding 3:55

They are interested, but not doing it by themselves. They expect to build an automation platform that will do everything, and they don't have to look at it.

andy lapteff 4:04

So they just want to press like an easy button, just make the thing happen. Yeah. Which is super complex, right? Like automation platforms aren't something you can just easily spin up deploy. Like, do they need to understand? Here's where I think we should start. What level of knowledge does a network engineer need to be able to start becoming proficient in automation? I think they need to learn Python. Am I right or am I wrong? Is that Python will be to start? Yeah. Python's where you start. Do we say Python is where to start because it's the easiest to read syntactically? Like it's most like the English language. Does that sound right?

Bart Smeding 4:41

Yeah.

andy lapteff 4:42

Comparatively speaking. Because I have friends who say, no, you shouldn't use you know Python, you should um should learn Go. But I think Python's easier to start with. For someone who's never coded, does that sound accurate?

Bart Smeding 4:52

I think it is the best way to start, yeah, for my opinion, but there will be different opinions, I guess. Unless you want to spell it and go shorter.

unknown 5:00

Yeah.

andy lapteff 5:01

So I really liked Ansible when I got introduced to it because I didn't have to learn Python. And I could create a playbook and a config file and I started with some config.

Bart Smeding 5:10

Okay. Yeah.

andy lapteff 5:11

So tell me about Ansible. What is your the the hardcore people are like Ansible's running Python under the hood, so just go to Python. But if Python's hard and people are struggling with it and they can get some success with Ansible, is that a good place to start?

Bart Smeding 5:25

That is a good way to start. Yeah. If coding is not really your style, or you don't don't really understand the code, then you can start with Ansible. It's more easy to uh create some playbooks that will do some automation on your network stuff.

Bart Dorlandt 5:38

But still, right, if you look back years of of history, right, especially if I look in the Netherlands, there's a lot of companies that that pretty much only run Cisco, right? And Cisco has been feeding UIs, right? Traditionally just CLI notepad that you do your thing and then you get you get somewhere, which is on its own kind of a sort of programming with a lot of different syntax, right? You get to do stuff, you give some syntax, and it does BGP and you have a whole protocol working, right? Then you had products like Cisco Works and you had UI, but even a lot of programming was always kept away from the engineers, right? They kept doing the networking and the design and architecture and seeing how that network specifically worked for the business. While at the same time, if you look at at Wi-Fi, right, there's been controllers for a long time that do things automatically in a way where you have one central point of input and your whole network is working, right? I might be oversimplifying this, but it's far more orchestrated than the physical LAN is or has ever been. In the end, there's so much difference that you can apply and so much weirdness that that you might need or will need to get your networking done and to make it work for your business, because in the end, a lot of businesses

Wi-Fi is solved; the LAN is not

Bart Dorlandt 6:52

are different.

andy lapteff 6:53

I have two comments. I've heard other very smart people say that, well, we have controllers for Wi-Fi and we can control those networks programmatically, so it shouldn't be that hard to like cheer point to a LAN or a data center. And what I hear often is that, well, Wi-Fi is way simpler than like a data center fabric, as an example, right? Like you either have RF or you don't. It's either on or it's off. You either have a weak signal and you have to adjust power and roaming or not. Whereas, and I don't know if this is true, I'm not saying this is my, but you know, right now we're on Wi-Fi. Like I have the SSID, I put in the password, it works, and it's gonna let me roam. And like, I think the Wi-Fi problem is solved, I guess is my point. Like, there are good automation controllers that you can just buy and Wi-Fi kind of works out of the box. Is that a fair thing to say? It's a fair thing. Because when I think of automation, I don't think like, oh, we need to automate Wi-Fi networks. It seems to be the LANs and the and the data centers, and you know, to go deeper into the network and automate those in a way that's as easy as like I've used a lot of different, I've used um well, I don't I don't want to name drop vendors, but I've used a bunch of different Wi-Fi controllers. It's lovely. It's a UI. You click it, it works, it's poof, it's magic. I have heat maps, like, oh good, right? Oh, somebody had a problem. Oh, I can see the signal on their phone. Let me out adjustment, auto fixing. It kind of seems a little simpler. Whereas, like, oh, I'm deploying an EVP NVXLAN, you know, fabric with spine leaf, and I have all these compartments, like it But do you have that on

Eight months for the CLI, never eight for Python

andy lapteff 8:16

the Wi-Fi? Probably not on the Wi-Fi. Well, that's what I mean. It's like an end user, it's simple, right? The the other thing I wanted to say is, and I'm glad you said this. I worked with a guy, Chris Margette, and he's a big programmer guy. And he used to tell me all the time, because I was a I came up on Cisco like everyone, and I I would generate 20, 30 page notepad plus plus scripts for like very long, you know, data center changes we would make. And he would tell me, You mean to tell me that you can write a 30-page Cisco CLI document and you don't think that's programming, but you're telling me Python is so hard. For some reason, and it's probably I'm having this realization now, but I spent eight months in Cisco Neticad. It was a four hours Friday, four hours Saturday. So eight hours a week for eight months learning Cisco CLI. That's a lot of time. You eventually pick it up. I don't think I've spent anywhere near that much time with Python. So I guess to Chris's point and what you said, like, yes, Cisco CLI is basically programming, and I don't understand why Python seems so much harder to me. And I think I'm just having this realization now that I haven't spent anywhere near the amount of time with Python. Where would you point people to learn Python? Because there are a hundred different courses on Coursera and there's the Kirk Byers one, and like I seem to have taken them all and I've struggled with all of them. Do you have like do you guys teach? Do you have your own class or do you direct people? If I were to ask you what I should start with Python on, what would you recommend?

Bart Smeding 9:45

Uh I started with the Kirk Byers uh as well. Yeah. Uh after I did the Ansible stuff. I want to dig in more to the Python because I want to create uh uh modules in Ansible to do more stuff than the standard modules, and then I started with the Python. I started also with the PCAP training and uh certification and not the networking PCAP. That's what I was gonna ask.

andy lapteff 10:11

I'm like, are you feeding Wireshark PCAPs and Python? Python. Yeah, and uh I'm sorry, what's the PCAP? Is that a Python? I don't know what that is.

Bart Smeding 10:19

Uh the entry and uh advanced uh certification okay, yeah, yeah. It's free to uh free to start, uh and afterwards the kid buyers one and that helped me to start up with the network uh network stuff.

andy lapteff 10:30

So PCAP is where you started? That's the name of the certification, that's like an associate level beginning, yeah. I know three. I've never done it actually, but okay.

Bart Dorlandt 10:38

But I guess we're we're glancing over the point where which should be highlighted is he had an objective, right? He had a use case where he wanted to go towards on on hey, yes, I've been using Ansible, and yes, I want to do more, right? I want this that it exists of modules behind behind the curtains, and I want to I want to build my own, I want to achieve something to do that, right? And that is the greatest motivator to get stuff done, right? You have a use case, you have a you have a goal, you work towards that. I had the same thing when I started, and this was way back, right? I started doing at a a client, and they had this massive, massive network, and uh BGP all over the place, and everybody, yeah, there were a lot of designers and putting configurations in there by hand, notepad, everything, the the usual again, years back. Everybody put stuff in, right? So everybody had a bit of oh, let me enable OSPF and this link, and we're gonna extend that, and pure fine, and then we have to advertise that into BGP. Okay, let me put in a network statement for that, and that was done all the all the time, but nobody dared taking stuff out. That's where I started. I was like, okay, I got some time left. I didn't know Python at the at the start, so I started with bash, and I was like, okay, give me give me a show route of the thing, see what's in there, active, compare that towards all the network statements in BGP, and I did that because I didn't know any better. Um, and only later on I was like, oh, maybe maybe there's a better way, right? I I

Five maintenance windows a week

Bart Dorlandt 13:07

heard about Python, let me dive into it. And I I just endlessly watched videos and doing things. But again, when you have a goal, then then that makes it so much easier to not lose your focus and just keep on going, like, I want to achieve this. How can I get better? How can I get the job done?

andy lapteff 13:24

You hear that a lot, find a use case, right? And and I I've said this a million times, but when I was at a company and they said we have to learn to automate, the challenge I had was at the time, I think I was working four or maybe five maintenance windows a week. So that's like five nights a week after working nine hours a day just to keep the lights on. Regular moves, adds, changes on the network. And to try to, and this isn't like me complaining or whining, but I wasn't sure when I was supposed to learn a programming language, right? Um, that seemed very difficult. And I guess in tech it's like, well, boo-hoo, nights, weekends, holidays, this is what you signed up for, this is what you have to do. And and I didn't because I was just so burnt out from all the maintenance work. And similar to the point earlier on the CLI, I had to spend more time with this material to really internalize it and understand it. And I didn't feel like I had that time. Were did you feel like you had enough time to learn this stuff? Or were you just staying up late every night until you understood it? Like what's the sacrifice that you had to make? Yeah, yeah.

Bart Smeding 14:30

And I think also uh you need to uh become familiar with it and uh see the benefit of it, then you will start loving it and uh do more and take more time for it. Um, but it took a lot of evenings and weekends, yeah. There's no show. I saw the benefit of it, and then I uh I programmed first one one one uh small thing, and then it would come bigger and come bigger, and then I thought, wow, what I know push the script and it do something in two minutes, what I before did in three hours. It's time, right? That's the magic. And then I say, Well, yeah, I I get time back. And put a lot of effort in and you get time back.

andy lapteff 15:09

And that's what I'm digging at the whole reason NAF exists. Like, why if you can create time, if you can do three hours of work in three minutes, like this seems so intuitive. Why wouldn't you do this? And the only conflict constraint you know, point of friction I had was just the because at the time I was also studying for my CCMP. So like I'm already studying an hour a night and sometimes five o'clock in the morning, trying to like get a different certification that I was starting. And then I have to now fortunately there's coding and dev stuff in the certifications. It wasn't like what I was learning. But I when I was a cable guy climbing poles, getting hurt, not getting paid enough, and I met this girl who's now my wife, and I wanted a better life, I wanted a better career. So I was motivated, man. Like I have to get out of the field, I have to make more money, and I love this networking stuff. So I worked my ass off for probably the better part of two years to get my CCNA and get my first network engineering job. But once I had that job, someone telling me I better learn Python, it didn't feel like the stakes were the same, right? Like I had my career, I had my role. Now I have recruiters reaching out to me. Like I feel, you know, I have my wife, I'm now making better money. We can put my son into better daycare. So I don't think I was as motivated. I'm trying to figure out why we're not automating. And, you know, you could probably ignore automation for a while. People have been, right? Like you look at the the poor adoption rates. I know for me, unless they were like, if you don't automate something in the next six weeks, you're fired. Like I would need that kind of motivation, just like I did when I fell in love. And I'm like, I have to provide for our future family better. And I killed myself on that. I never put that kind of work in the Python. And I don't know if that's common for folks out there who aren't automating, but I never put the work in.

Bart Dorlandt 17:04

Okay. But in the end, to to achieve something, you need to put something in, right? There's no magic wand to to achieve that, right? Finance, you need money to make more money, right?

andy lapteff 17:15

And here you need time to to study and do but if most people aren't automating and I can keep my job without automating, and I'm already studying for other certifications because cloud's important, and my place says I need cyber, and now we have zero trust.

Bart Dorlandt 17:27

Like, you know, yeah, but I I'm sure there will be there will be places, right? But are you one in a dozen? Or are you the exception and are you skilled enough to be in that that different role, right? I guess that's that's also a factor, but still the one in a dozen still will be around for for quite some time.

andy lapteff 17:43

I think as long as people can hide from this, they will. And but that's choice, right? What's the ambition? So I spoke at Autocon 4 in front of a thousand people and it was terrifying. But the story I wanted to tell them was like the decisions I made to not prioritize learning automation stopped my career. I got laid off and I couldn't get another job in networking. And that might have been partly because I was not adopting the skill sets that are modern that you need. So I don't know if that would be if that would happen to other folks like me, but I have been out for a year and a half saying, do not wait for your career to be disrupted to then have to learn this stuff. Because as soon as I embraced Python and I started studying it, and I got a book and I started labbing, and I was able to get into a router with the NetMico library and SSHN and do a thing. And then I had Python study groups I was doing online. Poof, I get a job, a great job. You know, I covered somebody in NFD and I got a job, and now I'm working, you know, for an automation, a data center automation platform. But I can tell the difference between my career before I embraced automation, where I was like, no, I don't want to do this, this isn't for me, and how the industry treated me. And then once I embraced it and did the work. And my fear is that these LLMs are getting so good that the people that thought they could hide before are probably gonna be replaced by people embracing this tooling, which is getting so good, which is why I'm not trying to be the annoying guy who keeps making content about guys, you have to learn network automation. You're gonna be out of work soon. I know it sounds crazy. Well, I think there's a massive disruption coming because if you think about the business, like you need efficiencies. Why? So here's one example. I worked at a company and I had to update the SNMP community string on 600 switches, and it had to be done, I don't know, in like uh a week or two because of some security audit we had, something was vulnerable. That would have taken me three months because the only way I know how to do that is no Pet Plus Plus, log in, get in the jump box, do the show run first, show run pipe include SNMP, whatever, you know, like the the pre, the post, look at the log, make sure you don't break anything, and you can't break anything with it, but you still have to do the thing.

Bart Dorlandt 19:55

Well, you could at least do six at a time, right? In iTerm and then open six windows and paste that whole thing at the same time. But but six hundred, like it would have taken me forever.

andy lapteff 20:04

It's difficult to read six hundred tabs at once. And this and this guy, George Tempesta that I worked with uh knew Python and he took me under his wing and he spent a few weeks showing me. And what would have taken me months probably, we did in a few nights. We could have done it in one night, but we were trying to be careful, and it was a new script. And the time, man, like why would a company who works within a capitalist, you know, uh system that needs to show growth and revenue, why would they pay someone to spend three months doing something that someone else could do in five minutes? Now, before Python was too hard, and I think organizations empathize with like, okay, well, you're not a coder, I get it. Well, now I could do it with an LLM. I could probably generate that and test it and see it's fine and do it. And if I'm one of the people, I don't know if the people that aren't automating are also the people who aren't adopting all this AI stuff, but um it's it's just a long way to say that, like, God, I hope people start embracing this soon. I don't know how the industry is gonna change that 70-30 split. And I'm worried for the people like me two years ago that are gonna hide because well, I'm a networker and I don't have to know that stuff, and we'll get somebody else, and I'm not a developer. And like, I don't know how much longer the market and the companies and infinite growth in capitalism are gonna support the people who refuse to learn this stuff.

Bart Dorlandt 21:56

What are you going to do in 10 years then? Yeah. Well, right. I I like it But it hasn't it doesn't have to be binary, right? It's not all in. No, but it doesn't have to be all in, right? And that's that's where where people often get like, hey, you might have a suggestion. What people hear is like, I need to migrate myself towards that, right? And that's not it. It's it's about knowing a little bit more than yesterday and achieving a bit of a bit of code, a bit of do a function, right? Be able to write a loop, log into your first device using that Miko, and now you have, uh, which could have been in combination with with the person you just mentioned, it's like, hey, you know the syntax of the device, he knows how to write a loop and do that for 600 devices, right? But he needs a bit of your knowledge, you need a bit of him, and he can be your guide, right? He can be your guide towards the hey, how do I get more efficient, as you mentioned it? And and that's a great position already to be in. If you have somebody that you can learn from in this way, that already saves so many hours on on YouTube.

Bart Smeding 23:01

Yeah, and start small and extend it, and it's also good if the whole team is behind uh that that mindset.

andy lapteff 23:08

And and has your experience been in these companies, the whole team's into it? Uh I'm I'm asking you right. Uh I know the answer, right? And and and that's again, I think that's 70 30 split. Again, I'm not disagreeing with anything you're saying. I'm trying to nudge us toward you know how to get them there. If keeping your job isn't motivation enough, if doing three hours of work in five minutes isn't like it's it's such a weird I almost think this is like some cognitive bias stuff happening. Like people I know for me, until I lost my job, I think as humans, pain is what motivates us. We don't evolve because we're comfortable. Because in my mind, my story was I can't do this, I'm not smart enough. Now, once I logged into that router, it wasn't hard with a NetMico library. And I it was a class to on life food. And like, so to your point, I found a video and I started taking a class and I did it, but it did the thing. I thought that was I couldn't do it. Just like I thought I couldn't create software, and I've created five apps in five months, which is crazy. Now, again, I don't understand the code, but I'm doing work to try to bridge that gap. I'm eventually going to understand it. Something I wanted to ask you that you said about Ansible. Ansible is a very easy, to me, like entry point. And I and I like it. There's two things about Ansible I would say. A, how do you test your changes to make sure you're not going to break things? Because I think that's what's scary about automation for a lot of network people. You can look at some of the biggest companies in the world, and I'm not going to name them and punch down, but people with unlimited budgets printing money melt their networks, you know, once a year, once every other year. And it makes national news because 80% of the internet services go down. AWS East Plan. You know, um, and and I know it's complex. I know it's hard. I know there's a lot of technical debt. But when somebody like me is like, this has nothing to do with where I work, but a smart guy I work with Mike, he keeps mentioning, and I think it's really smart, that networks are fragile, like moving slow. Think about change management. Think about change windows, think about all the process that is involved just to touch a network. In my experience, a big thing happens, something breaks, and the response is slow down, add more process. Let's try to avoid this. So you go slower and slower and slower because you never know when you're gonna touch something in a network to either kick off a bug or something weird happens, or oh, that static route we didn't think was doing anything, black hold everything. So if if moving slow is scary because we break things, automation amplifies what you give it. Like you can break more things quickly. Let's say I'm someone that's like, okay, fine, these guys are compelling. I'm gonna try Ansible. Fine. I'm sick of Andy complaining about it. These barts are amazing. Let me try Ansible. How do you test your changes before pushing them to production? And because you're supposed to test stuff first, right?

Bart Smeding 25:53

To make sure you don't break it. If you start from the beginning and you start with Ansible, I will recommend to you start with uh not changing something, but gather information from your devices. Uh check

Read-only first

Bart Smeding 26:04

if there's a configuration drift to run that. Uh start creating playbooks to check if your network is uh healthy. Read-only. Read only. Yeah. And if it does all work, then try to change something small, like an SNMP string or it's how would you know if a read-only didn't work?

andy lapteff 26:21

I'm not trying to be provocative, but if it's a read-only, what would be a failure? Like you don't get information from a device. How would a read-only fail? How would I know it failed? Does that make sense? Like let's say I have 100 routers and I have to check an SNP computing string on them. I guess if I get 99 results, one of them didn't respond. Maybe there's an ACL blocking it, like, right, whatever. Like that's what that would be. I'm just walking through the menu showing. Right. Yeah.

Bart Dorlandt 26:47

Because people are afraid. Well, you say it really easy, right? Because you know you see it in front of you, you will see all the YAML syntax just flowing through your mind. Oh, yeah. I'm still trying to bring YAML. But I mean, uh usually what I guess the point Bart is making is is yes, you start small, right? Start on one device, do a read-only, and you verify by hand, right? Or with your own eyes and seeing that the output is what you expect, and and that you can do a bit more, but there's going to be a point that you don't want to do it by your own eyes and hand anymore, validating whatever, right? So he's pointing out that you want certain verification or validation steps in your playbook, like hey, I'm expecting a certain output which is larger than zero bytes or has a certain string in there that needs to be present for me to know that at least I got feedback from the device, um, regardless of the of the of the variable that might be in there, right? That you might look at your SP string or your your IP address, and you're not gonna match on the IP address because it's variable, right? It could be different on every box, but you know you're gonna look if you look at the SP community, you're gonna find the word community in the output, right? If you're doing CLI based. So I guess that's your point on on making the validation.

andy lapteff 28:04

Do you build that into the playbook and will Ansible tell you at the end? Doesn't it like print out some kind of report? Uh it's been a long time since I've done Ansible, but won't it tell you if weird things happen?

Bart Smeding 28:12

Uh if you're expecting something.

andy lapteff 28:14

Like can you build that into the script or do you have to manually read?

Bart Smeding 28:17

Like, you have to manually read it, yeah. Get information, save that, and then check if there's okay common string in it. But it's a place to start.

Bart Dorlandt 28:25

That's the output, right? But I guess you point out it's like the overview at the end of all your tasks or that it communicated and like oh it's something failed. Right, but it could send a command, which is valid, but if it didn't reply a response, then it's still valid, right? Right, right.

Bart Smeding 28:38

Yeah, but if it cannot log into the device, it will fail uh at first. And gathering facts is mostly the thing you start with, get information from the device.

andy lapteff 28:46

Right. So if it can't get in, I guess what I'm what I'm getting at with Ansible, can you build in let's say you can't log into a device? How do I know that failed at the end? I'll get a failure in that thing at the end. Yeah, yeah, yeah. Will it tell me what failed, or I'll just get a thing I'll have to look at? Like, oh, something weird happened. Let me dig in deeper. Or can you build that in, like tell me what failed? I don't I don't know how granular you can get with Ansible.

Bart Smeding 29:08

Yeah, you can grab the outcome of the SSH test and then you could print it out.

andy lapteff 29:13

I've heard people say that every time Ansible changes something, the playbooks break. Yeah, we have less one less year. Yeah. Why is that? Is it just these are software things I don't understand probably, but when I say Ansible, people go just do Python because they're gonna change and then the playbooks will break, and it's exactly the last couple of years it could have come a couple of times, yeah.

Bart Dorlandt 29:34

But it's not guaranteed, it's not guaranteed that the alternative doesn't break, right? So yeah, yeah, and they might say Ansible break, right? Then they say, well, SSH, the the the syntax of the vendor changes per version, uh right, and therefore your your CLI and your regex and whatever you have in your Python script will break, right? Then they say you have to go API, but the API, they do the same thing to the API as well, right? Or it it depending on the the vendor, the version, the the whatever you talk to, yeah, they might still they might have some rigorous versioning in there and they're very strict and that's very good, or they don't, and they uh they might have updated it uh in in whatever release, and then still your code doesn't still crashes, right? So you still need some some failure handling to deal with that.

andy lapteff 30:19

Failure handling. I've heard people say that before. Is that something that you build into the code? Like, does Ansible do failure handling, or is that strictly uh yeah, you need like a program? Yeah, you mentioned API. So so again, where I work, it's it's very

Everything breaks; you own the failure handling

andy lapteff 30:33

API driven. That the NAS is that the gold standard? Like you have Ansible, you have Python, you have different programmatic ways to communicate with devices. I I see a progression with people, they'll start with Ansible, and then for reasons, they'll go to Python and then maybe go. And like they, you know, they they go through this process. And is API the end? Like once you get to API, are you doing it right? Or is it not that cut and dry?

Bart Smeding 30:57

It depends on the fender, the API, the device. Like if it's a good API, is it that doesn't change and have a version is change, right? Like it that's but if they do versioning, then it's not a problem if you stay in the in the version that works. Oh but if they change it without versioning and it breaks, right?

andy lapteff 31:15

Defenders do that, they'll change an API, not version it, and then you wouldn't know, right? Your your scripts will break. Yeah, because they didn't tell you.

Bart Dorlandt 31:22

Yeah, well, or this version via the SDK that goes with it, right? But I guess all in all, it's a means to an end, right? You want to make your configuration change, or you want to read from the device, or you want to do something, right? And then telnet, SSH, API, REST conf, netconf, uh, whatever is is an a means to an end to get there. And one is more um programmer oriented, right? In structured output and having some kind of of JSON output or um or XML or equivalent that you get and that you can therefore parse it more easily versus the traditional API called CLI, right?

andy lapteff 32:04

Um I should say wait a minute, did you just say the CLI is an API? Yeah, of course.

Bart Dorlandt 32:09

Oh my god. I did not know that. Well, isn't it? Are you don't know? Aren't you aren't you as a human parsing the output on the screen to know what

The CLI is an API; you are the human regex

Bart Dorlandt 32:18

what it means? Yeah. So instead of a script. And you're you're you're programmed to read the show version to see what how long the device is up and which version is it it is running. You are the human regex.

andy lapteff 32:32

And that's where my value is, damn it.

Bart Dorlandt 32:34

And that's why I can't hand it over to automation.

andy lapteff 32:37

And the the human regex. How do you compel so Bart, you mentioned earlier? What what Bart? Smeed, Smed, how do I Smeading? Smeedy. Smeating. Smeeding. Sort of. You're not making the stinky face to me anymore. I appreciate that. We must be getting along now. For viewers who didn't know, Bart came in and gave meeting, gave me the stinkiest face. The first five minutes, he's giving me the I don't know about this Andy guy. I think we're better now. So Bart Stinky Face Meeting said earlier that um that's for the S then. I'm curious when you go into a shop, the the people engage with you, the the leaders of an organization, and we want, hey, we want to automate, and we heard of you, and great, we we we had a conversation, we're signing a contract, you're gonna help us. And then you I guess you interface with their network engineers, uh, you know, eventually. Okay, guys, the company hired me. I'm here to help you. You mentioned some of them are resistant to to this. I I get that. That was me. How did were you resistant to the person or the goal? Neither. I was resistant to the possibility that this was within my cognitive ability to understand and deploy. I didn't think I was smart enough to do it because of an experience I had young where I failed out of computer science. Now that was C. But that but that's the 19th.

Bart Dorlandt 34:05

That's the same attitude that you described before, right? Something happened, something broke, and people told you slow down. Well, yeah. That's the same people that didn't understand what the problem was. Right. They just said, well, hey, something broke, we need to look at it. Well, they didn't say we need to look at it, they they put the brake on it. And then and then afterwards, they're like, You're gonna figure out, hey, this has actually gone wrong. This is how we can fix it, and we can ramp up the speed again, right?

andy lapteff 34:31

Think about change freezes. So I don't know in Europe if they do the same thing, but in the states, there's a month, a year that we are told to never touch the network from Thanksgiving to right after Christmas, because that's the biggest spending, I guess, cycle

Change freezes and the feeling of safety

andy lapteff 34:45

in the states, and they don't want networks to break because they want now. I worked in financial services payment cards, so I don't know if it was more pronounced there. But change freezes, our industry has created change freezes to create the illusion of stability, if you think about it. And these aren't my words, these are somebody that I heard say, but it really resonated because it's the first time someone put into words, like, oh right. Like if we can't touch the network certain times of the year when the network has to be up, there seems to be an underlying problem with networking. And I don't know if automation can fix that. Or it's a good time to start learning Python. Well, so here's here's that cyclical argument I have in my head, and this I'm glad you said that. So how can learning Python help my network not be so susceptible to falling over every time I touch it? Right? Like, is it standardization? Is it more well? I mean, that's well, I think that this is the cultural element, right? Like, why won't network engineers automate? Well, every time we touch the network without automation, there's a 50-50 shot we might break something. That was my experience. Like, right? We it there's a whole culture, like send it, and like it just, you know, we don't know. We until we press the button, we'll see. So maybe we need better pre-validation testing, like pre, you know, pre-change testing. Um, I know that places I worked, we didn't really have a functioning lab to like test changes in. Like, state seems to be the missing piece. You can test a change in a virtual lab or in some routers and switches, but does that environment you're testing in have the current state of your network? Because without that state, you really don't know what this thing's gonna do. You know what the protocols are gonna do, but there could be some weird again. Like we used to step on there'd be an outage, and somebody put a static route in or weird ACL to fix it temporarily, and a temporary fix never comes out. And then, you know, I would invariably do a change that looked good. I tested it and whatever the hell I did, Evan G, like, yeah, the protocol did the thing. I think we're good on this version. BGP did you know what it was supposed to do. And then I would step on a landmine and all hell would break loose and something would happen. And they're like, what did you do? Slow down, nobody do any more changes, right? So I'm glad you said that. How does learning Python can learning Python and network automation help make my networks more reliable?

Bart Smeding 37:02

Yeah, I think it can it can be used for all kinds of testing before you can run Py test and all kinds of things.

andy lapteff 37:09

Uh tell me about PyTest. I'm sorry to interrupt you. So John, I know, is big in the Py or is it PyATS? Py test? I get all the PyATS.

Bart Dorlandt 37:17

PyATS. John is about PyATS.

andy lapteff 37:20

Okay. PyTest is different, right? Py test, yeah.

Bart Dorlandt 37:22

So is that the same thing or is that I'm doing uh I'm providing a workshop tomorrow on that. With ORS and PyTest or PyATS or is it the same thing?

andy lapteff 37:29

Py test. Py test. Tell me about PyTest.

Bart Dorlandt 37:32

Please. I'm I'm putting a question on the side for now, right? Okay. Have you ever looked at the stats on the other 11 months versus that one month of no touching the network, if they are actually more or less breakers during that? That's a thing to think about. Try to see if if during the rest of the year the network actually broke more often or things what is the comparison? It's something to look in uh in. Does it really help? Does it does the change freeze actually help? Right? Or is it just the idea of safety?

andy lapteff 38:06

But we'll put that aside. It's an interesting point.

Bart Dorlandt 38:14

Yeah, or does it need continuous work to be healthy?

andy lapteff 38:18

I mean, I think it's technical debt, right? Like you mentioned something earlier about no one ever takes something out. Yeah. That that happens in configs, right? Like, I don't know. Does this need to be here? Not sure. How do we test to take it out or not? So that there's just so much, so many mountains of technical debt, I think, in an older environment that it's hard to you know. I think you need standardizations. Here's a perfect example. We had standards that we had to adhere to in places I worked, and we would override them frequently and deviate from the standards because a business unit had two million dollars and had this app we had to deploy yesterday because people are can't wait to use it and give us money, so there'd be pressure from the business. Like, well, I know we're not supposed to do the thing, but you have to do the thing. But we have standards to so we we we would create our own problems, is my point. So there's a lot of cultural stuff happening too.

Bart Smeding 39:12

Standards are good, but not really necessary. If you have a good CMDB where everything is documented, where you can have a intended configuration truth.

andy lapteff 39:22

Yeah. Oh my god.

Bart Smeding 39:24

Then we didn't have one of those either, my friend.

Source of truth, or Excel

andy lapteff 39:26

Yeah, yeah. It was either live or Excel. Excel, yeah. Well, Excel.

Bart Smeding 39:30

If you have it and you have everything in there, then it doesn't matter if the one side differs from the other side because you can always generate intended configuration for that side. So here's a good standard.

andy lapteff 39:40

So now this is a good exercise because now you're triggering the resistance that I would experience in the field. So, hey guys, we have to automate. All right, cool. We'll try. Uh Ansible or Python doesn't matter. Great, I'll do some Ansible. Well, we also need a source of truth. Well, who the hell's gonna do that? Well, you guys are okay. Well, we also need versioning, so we have we have to get like it seems like once you start to automate, it's just this endless like you can't automate without a path of joy, yeah. Right, but again, like limited networking teams typically are understaffed because they don't generate revenue unless you're uh a service provider. Start automate.

Bart Smeding 40:23

Well, they're still not generating revenue more time again.

andy lapteff 40:27

That's right, you're right. But it's trying to get out of like I'm working four maintenance windows a week, I'm drowning, I'm trying to learn this other certification. Now I have to learn Python, fine. Oh, it's not just Python. I have to learn a source of truth too. And I have to doc I have to document the source of truth, figure out which one it is, what are we gonna who's gonna populate all that? I have 15,000 devices. Where do I start? Like it this isn't a script to populate your source of truth first. I don't understand scripts. I have to learn how to write a script first.

Bart Dorlandt 40:56

Maybe we're too Dutch, we wouldn't ever be in the position of working five or six, seven days a week saying six uh or sixteen hours per week. You think it's a cultural thing? Yeah, partially. Okay, yeah, yeah.

andy lapteff 41:07

Well, I have right, I have friends in France who take a month off a year, and that sounds lovely. I would love to do that. France is a whole different level, also. Well, yeah, but right, right. There's so that's a good point. And uh culturally, where I am from, there seems to be a work, work, work.

Bart Dorlandt 41:24

The good perception is you work a lot in the USA, right? And and you get 10 days of of of holidays a year. It's not enough. And and half of those are when your kids are sick. We we look at it and like, yeah, yeah, you get paid a lot, but yeah, we family

Maybe we are too Dutch

Bart Dorlandt 41:37

too.

andy lapteff 41:38

Right, right. Yeah, that's definitely a so that's an interesting point I hadn't ever thought about culturally. Like if I'm working in an environment that is working me possibly to death, depending on how long I do that to myself, it's going to seem more impossible to learn Python or to populate the source of truth, as opposed to, I don't know, my French colleagues who have a month off, and this isn't a swing at them, but we're exhausted, we're burnt out, we're grumpy. Like you know what I mean?

Bart Smeding 42:10

All the programmers will come from Europe where there's more time.

andy lapteff 42:15

Yeah, I don't know.

Bart Smeding 42:16

It's it's an interesting Python is invented from the Netherlands.

andy lapteff 42:20

No, yeah, really?

Bart Smeding 42:21

Who invited Python? Tell me. Guido van Rosem.

andy lapteff 42:25

He lived in Harlem, but also my Guido.

Bart Dorlandt 42:28

Guido van Rosem.

andy lapteff 42:30

I can't pronounce that. Gina von Rosen? Yeah. Yeah, something like that. Yeah. Accepted. Approved. Nice try, American. Well, no. Well, this is the cyclical conversation that I think I love NAF. I love Autocon. I love that you guys are here. I love we're having these conversations. I think that this isn't a simple fix. And it seems like the ways we can fix. So I am playing the part of that traditional network operator who doesn't have time, can't do it, can't understand it, feels it like whatever, right? I didn't go to school. You know, I I'm not a coder. And you guys have very good uh retorts, very like, well, learn Python, you'll get more time back, or you know, get a source of truth. You'll be able to. I don't know how we get out of that. Like that seems to be the key. How can we move our industry toward more automation? We have to get out of the conversation we're having now, which is like, well, learn Python. Well, I'm already drowning in work. Like maybe it's moved to Europe, like right. I I I I don't know, but it it's this. I don't know how we get out of it. And and I'm glad the conversations are happening, but I have yet to see this.

Bart Dorlandt 43:40

Like it's it's never one thing, right? It's it never is either either your company doesn't allow you to innovate towards that, right? Because as you pointed out, you don't have time, right? There's always a fire to be uh to be dealt with. Culturally, they'll tell you it's a priority, right? But but they're not gonna give you from a Dutch perspective. We will we will tell you, like, uh we we have far uh fewer layers potentially than than you have in the USA, right? And therefore, we would say, okay, well, you want me to achieve that. I need time to achieve that, and you

Two priorities, pick one

Bart Dorlandt 44:12

and you got two priorities, pick one. Now, this is a very and this might be a very cultural thing, and I think I guess part has to say something about it.

Bart Smeding 44:20

Yeah, maybe it's it's also or maybe we need some uh global certification for it, but because people do run uh big company certification and they spend time for it. And why doesn't they spend the same time for a certification or learning a network automation? Because there's no certification behind it, uh job offering, don't ask for it in kind of certification.

andy lapteff 44:45

That's the whole DevNet thing, right? Like, did that like Cisco DevNet is an example. That was a big very comparison, very new, right?

Bart Dorlandt 44:53

Yeah, yeah. Relatively speaking.

andy lapteff 44:55

But I wonder if that moved the needle at all. I I don't know. I don't have the data. Right. Like, oh good, now there's a certification. Are people getting up skilled?

Bart Smeding 45:02

Certification needed to get people started and uh get time to study for it because uh company will ask for it now, right then.

andy lapteff 45:10

And I do remember the packet pushers did a salary survey that they shared at this USNUAPA Nog uh event I was at, and there was a stark delta change in salary between folks with network automation skills and folks with hell. I forget what the percentage was, but yeah, their salary data showed that if you have network automation skills, you make so that's a good motivator to niche, right? Still, right? It's it's growing, but it's still a niche. But that's another motivator, right? Like maybe it'll make your network more reliable, right? You can definitely save time once you get good with it.

Bart Smeding 45:43

Save time and make it more reliable. You can and it's fun test all your changes.

andy lapteff 45:47

And if companies are paying more. For the skill set, right? Like there seems to be a lot of compelling. Yeah, there's there's a lot there. I don't know how we get there, but we have to keep having these conversations. Um, and and you reminded me something I just wanted to add. I think that boundaries are important. And what I

The Tuesday confession

andy lapteff 46:00

mean is you had said, you know, if the company uh prioritizes it and says this is what they're going to do and give you the time. So the company that I worked for, I was referring to earlier. I had this conversation with my boss. I'm like, dude, I don't know when I'm supposed to do this. I'm already studying for my NP at nights and weekends. You're telling me I have to learn this, and I'm drowning at work. So to his credit, he said, okay, Tuesdays, that's your automation day. I don't want you to do any work for this company. I just want you to learn. But I here's what happened. And this is my fault. There was so much work that had to be done by four of us. Like we were understaffed, overworked, that when Tuesday would come, like, oh, let me just let me write tomorrow's script. Like I would I was trying to catch up in an impossible game, like one of those hamsters in the wheel. Like eventually I'll run fast enough and get out of that wheel. I screwed up, I think, because Tuesdays I would spend all day working on scripts for like later in the week, not learn Python. And that's on me because he gave me permission, but I didn't, I guess, enforce that boundary of my actual work I do and change windows. Like my whole career, it's almost like an identity thing too. It's not just cultural, like my whole career is a CLI. That was the whole certificate. Like, and then my whole value is change windows, new services, fixing stuff, whatever, right? And and and then I have to, okay, Tuesdays, I'm not gonna do the stuff that's made me valuable for 15 years. I'm gonna do something else, which will eventually make me more valuable. It's a it's a longer time horizon. I have to leave this stuff alone. And for me at the time, I can't leave this alone. There's so much of it. I have to catch up. And that was my bad. Like, I should have listened to my boss and been like, no, no was a full sentence, right? Like, congratulations, you're still human. Right. Yeah, but I when P oh, we have this thing, we got to do it. Like, no, this is my Python day.

Bart Dorlandt 47:59

Go off Teams, go off Slack and do your thing, right?

andy lapteff 48:02

That that was that was your golden moment. But maybe it's a can maybe it uh you got me thinking of the cultural thing, like go off Teams. That's crazy. I mean, Teams is on my phone. I'm never a we've but you're you're there.

Bart Dorlandt 48:13

You were working seven days a week already, so you were the other six you were present, right?

andy lapteff 48:16

But um I but but I accept I'm gonna think about this after the episode, but for some reason Is it like unlimited time off, unlimited PTO, right?

Bart Dorlandt 48:23

Is that the same thing? Well, right, like it's a weird we can, but we can't, and it's kind of that's what it feels like, kind of that's a really good analogy.

andy lapteff 48:30

Like, oh, we give you unlimited. Well, is it

Unlimited PTO: we can, but we can't

andy lapteff 48:32

really like unlimited? And like we don't have that. I mean, I'm looking at you, right? It's a it's a it's an American thing, right? But just because they give you unlimited, it's not really unlimited, and you don't really know how much you have, and then you kind of feel funny to take it because you're not right, and then so it's almost better to have a fixed, right? So yeah, it's it's this weird. I think what I'm learning in this particular conversation, which I hadn't thought about before, is the difference in culture. Now, here would be a good data point, and then we can wrap. Are European are countries that aren't you know working as ridiculous, let's say as the states are are their rates of network automation higher because they aren't operating under the I don't want to call it a toxic culture, but this impossible work harder rate. Oh, he's giving me the stinky eyes again. If it's cultural, which is what I keep circling back to, then I would expect folks in Europe that are in networking automate at a higher rate than folks in the states that say they don't have the time or feel safe to take the time to learn it because they're too busy keeping the lights on. I I'd be interested. And if you guys don't know, I'd have to ask Scott and then, but I'd love to see the data between different areas of the world and who's automating more. It's a very difficult thing to answer, right?

Bart Dorlandt 49:51

On on well, if you look at the at the states, right, there's obviously uh a different amount of people, right? And different areas, right? A lot of a lot of parts of the states

Does Europe automate more?

Bart Dorlandt 50:03

are not known for internet technology, right? Right? So they're rural areas, they're like you you wouldn't you would people people might still do something with it, but it's on a very different thing. At the at the other at the other side of the coin, right? There's San Francisco and doing uh all kinds of huge companies that we don't have in Europe, right? Which are very much oriented on the automation thing of Meta, AWS, Microsoft. They do a lot of things with automation. They they those are huge companies. We don't have such huge companies in Europe. So is it is a fair comparison in the end? It will be very difficult to answer.

andy lapteff 50:48

Because if we can't find the root cause, I mean, you know, like networking, when something breaks, like, well, what caused this thing? Usually a software bug. But you know, what caused it? If we don't find the root cause, I don't know how we ever get out of this, you know. Things are hard. Is it well you got to do a Python, but I don't know. Well, you got to do source of truth, but like we we gotta get out of this loop. And and I think we will with you know, through these type of conversations. Is there anything that you guys wanted to talk about that I didn't bring up?

Bart Dorlandt 51:13

I guess um, hey, you've pointed out on on your angle of the view of yeah, of I don't want to do this, or this is what I do, and this is what I do best. But there are plenty of people I would say, or I would guess that there are that that see a challenge or that see an ambition in like, hey, I want to go there, but are limited by possibilities, or um either from from time, right, as you pointed out, um, or or other factors of money on training, or don't uh don't have the sport at home or the time. Time is usually a big thing in it all, right? And that's um I would say right try to change, chase that that dream, right? The ambition, because it's it can be fun, right? As you pointed out, once you have got that first script, and you got the first NetMico, and you can do one device, then you can do 10 devices, right? Write a four-loop around and you're good.

andy lapteff 52:08

I was so happy when it worked.

Bart Dorlandt 52:10

And that's that's what you're gonna get.

andy lapteff 52:12

I'm like, oh my god, I did something. Now it's laughable to you guys, like, all right, buddy, like good for you. But still it was a big win for me.

Bart Dorlandt 52:18

It all started so in in the

The first script that worked

Bart Dorlandt 52:21

end, it's not that different, right? You are an engineer and you love solving the puzzle of getting and achieving a certain goal, and in the end, the software or writing a piece of code is doing the same thing differently. It's still a puzzle, you still have something that you want to achieve, it needs to be complete, and there's different things. I mean, in your in your Cisco code, you won't see a try-accept, and now you have that different again, different piece of puzzle to to achieve.

andy lapteff 52:52

And I think if anyone values their time and you can save time in your job, why wouldn't you?

Bart Dorlandt 53:00

You might just automate your podcast.

andy lapteff 53:03

I'm working on it, I'm trying. You should see me the night before each one of these. I'm doing so many things up way too late. Where can folks find you guys?

Bart Dorlandt 53:10

I'm quite, or we are quite active actually on LinkedIn, right? Um, Bardorland is my handle. Bart's meeting. We're both on the uh on the nav Slack as well. Um, I might be a little bit more active there than uh than Bart, but um have to work on it or automate it. Either way, we're we're trying to uh we're thinking about uh about joining our blogs into into one um netdevs.it. So that's uh that's where people can find us. Um I guess that's the one. And I got uh dreamnetworking.nl.

andy lapteff 53:41

Check them out. These guys are awesome, they have a lot of good insight and things to we'll send all the links to each yeah. The links will be the links will be in the show notes for all things art of net engine. You can check out our Linktree at Linktree forward slash Art of NetEng. Be sure to check out our Discord server called It's

Where to find the Barts

andy lapteff 53:54

All About the Journey. There is a new website that just launched for the Art of Network Engineering. It's at the same uh domain. There is a really cool resources section with a ton of mostly free training resources in basics of networking, automation, RFCs, different communities you can hang out and check it out. There's also a newsletter coming out very soon. You can sign up there. As always, thank you so much for watching and listening, and we'll catch you next time on the Art of Network Engineering podcast. Hey folks, if you like what you heard today, please subscribe to our podcast and your favorite podcatcher. You can find us on socials at Art of NetEng, and you can visit Linktree forward slash Art of NetEng for links to all of our content, including the A1 merch store and our virtual community on Discord called It's All About the Journey. You can see our pretty faces on our YouTube channel named the Art of Network Engineering. That's youtube.com forward slash Art of NetEnge. Thanks for listening.