this is the art of network engineering podcast in this podcast we'll explore tools technologies and talented people we aim to bring new information that will expand your skill sets and toolbox and share the stories of fellow network engineers thank you for calling the art of network engineering if you know your party's extension you may dial it at any time followed by that pound sign john please listen carefully as our menu options have changed for advice on achieving a ridiculous amount of certifications in a year and maple syrup press one for photography beer and beards press two to get regular updates on the brisket pork or turkey of the month press three for all other inquiries please leave a brief message after the tone and thank you for calling the art of network engineering all right when we have a corporate phone system someday like that's that's okay like that's exactly what you're gonna get when you call our number it's like who do you wanna talk to that's fantastic thank you tim he is at tim burtino tim how you doing doing well aj good to see you guys what's new it's sunday we don't normally record on sunday so this is new and different and sunday you know it's usually the relaxed day and but now we're working working on a sunday yeah that's uh that's my fault i'm sorry i no it's not your fault i blame certain hosting services that we take advantage of for the mistakes yeah but we'll learn we'll get past it well that's right it's monday for a deal a deal in the chat yeah he's ahead of us he's in the future i i do want to say that again it is it is completely incredible that he does this 1 54 in the morning for a deal i want to say he's been doing this since the first one we did with the patreons right yeah yeah yeah so that's that's crazy this is for losers i do not agree with that sentiment but it made me laugh that's great uh dan howdy dan howdy how's it going oh not too bad how about you ah i'm doing good doing good i uh i appreciate you repping the switchback hat this evening that's a yeah burlington beer company so it is and just so you know just want to put that this out there that i am not an alcoholic i do have two breweries that i'm wearing but i'm not i promise shirt and hat for the listeners combo yeah so i'm not an alcoholic but i drink often i do i've been uh savoring the uh vermont beer so i just i finished off the regular cider the citizen cider oh yeah okay the other night but uh i've got two of the ginger ones still left you got any that maple style left yep i drank one of those last night too oh wow look at you yeah well when you need more let me know i'll ship some what's up with you today aj oh you know i had a fun little weekend got out did some chores i did some painting yesterday we're getting ready to do some some projects up in the murray home so we had this big pass-through window that goes between our our kitchen and our living room and it was like stained this nasty red color and i i finally painted over it white so primed it painted it you can't even tell it used to be shellacked so good stuff good stuff yeah it's a good project yep yep getting ready to do our kitchen here in a few weeks and take some time off from work tear the kitchen out put a new one in yeah that's no big deal or anything you shouldn't you should have had me up tear the kitchen out we're not doing anything crazy like we're not changing the footprint we're like the the craziest thing we're doing is installing a dishwasher but i mean other than that like we're just putting in new cabinets and leaving things the way that they are oh no yeah and then there's like a peninsula where we're ripping that out we're putting in cabinets and uh butcher block top so what else would be left aj you're like nothing major but the cabinets well i i mean like the flooring literally everything but the kitchen sink yeah we're doing that too yeah dang the one time i get to say that and it's wrong uh no um you know if we had unlimited budget right like i think we'd like to like actually move stuff around like put the sink up against the back wall like cut out a window so you can see in the backyard or whatever but yeah yeah just uh just a quick refresh this is not our forever home that is for sure it's good enough for right now but i digress let's uh let's get on to some winning we've got lots of winning to celebrate this week um you still got that goat handy there tim i do here we go yeah uh so this is fun i love announcing the the personal wins the you know certs are great i love to announce those two but this one uh network geek girl her oldest daughter passed her driver's exam and is now a licensed driver so congratulations congrats network that's gonna open up time for everybody right right yeah uh this is very exciting a young man from the discord accepted a knock roll and this is their first network focused position so congratulations a young man nice yeah it was good to have conversations with him over the last couple weeks because he was going on multiple interviews and asking multiple people questions getting advice so it's awesome to have him pick one and sounded like he had his pick of of a few different so that's good yep yep that's awesome uh this is pretty cool kaiser clark received a meritorious service medal from the u.s air force so congratulations yeah that was awesome senior airman clark thank you for your service yes thank you frankenstein frankenstein and the discord completed the fortinet nfc one and two those are like the hot ones lately i don't know i see a lot of those popping up so that's that's pretty cool a lot of people getting there yeah well fortinet's starting to kind of take off in it i mean it's yeah their sd-wan solution is kind of compelling it's it's very interesting gotcha now did they have like a full stack though like like ap switches oh yeah api switches firewall yeah yep they sure do interesting yep gotcha all right here we go a fun one to try to say what are you looking at me for tiggers it's no tigger bitch okay uh but i i'm gonna go with bt rose uh very sorry in advance uh that i butchered your name but congratulations you passed the ccna so that's hey that's a big one that's a good one yes uh n packet got a job as a network security engineer with an msp and and they say that it effectively doubled their pay like that whoa that's huge i mean wow yeah how often do people get to say that i mean i've gotten races but i've never doubled my pay yeah that's kidding that's awesome congratulations not saying i wouldn't like to but i'm just saying i haven't right uh heath accepted a 12-month contract as a noctek and special thanks to brittany our resident recruiter network engineering recruiter she she hooked him up oh bringing it full circle yeah yeah so congratulations power the community both of those congratulations to heath and brittany uh and andrew roderos received a second set of sans coins so uh they've got some really cool pictures in the winning channel you should check it out but he received a set of uh sands for the sans institute uh and the challenge coins were from uh like a capture the flag competition so he did well and a ctf uh they sent him a set of coins it's pretty pretty cool stuff nice all right so that's the end of the wins for this week we got some new patreons joining us uh we got eric james and moments before signing on tonight gerard came in and uh signed up for for patreon and uh and now is is joining us on the live stream so thank you everyone for your support of our work we truly appreciate it if you're interested in being a patreon you can go to patreon.com forward slash art of netenge and sign up there thank you so much tim can i get another good screen yep i wasn't ready sorry it's music just beautiful music i am no andy actually andy's never ready for that so so you're kind of like right on par with andy and ready just i don't know if they go together nice so uh here we are um we had such a great conversation last week on change management we thought that we would follow that up with documentation uh and i i know again like change management it's not one of those like excited to talk about or do conversations but uh i think it'd be pretty helpful so um i have got a ton of experience around documentation from my own networks that i've managed when i was on the o m side and then of course today in the var world i provide a boatload of documentation to my customers after doing a project so i'm happy to kick the conversation off but i'm certainly interested in hearing what you guys do in in your environments uh so so dan can you uh can you start us off here yeah so uh my experience with documentation if we want to go back to the beginning was uh when i got to where i'm working at currently we didn't have any documentation so i basically had to start from scratch on that stuff and the way i did it was i started by um like logging into the core and then we're at a cisco shop so using uh cdp neighbor show cdp neighbor and just looked at all the neighborships or the neighborships i don't know if that's a word but all the neighbors and and uh basically drew out the the the diagram of the topology of the network basically just jumping from box to box and seeing what was connected to it writing down the interfaces that these are connected to all that good stuff ip addresses for you know management um all that good stuff nice so when you when you documented all that and you wrote it down did you transport sorry transfer it to some sort of a master document or did you throw something in like a a visual thing like vizio what did it look like after you took all the notes so i started out with vizio and i had this massive vizio diagram of all of the you know switches routers firewalls everything we had and then i broke it down into like um basically like the you know what's the the three tiers of the network you know i had access layers so i broke it down into basically a visio of the access layer going back to the core and then distribution going back to the core and then like when going to the core and edge going to the core so i broke it down into you know different uh visio diagrams to do that but um and then also on top of that i took an inventory right we had an inventory system but it was it was rough i like it and people who managed it didn't understand anything about the devices so you know they they had certain things that was important to them that wasn't necessarily important to us on the network team right and so i started an excel spreadsheet and pretty much logged like the serial numbers ip addresses the host names on the boxes uh whatever else any anything i could find on it i would put it in that spreadsheet um sometimes it would be like what software it was running that kind of thing uh also if it was something i put in myself you know like replacing a switch i'd put the date that i put it in basically um that kind of stuff so let's let's take a step back here because you i mean you didn't join that company on like day one of the company like they've been around for a while so yeah you you got there and they just straight up didn't have any like documentation at all yeah and well it it was the wild west it was very much so the wild west um yeah they only needed one core switch yeah yeah they they had a second one but it was sitting in the corner of the room with nothing plugged into it so now uh yeah it was definitely the wild west the people who were there were the ones who basically built the network from scratch essentially right and so um they kind of knew where everything was and like the ip addresses of everything but it was also one of those kind of environments where they kind of held on to what they knew and didn't really tell anyone else that kind of stuff so it's that kind of mentality um so yeah i just kind of i just got in there and started doing my own digging to figure out where everything was and how to connect to it and then trying to figure out passwords because everything was local we didn't have radius at the time okay um so it was all like local admins and all that stuff and password yeah admin and password and not all of them were the uh for the same password or you know the same user and password you know security obscurity yeah yeah exactly so yeah it's it's admin and password and then the last six digits of the serial number on the switch yeah man that'd be a nightmare but there's no inventory so we don't know what switches are installed where so yeah uh the other thing that you touched on was cdp neighbor and i i just want to highlight that for anybody in our audience that is you know studying for the ccna or new to networking so cdp is cisco discovery protocol it's a layer two protocol where devices will share information uh when when dan said neighborships yes that's a word they took neighbor and relationship and jammed it together and it's commonly used in uh routing relationships adjacencies so when you have two osp routers ospf routers talking to each other they've formed a neighborship same with the igrp bgp whatever the case may be and so with cdp you can get information like you know the model its capabilities the software that it's running an ip address it's not always the management ip address i think you're going to see the highest ip or the lowest ip on the box i forget which one they display but if if you're like looking at a core switch which is going to have you know dozens of ip addresses if not hundreds associated to it you're going to see you know either the highest or the lowest ip that's on the box and the same with with any other device too so if you you're you know doing show to c cp neighbor from a switch and you're connected to a router and you see a router that's got a number of interfaces loopbox whatever you're gonna see you know probably not the management ip address of the box hopefully an ip address you can get to but might not be the management ip address yeah um if if you're not working in cisco land you can use lldp which is link layer discovery protocol cisco devices can also talk lldp it's not a service that's running by default you have to enable it but you'll be able to discover non-cisco devices on the network so if you're peered with juniper hp some pcs will speak lldp you'll see phones do lodp information and in fact that's how switches can negotiate power and voice vlans for phones when they see a phone device connected they offer it that information so the phone can function appropriately so it's definitely a good way to map a network too doesn't vmware also use a little dp uh vmware will actually use cdp oh really okay yeah it's is it cdp was a cisco proprietary protocol it is no longer cisco proprietary but it is still called cisco discovery protocol but you are we we are starting to see the proliferation of cdp in other vendors it's up to the other vendors to implement it right absolutely yeah i got you but you know talking about cdp like i i would highly recommend anybody who is starting in their career or you know starting in a help desk or if they have access to network devices or if you're starting on a network team as a junior you know definitely i think that's a good way to start right just to learn how the network's connected to each other because you know what if they do have documentation and they it's probably outdated exactly i'm just about to say that because uh it could be it could be a diagram from like four years ago now if you're in a network that doesn't change very often that it might still be somewhat accurate but most places are constantly changing and so you know i i maybe take it as like a reference but i'd probably still do my own exploration yeah you know through the network just trying to find where everything's hooked up and and it it just helps with you know show commands and stuff like that sure just to reinforce that so i definitely would recommend starting there but what are some other uh commands that you might use for info gathering like show inventory comes to mind is there anything else that you would do when you're on your little discovery uh show ip interface brief maybe yeah see what you know what's ip addresses on different interfaces what they're connected to that kind of thing yep yep so i want to dig into that a second dan you talked about you gathered notes from your different show commands and you were starting to build this vizio together how in depth was that vizio was it physical connections only were you putting uh vlans on it as well and in the layer three ip addresses how in depth was it yeah so when i first started and i struggled with this today too right uh it i've struggled with it for nine years um but uh i started out putting everything i could think of on this diagram right and it just got massive in busy lines going this way in that way and like i have a love hate relationship with vizio but i don't understand how microsoft has not fixed their tool to where connecting a line from one object to another object is just it blows my mind to do that and uh so yeah but i i i definitely put way too much information on there so now what i tried to do is it's more about like what's the process what's the high level right like so if i'm if my intention of creating a diagram is for like our systems team to understand how their you know how users access their servers and you know whatever application might be running on that um i will basically be like okay here's a cloud of users right here's a machine user machine and then they hit a dns entry that is hosted on our f5 right and then the f5 is pointing it back on the back side of it to the to the server over like port 80 if it's like some kind of web server right and they don't see all of like okay it goes through this switch that switch this switch over here to the core and then down to the f5 you know it doesn't it doesn't show all that it just shows user f5 their server basically and have you know ports that they're using because usually that's important right because like if they're if the user has to access you know the web server by https but then on the back end it's 80. it's useful information to put on the diagram so so if either continuing the vizio conversation have either view messed with layers in vizio so let's say you wanted kind of a master document or a single document from a topology perspective of infrastructure gear and you wanted to have physical layer connectivity uh mac addresses vlans ip addresses but that's entirely too much information for a single layer have you ever messed with you know a physical an actual physical layer a layer two layer uh an ip layer within vizio itself to kinda be able to pick what information you wanna see i i've never messed with it really so i just kind of wanna see what you guys have done so when you say layer are you talking about like the tabs not a tab you can actually define um layers and assign objects to those layers okay so you could put like that so you could put for example uh text on the text boxes on the document that had ip addresses and you could assign those each of those text boxes a layer that says this is layer 3 or the ip layer or whatever and if all you wanted to see was physical connectivity and didn't want to see any of the other documentation on there you could just select physical layer only and it would only show what you have deemed as that physical layer so i could see that as a as a benefit potentially of having one single document but being able to abstract things away from it that way you didn't have to have a bunch of different documents i guess you could go the tab route too that does make sense but because sometimes you want to you just want to convey what the layer 2 topology is without looking at at all the other information right that's what i've done i'll use the tabs across the bottom and i'll do like one for physical connections where everything's actually going and then i'll do like a layer two all the vlans and whatnot and layer three all the routing and so forth so yeah but i'm gonna have to look into that because that sounds like it might make things a little bit easier i don't think i'm making that up i think it's real you sound pretty yeah say it with authority yeah now when you guys do your physio documents do you use like the device icons or you do use like generic network oh dude i love stencils i love like the cisco stencils man especially the ones where where it's like chassis and you can go get the individual line cards and fill them in that's fun yeah so so that's that's a whole nother subject right there right because if you do like the physical chassis at that point are you drawing a logical diagram or are you actually drawing your physical diagram right like uh i've done that when i've done um like rack elevations you know like if i have a picture or a vizio diagram of what my racks look like you know i'll put the actual chassis in there yeah some people use like i've seen some people use um excel for for oh yeah rack elevation diagram or i don't know if you want to call it a diagram it's just squares but i guess it could technically be a diagram but but i like to use envisio with the actual model and sometimes it's frustrating though because they don't have all the models you know they don't have a they don't have an icon of all the models so you have to like get creative sometimes but yeah well so at a company i used to work at we had a plotter like a large printer yeah so that they would print off engineering diagrams on like blueprints and stuff like that so we actually made a vizio that was the rack elevation and in the rack elevation next to each device we had the host name which was on a label on the front of the device we had the management ip and then we also put what port on the kvm it was so that way like you know you're you're facing the server rack and behind you on the wall is this massive drawing that has all that information so you come into the server and be like oh i want to get into you know one of the vm servers and it'll tell you which which kvm well that's that's helpful yeah yep it's good stuff all right tim what uh what do you do so i want to step back and talk theory for a minute sure sure sure so let's kind of frame documentation from a standpoint of why do we need it um yeah yeah so what what documentation can provide is information about your network about your infrastructure either in visual form or you know just written form um about how things are set up and how you're doing things now why is that important i think one thing that um i'm definitely guilty of and maybe some other network engineers are the same way is that you get in your mindset of this is my network i know what i'm doing i know why i'm doing everything why do i need to maintain all this documentation and then you just need to step back and say well well hang on hopefully i'm not the only one that that ever has to uh get pulled in if there are issues or need to build upon the network so having uh strong and concise documentation whether it be the written word or whether it be uh the diagrams that we've been talking about so far becomes really important and while the diagrams are really good and having the the visual part because that's so much easier to look at and be able to try to tie things in your head if you have that map while you're trying to troubleshoot an issue or while you're just coming into something blind and you don't know what the network looks like that's huge but i think another thing that would be good too especially if you're in the uh kind of design part of the network infrastructure where you're choosing what things are going in and how they're getting configured i think it's good to have kind of that overview documentation too that just says these are the things we picked these are the standards these are why we implemented the things the way we did because somebody else could come in and say well you put in this routing protocol why didn't you just do something else and then to be able to go and reference something and see why uh can really unveil um big details of the network as well so i just wanted to touch on that i kind of want to hit on that a little bit um so this we you know we did a big uh meraki project or you know we upgraded our entire win and uh so i tried something new this time i i created a pretty long word document and i put in every single configuration i did in the meraki dashboard and i broke down into why i did that like why did i make the decision to check this box or push this route or you know whatever the case may be and you know i just want to say you're an incredible person for doing that it took me it took me a long time uh to do that um and it was a it wasn't a document i just wrote like in a week right it took several months to write that because it took me several months to figure out how i wanted this to be done right and uh and so um i i really liked that process though because it was it was one way to like check myself you know like i didn't just do something then i moved on to it i had to read the document before i pushed it into the dashboard right and and uh it was it was it was nice because and then obviously i you know had one of my peers review it and make sure like i was thinking right i also reached out to um our cisco we had a tsa that was assigned to us and i i sent to him and was like hey you know does all this make sense and he was he would give me like hey these are you know like best practices and stuff like that and reading through meraki's documentation about what they recommend as a best practice and uh so i i made sure that every every setting i did in there there was a reason behind that right and i wrote that down in this document and i can't remember how many pages it was it was probably like i want to say 47 pages but wow to write that and um but but i i like that and the next time i do another design of some some uh something like that i think i'm gonna do the same thing to come at it with the same approach it takes a little bit longer to do it but i felt good about it afterwards so well i think that's great from a project standpoint so you have that documentation that's part of your project that you need to get through that implementation and design or design and implementation and then that can transition to become permanent documentation when the when the project's done now i was saying a minute ago that we document for others right so somebody else can come in and and figure things out and be able to operate and troubleshoot but dan what you were just saying is how you built that long document for implementing meraki and changes and all that good stuff it's also good for yourself uh because i don't know about you guys but there's been multiple times where i've implemented designed and implemented something a year ago and you don't have to touch it often and i'll look at it again and go why did i do that there's got to be a reason why i did this and then i can find out oh okay yeah i implemented it this way because of reasons and now that i know and being able to kind of have that standardized documentation and where you keep it so you know where to to get at it uh is is huge right right yeah and another thing too is help yourself with breadcrumbs right um i i created a table of contents basically on that document and hyperlinked it to uh or whatever it is uh to the pages within the document itself so like if if lis i came across a certain setting in the dashboard all i had to do was go to that section in the document and then i could see why i made that decision now some of some stuff is like self-explanatory like you don't really have to put a whole paragraph of why you made the decision to check this box uh but some stuff it it's like why did i do that i'm trying to you know and so i go back to the document and i wrote a little paragraph about hey i did that because you know this person recommended this is a best practice for rocky you know all like yadda yadda i'm going to take a phrase from you dan because you just you just said something that triggered me um and it wasn't uh it wasn't in the context that you meant it but you said a word that made me think of something you said something around self-documentation so we're talking about so far we've been talking about ways to document our infrastructures that are um that are outside right they're they're in document form they're in visios they're in drawings what have you how do you guys feel about what i'll call self-documentation within the gear itself and that's make having proper descriptions banners that kind of thing because for me it can be you know having the written documentation outside of your gear is important but if you're in just trying to find a quick answer you don't want to open up multiple documents and try to find something if you have you know that standardized way of on all of our core routers we make sure we have uh descriptive uh interfaces and if we have connections to carriers maybe those port descriptions have circuit ids or whatever on there i think self-documentation in the gear itself is is really beneficial as well absolutely yeah i i have a standard of how i write uh interface well usually it's like okay so what what is this connected to that's like the first part of it and then i'll do like i'm weird i do like the colons right and then i'll do like all right is this a point to point is this like a um like hsrp or something if i was doing that uh what what how what's the connection type oh that's cool is this going over metro e you know that kind of thing and then at the end of it i'll either put you know what's the ip address that it's connected to or anything like that um it it kind of varies i'm not super strict on that uh because sometimes it's like hey this connects to a server so i might put like hey it's just a server connection right yeah and i'll put the name of the server in there and and a lot of times i'll put the ip address of the server in there as well so if i'm in like if a if a system engineer is is like hey something's wrong with this server you know can you please check the interface on it but they don't tell me like ip address or anything like that you know i can go through and do like a show a show command on there and just kind of look through my interfaces until i find what they're looking for um but yeah so yeah show interface status that's one of my favorites yeah and that's one of those things too with it with interface documentation is i want to be descriptive which you can i don't know how many characters you you have to deal with um in the cisco world for for descriptions but one of the things oh is it okay which you can see from a from a show run but what i i like to be able to you know get to the point because i do like that show interface status command and if you put too much in there it's just going to truncate it you got to you got to do a show run if you really want to see what the full thing is so yeah you just got to find that happy medium i try to keep it brief and that's kind of why i was like hey what's it connected to what kind of connection it is and like gives it an ip address you know what whatever and it usually keeps that uh show ip stat or uh interface status or whatever it is um it usually keeps that pretty brief right there yeah tim so you kind of mentioned it before um and i want to go back to that point where we create documentation for others and so um if you recall our episode with monsoor uh andy's buddy who works for cisco tack you know that's that's one of the first things they like to get like when you're working with tac if you're starting a new case and you have a troubleshooting issue or an issue you're trying to troubleshoot the the best thing that you can do is provide your attack agent with network documentation yeah not necessarily as deep as the y because the y doesn't matter at that moment it's it's how is it connected uh and so if you you have even a high level diagram just showing how stuff's connected that's going to go a long way for your attack keys and i'll say that too is working for our partner i mean sometimes i have customers where they buy professional services hours to have an engineer come in and help troubleshoot an issue and it's it's a russian roulette as to what kind of documentation i get right like sometimes i get beautiful documentation sometimes i get i get documentation but it's really old and then sometimes i don't get anything you know like maybe a cocktail napkin we drew it out at the bar before i started you know troubleshooting but hey i love whiteboard sessions yeah oh yeah for sure um i definitely want to touch on tools so you know vizio certainly comes to mind there's there's plenty of other diagram uh applications out there but i think there's lots of other applications too that you can use you know dan talked about putting a bunch of stuff into word and uh and so what do you guys use to actually create your documentation is is everything stuffed into a word document and you got like a library you know network shared folder full of uh docs somewhere do you guys like file it into a document management system that's uh makes it searchable what do you guys do so one thing i love is uh microsoft teams right you can create a team in microsoft teams and so like we have like our network team right and and so we put all of our network documentation in that team uh file system i guess i think it's sharepoint on the background but uh um so that makes it to where if i put if i create a document in there all of them can see it and if they need to add to it like let's say we're we're doing a you know an inventory audit right and we're going through and making sure like if there's a switch that was put out there but like the serial number wasn't documented or the management ip address was undocumented or whatever the case may be right the host name was undocumented um if we're going through our network and doing an inventory audit you know we can add that in there but any of them can add it right because because these documents are they're they're you can be edited by anybody that's within that team and so i think that's awesome like that's a game changer right there no more of the having to create a you know an excel spreadsheet put it on a network share so someone else can put it on their machine edit it and then update it onto the the share it's like no nah stop that this you you just edit it in teams yeah where i work too we use teams um and so because we work with customers we have a team site for every customer and then each channel is the project that we're working on and so it's not always like that i'll caveat that by saying that sometimes if the customer has security requirements then there's a new team for every project for that customer and only the people assigned to that project get access to that so and typically it's hard for us to go back and get access to the old project um for for reference purposes or whatever just because you know there's certain security requirements of the customer that they can't have everything lumped into all one but most of the commercial customers they have their own dedicated team site and they have each channel is is the project so it's really nice to be able to go back and look at stuff like if i'm new to the customer but they've been with the company for a while then we can kind of go back and see like oh you know here's what was done previously by by the company and so um i in it's a shame that not all of the managed services customers documentation is handled that way that's usually just how professional services is dealt with and i i understand the frustration of the customer when when they're like well you guys manage the network why are you asking me you know and it's just like i don't have access to that side of the house unfortunately i have to go back for it please uh please but yeah i know i i love me some teams for sure so you're able to get most or all of your customers uh leveraging teams i mean or is it just kind of a given that they're already running it internally a lot of our customers are do run office 365 because we we also do professional services for office 365. oh okay we we have an entire like microsoft side of the house that does that stuff um but we use it internally at the company that i work for and we can easily create you know from microsoft teams we can create a sharepoint site that the customer can log into and they can upload their documentation and provide that to us or you know config output or whatever the case may be that way we're not sending it back and forth through email even if it's encrypted email we're not doing it that way we're you know because then there's typically a attachment limits and stuff like that so we just give them a sharepoint site that they can upload the documents to and then that's how we pass like the final product that we owe them for for the documentation on the stuff that we did like the as builds bill materials and stuff like that yeah i feel like that's you know a better workflow than sending email and stuff back for sure so i do want to give um i do want to give another example of a kind of a topology documentation tool because obviously vizio's is the big one right but maybe you for whatever reason don't run o365 or or local copies of office something else that i've run into i think it's changed names it used to be called draw.io i think it's uh diagrams.net yeah so i'll get to it through draw.org yeah you can they redirect it um but that's one that i've used uh before i think i put some some diagrams together for some of the the blogs on on the a1 site and it's it's just kind of an open source um thing online and you can do all of your diagrams just within the web browser and you can save them off as different file types either xml or png or or something like that so that that's been really useful is just kind of something you want to throw together quick or if you want to you know build your documentation around it it's just another option yeah yeah and that's pre-loaded with tons of good stencils too like all the cisco stuff is in there yeah i've seen some really good uh azure and aws stencils like pre-loaded okay you don't have to like go download them or anything like that they're there so one that i've been using here lately um chris miles in the discord he recommended the lucid chart and so i tried that and i absolutely love that one i don't have to fight the connectors like i do with physio i've never heard of that it's a like a diagramming software yeah well it well so i think they do have an app i think aj pointed that out to me the other day um but i just use their web whenever i whenever i create stuff i just they do have apps for you know smartphones and tablets so you can actually hear your dynamics exactly you know large screen advice now you do have to pay for that one yeah but i don't know it's it's much i don't know it's much easier but i i don't think it's anywhere near the price of vizio yeah i don't think so yeah i mean vizio's like several hundred dollars if you're going to buy a copy and i think that i think there's like two levels of vizio there's like regular vizio and then visio professional and of course if you're going to do network diagrams you need vizio professional and i think that's upwards of four hundred dollars i can't remember the price on that i know it's a little bit cheaper with the office 365 offering because then you can pay monthly um and it's still a lot less per year but we have that included in our our subs for it so yeah so but anyways i i have experience with it and i love it so i would i would recommend that as well as another option um i have used dr uh draw io in the past uh i i did like it um also with lucidchart the cisco aws i'm pretty sure azure stencils they have all those in there as well kind of like what you were saying with uh draw i o there so you don't have to go download them or anything and then you can also add your custom ones in there as well so sure so one thing i want to touch on is one of the big grievances that that i see and have seen is that documentation is can only be as good as the last time it was updated right so if you guys ever looked into or seen any offer i know there's other there's offerings out there but i don't know if you guys have messed with any of the the ones that are maybe network management systems that offer some of this kind of uh automated up-to-date network mapping diagramming have you guys do you guys have any experience with any of those kind of things so basically taking it away from the manual vizio to try to keep it updated is is that feasible i i've heard of some of those kinds of tools in fact for active directory you can actually have vizio crawl active directory and it will diagram out your ad structure for you wow i've never heard that yeah yeah it's really cool i've actually used that before when i had to do some ad documentation um i know that um i think we were talking about this previously there used to be one called net brain or something like that i've yeah it does sound familiar yeah not one i've actually ever used i know the tools are out there but i've not i've not yeah i've seen a demo of the net brain and i've also seen a demo of um live action live action oh yeah i don't think their claim to fame is a network diagrams per se but it does diagram out how your network's connected to each other so i guess it's just a added feature i don't know because to me that that's just got to be frustrating that you spend a countless amount of time getting especially like what you said dan you started at a company you spent so much time going in and learning it and that was a great way to learn it so it wasn't it wasn't lost or anything but you spend all that time and then you know a year or two later somebody you bring in a consultant you bring in somebody to help troubleshoot something they're like let me see a diagram of your your core distribution access and you're like yes i have that you open it up and you look at it you go wait that's not right anymore that's not right anymore that's old that's that's not even there yeah so yeah i think that's that's something that there's definitely room for improvement so if you can find a tool to help you or or something because it is you you got to keep that updated whether you're doing it or whether uh you have a tool that's doing it for you otherwise what's the point yeah and you know one tool that you get is a is like a junior network engineer that's always a good tool i don't think they like to be called tools dan yeah no i i definitely so you know we've been talking this whole night about how i did this and i did that and i did this right uh the updating it is is something i struggle with right and i need to get better at uh it's it's easy or i say it's easy it's easier to do it once when you put something in or right after you put something in but keeping it updated when you make changes to that that's uh i i struggle with that yeah because it might not always be the first thing or even second thing that you think about is you you get done with the new project and you know you're just happy that it's done you don't think oh crap i gotta go update my all my vizio documents and everything but so i don't know if that is that something you can just throw on your calendar quarterly to say hey go look at this document and and see if anything needs to be changed i mean otherwise i think it just becomes stale that's that's something when when i was an i.t manager i built that into the workflow for every project we we tacked time on at the end of the project to do documentation and you know it's never fun and we've been you know working on a project for so long we're you know we're through the project we've implemented the solution uh now we want to get on to the next thing right it's so tempting to just jump to the next thing but we always made time at the end of the project to do the documentation now for if it's like a big network refresh we're probably working on you know we have diagrams to go off of you know from the design portion right and it's pretty easy to convert those to the final product um but we always tried to build that documentation we tried to like make it part of our culture because it just became so much easier to troubleshoot issues particularly in a system that like you know the application side of the house you know had to document like how stuff was connected because like we set up the server as the infrastructure to support it but after that like you know we had no idea what's going on behind behind the scenes on that side of the house so you know even they documented workflows that way we could help you know troubleshoot issues and stuff like that so it just became such an important part of what we did to help you know maintain what we were doing uh in-house so so i'm gonna say something and i'm probably gonna trigger some people but i'm gonna blame bad documentation on bad management i think i think you're right i agree with that yeah well it's never my fault dan yeah no i mean i'm only you know doing what i'm told so i'm not if i'm never told to document stuff then uh i'm essentially doing what i'm told right so yeah i and i don't mean to like cause any trouble with any i.t managers that might be potentially listening to this but uh i i i kind of i kind of feel like that like what aj just described is perfect or it explains why i said that right because he he was an i.t manager and he he built that into their into their workflow for every project that was that there was a lot of time to that that's vital because because sometimes when i'm at work and i and i go on my little you know i get out my 64 color crayola pack and i start drawing out all my network uh does it have a little sharpener in it built into the back of the box so when i'm out there drawing out all my little network uh icons and all that stuff um you know sometimes i feel like i'm wasting time in the sense that and maybe not wasting time because i know what i'm doing i should be doing that but i feel like there might be other priorities that i need to be working on rather than drawing these little icons um so but if i had a management that said no we want you to take you know three hours you know after that project and just write out everything draw out everything in maybe more than three hours right because sometimes you it takes you an hour just to connect a router to a switch and visio so so you know whatever time but they allot that time to say hey you you have to diagram this or you have to write up a word document about it or a spreadsheet or whatever it means uh so yeah i'm gonna stand by that that uh bad documentation is a is a sign of bad management so dan i don't know if you meant to do this but you were doing one hell of a job at standing in for andy tonight oh thank you i don't want to be this guy and call anybody out yeah oh that's that's incredible bean counters yeah yeah i will echo that because it is so easy um as as a technical person to get to the end of a technical project and just say okay i got this done yeah i really need to get this documentation put together but oh look i've got these other things in my inbox that are maybe technical related that people need to get to i think it's so easy to say well i'll just schedule some time next week to fill out this documentation and i'll get on to my more important stuff and then something else comes up and you push and you push and you push so yeah i i do like that um i think you guys need to start keeping a count of how many times i say the word culture but i think it is uh part of that you need to bake that into there so the fact that you did that as a manager aj that was really cool yeah and i bring that up because i listened to the last episode i'm pretty sure i said culture like 14 times nice well i mean it's true though there's so many things that have to be a part of culture and i know that we've talked about it but the idea of doing something versus the inherent understanding of why everybody has to do this is is very different and yeah i think it's a lot easier to shirk something off when it's optional but when you know that other people are depending on you to do your job then you got to do it right like that's that's why it's just got to be part of the culture well and so to shift gears a little bit i also want to talk about uh not just documenting what we've done but what about like how to do something like in in in your environment so one thing i've done in sharepoint sharepoint i used to hate sharepoint i'll i'll admit this but now i'm starting to come around because maybe it was just the way it was implemented at our place i don't know uh it was probably poor management yeah yeah was it was it poor management yeah uh i'm not calling out your place of work at all i was trying to make a joke it was terrible no it's not good um so uh you can create these like web pages within sharepoint right and uh so what i did there there was one that was kind of like a uh it was kind of like a blog style if you want to call it that and i wrote like how to do ssl offloading on the f5 and used like our because you know sometimes when you're reading uh someone who's done a lab on the internet right they they use very generic wording they use uh like they they call their certs just goofy stuff and and sometimes it's kind of hard to follow along with uh how that would look like on a production system right and so what i did is after doing my research on how to do ssl offloading i then wrote up a how-to guide on with screenshots from our system so that it's more familiar to whoever's reading the the guide basically right um and so i broke that process down with like steps that along the way you know you have to generate a certificate request and then once you get i i where i work we i the network team doesn't actually generate certs they they have to be given to us and so i give the the request they give me the cert i import the cert and then i have to create the ssl profile and then i apply the ssl profile to the virtual server which is f5's way of calling it a vip address basically and uh and and that's how you do the ssl offloading right and so did you teach a class in this or did you just write a blog about it i just wrote a blog about it just just a blog post uh and and so that way if anybody comes behind me or we get you know new people in or some someone on the system team wants to get their hands in that five or something like that um there there's a guide there on how to do like the ssl offloading and and so have you guys done any of that stuff like oh yeah i'm glad you brought it up um because that's definitely the how to's are definitely a form of documentation um yeah kind of like what aj was alluding to earlier is he was talking that the systems and topology documentation is part of the workflow of a project well a work a project workflow something that's a repeatable process is definitely documentation because you want to have those at least high level steps written out in how you do whatever it is so an example would be turning up a new facility turning up a new site you want to go in you want to do your your walkthroughs ask questions a b and c to be able to to uh implement what gear you got to put in and and so on so you want to have that that step by step this is what you do then this than this otherwise things get left out and you look less professional so yeah the how to's and building those steps are definitely a form of documentation i'm really glad you brought that up yeah and so another one too just to give example um i created a pdf on basically how the help desk if they had a user that maybe i i don't know what scenario it would be in but if they if they didn't get an image from us right and they had to install you know their their vpn software their agent um i put a diagram in a um document a pdf document together basically showing how to log into the portal how to download the the file how to install it all that's that so yeah you're getting into knowledge base articles now yeah exactly so that's kind of where i was going to is uh creating that step by step so that you know the other tiers you know a help desk or like a desktop support or anything like that know how to use the systems that we just installed that kind of thing right because it's new to them too uh so any way we can help them get them documentation on how to walk through that process everything like that that that's helpful as well and and i don't think some people think about that as document or um not that they don't think that's documentation but i don't think they think about that part of it you know helping the other teams as well and so one thing that i think aj you've mentioned in the past is you know you know uh wikis right because you can you can do wikis with or a blog or a blog yeah uh so do you have you used wikis like yeah yeah where you work now do you guys use wikis because we don't use wikis but it's something that i would definitely be interested in because you know we were talking about um uh what was it like in our change management we were talking about like if you have tickets that you know something broke and you had to implement some a change right you could have that in a database where you can do some searching and kind of find that and i'm just curious like outside of change management what have you done with kb or wikis before so um at a company i used to work at manufacturing company it had a lot of people that you know had this like really good knowledge about the company but they were starting to retire kind of thing so what we did was is we set up a wiki it was an open source product but had some really neat integrations like we could integrate it to active directory so people could just off into it from their windows machine and um it was a wiki just like wikipedia so anybody could log in we kind of created you know sections for each kind of department and then people could log in and share their knowledge and it just became this this thing that the company kind of relied on it was we didn't really have like an intranet but we had all these different like internal resources that people could go to for information and a lot of stuff was kept just in a file share and it's just like it's time to get stuff out like the wiki was far more searchable and user friendly than just getting stuff into a file share wordpress wordpress is actually an open source uh blogging engine that you could set up internally and just have people log in and create you know all sorts of guides and documentation and then you can tag them it becomes searchable uh that's it's kind of a beautiful thing okay um you know sharepoint has those features too um yeah there's there's lots of good stuff i i do want to shout out one of my favorite products when it comes to doing this kind of stuff uh it's called snagit by techsmith it is a fantastic piece of software and it's very inexpensive so it lets you screen capture and you can do it either through you know a movie you can you can actually record your screen while you're doing stuff and it's really fun because you can tell it like hey i'm going to scroll a web page so it will capture your browser screen and let you scroll the page while it's recording your screen and you doing your thing you can tell it to take a screenshot but it will take an entire screenshot of the webpage so if you have a long web page that you need to scroll it will actually take a picture of the entire web page okay it's really cool stuff uh you can take videos and you can turn them into gifs which is what i've done on my blog on a few times like i've executed some ansible scripts watch all the writers do their thing uh and then turn that into a gif that i then made you know one of the images for the the block article um yeah that would be nice actually what's nice in that tool is um after you get your screenshot you import it into the snagit editor and it will actually let you do things like steps so if you're describing steps in your document like you know click here click here click here click ok you can draw a red box around the ok button and then it has step stamps kind of thing where you can stamp one two three four you know whatever and so that way your instructions can line up with certain parts of the image gotcha if you're trying to create documentation but you don't want certain parts of it shared there's a really nice blur tool that will blur out sections like you can just draw a square around a section of the drawing or whatever that you want blurred and blur that information so that way you're not you know compromising anything so it's a really cool tool it's it's 50 bucks one time fee and then 1250 for annual maintenance and that gets you access to uh all the latest software updates so next year when they drop snagit 2022 you'll be able to get that right away and then um it gets access to support i've never had to use support i've heard it's phenomenal um but just such a great product nice that's awesome dude it's called snagit you said yeah snag it yep snag it okay i might have to look into that myself man yeah no it's really good i i'm happy to show to you too offline if you yeah if you want to check it before you buy it yeah for sure cool well uh any other parting thoughts guys we're we're at about time here i think we've documented the hell out of this one i see what you did there um i got i got nothing else around actually i do have one more thing around documentation and we kind of touched on it is putting together the documentation is one thing uh but it's gotta be easily found people need to be able to navigate it that's why i like what dan was talking about in in his documentation earlier of of creating links like a table of contents make it like a book that you can go back and forth between links so i think that's that's really important because if it's not easy to find people aren't going to go searching for it and then they'll end up just coming back to you asking questions and that takes more time and resources so that's that's my last thing about documentation i do want to shout out to dan uh for bringing up one of my favorite words of all time neighborship that is that is an unsung word another favorite word is pumpernickel but we're talking about uh network engineering so we'll go with neighborship that's my favorite it's still new yet it's got to work its way into the top nice that's great awesome dan any any last-minute thoughts uh yeah um management do better about you know documentation you know a lot a lot your engineers time to you know properly document things yeah uh i think i think that's that's my last words awesome yeah and i would just say you know there's lots of free documentation uh or or free tools out there to to do documentation you know tim brought it up draw dot io um you got you know microsoft word or or any open source equivalent so start documenting stuff today even if it's just lab stuff right like get experience documenting because you know unless you have just horrible management that doesn't expect you to document anything like you will be expected to document something someday so even if it's just part of your learning journey you know spend the time doing the work um and it'll be a good experience for you i i personally love draw to i o if i'm when i'm doing stuff in the lab if i'm you know working with a new customer and i just need to kind of drop stuff out there you know that's the first place i go to so good stuff all right thank you so much guys special thanks to our patreons for hanging with us here on this sunday evening we certainly appreciate you being here uh again if you want to join uh our patreons and hang with us in the live stream while we record this thang you can do that at patreon.com forward slash art of netenge if you're looking for other ways to support us there's plenty of free ways to do that like us follow us go to our youtube channel subscribe watch the videos let them play in the background while you're doing other things that watch time really counts really helps us make sure you like all your favorite videos and drop a comment or two as well because all of that really helps our online presence mojo you know man that goes a long way excellent well uh thank you so much guys appreciate it and we'll catch you back next week for the next episode of the art of network engineering thank you hey everyone this is aj if you like what you heard today then make sure you subscribe to our podcast and your favorite podcatcher smash that bell icon to get notified of all of our future episodes also follow us on twitter and instagram we are at art of net edge that's art of n-e-t-e-n-g you can also find us on the web at art of network engineering dot com where we post all of our show notes you can read blog articles from the co-hosts and guests and also a lot more news and info from the networking world thanks for listening you