this is the art of network engineering podcast in this podcast we'll explore tools technologies and talented people we aim to bring new information that will expand your skill sets and toolbox and share the stories of fellow network engineers hey there folks you want use switches we got you switches come on down to tim's transport town where we've got quality use switches to fit every budget looking for a stylish way to get your packets from subnet a to subnet b well check out this gently used pro curve 2650 features 48 ports of 10 100 make speeds and even supports gig via optic looking for something for the whole data center take a peek at this gorgeous catalyst 6509 supports dual supervisors seven line cards and is capable of switching up to four terabits per second this baby can run cat os and ios maybe something a little more sporty then you'll love this 3com baseline slim compact no management interface you'll be the envy of all the engineers but that ain't all we got routers access points load balancers if it gets your pack across the line odds are we got it down at tim's transport town take that first right after the art of network engineering and welcome to the show thank you jordan for that intro he uh has written the last couple and they i've had a blast so thank you jordan and so is that would that technically be an se i don't know you call them what you want dan yeah he can't he can't talk back to us so you call him what you want and this evening other than myself we've got dan howdy packet richards what's happening dan howdy howdy you doing good i am i am it's been a bit of a bit of a long couple days but we're getting getting down to the end of the week i hear you yeah we're about to do our disaster recovery this weekend oh so we'll see how that goes been doing some uh some uh f5 upgrades and now disaster recovery all over the board man i swear every week is different i was doing some uh palo altos today so jeez man do you enjoy being the guy because that's clearly what you are i don't know it's starting to wear on me i'm not gonna lie and we also have andy permit ip andy andy laptop andy how are you i'm good tim i'm doing good man i was i was supposed to read i was supposed to read a design chapter today from the ocg to prepare and work was too busy i didn't get to it so i'm going to ask a lot of questions i guess well uh aj's not here so we'll see how good we do it's on you bro if we mess this up we'll just record another day i am not looking forward to aj watching this one anyway all right let's get into the winds we have a lot to get through this week let's start it off with jeff kish and his wife celebrated their 16th wedding anniversary this week congrats to them you know how we feel about support systems around here that is excellent running on empty received another six and a half percent salary increase the only downside being that the responsibilities are increasing as well you know it's funny how that seems to work that way devnet dan passed the aws certified cloud practitioner exam nice work dan said that it was a pretty fair exam and a great intro into aws taylor is back at it again by passing the vcap dcv deploy exam which is vmware's data center virtualization deployment certification well done taylor martin nbmbne started a job as a systems engineer for a cisco integrator in denmark this also marks martin's departure from freelance network consultancy good for you martin i'm going to say neos started their professional journey with a first it job as level 1 tech support at a transportation infrastructure company in their noc this is a move away from inside sales and customer service how exciting packet brewer accepted a managed technical specialist role at a major isp to support clients for their smart devices like wi-fi security surveillance and voice this is packet brewer's first step away from the help desk onward and upward i found this one interesting especially the traffic network passed various fema exams this morning which proved knowledge around the basics about the incident command system initial response national incident management system and national response framework nice work flub a dub of phil i love saying that name by the way want to capture the flag contest in an ice training lab that cisco put on very cool adil khan reached 100 twitter followers look for that number to continue to jump chris deadman rowlet got an a for the pre-calculus midterm this is huge because less than a year ago he struggled with algebra and now he's crushing pre-calc natasha got promoted to a level three field service technician nicely done there natasha kaiser clark passed the network plus and won airman of the quarter for his unit in the air force so cool ray 6221 passed the pc nsa and credits keith barker on the cbt nuggets course for being awesome eric cho author of mastering python networking has started up his network automation nerds podcast give that a search and check it out now this has been such a fun journey to watch ipv faux has accepted a position with cdw in their ace program specializing in azure infrastructure chris credits netsequezy for helping facilitate the opportunity and loves this community so exciting and to cap it all off ron n received an offer for a staff network deployment engineer position with a government contractor it's going to involve a lot of travel but should also involve a lot of learning thank you all for providing the wins i counted 17 i love to see that number grow let's take it out in this episode we are going to get into kind of a kind of a deep set of topics and uh you may have some some opinions and if you're not a fan of anything we say or we get anything wrong please feel free to tr to tweet at no blinky blinky uh i'm pretty sure he wouldn't mind any direct messages as well uh he loves that kind of thing but uh anyway we are going to talk this evening about enterprise network design and guys i i kind of want to break it up into high-level topics i want to cover kind of campus design we'll get into just because it's dan's favorite topic we'll get into wireless a little bit don't worry we're going to keep it on the infrastructure side there's there will be no rf talk this evening a little bit of data center we'll talk about and then at the end i kind of want to get into some high level uh design tips that that i've learned along the way so getting into kind of campus design dan what is like the first thing that any cisco press book you've ever opened for ccna anything like that ever talks about when it talks about kind of campus design stp stp dang you stumped me uh i'm gonna kick this one over to andy cause he just gave me a symbol or is that scout's honor i'm sorry that's the hunger games like he's with you i was flashing your gang signs bro no um my answer was going to be the three-layer design the access distribution and core that's where i was three teams i want to kick this off by by starting with the real high level traditional campus three tier design and that's where you have a core layer a distribution layer and an access layer and definitely one size does not fit all for for any uh organization any vertical anything like that it really depends on the the size of the organization and what the organization is kind of willing to to bite off on but i do kind of want to talk about those those three layers so you guys want to start at the top of the bottom and i guess which is for the bottom let's let's start where the user is i like it start at the bottom so we'll start at the uh the access layer or some may call it the edge and that's where where you guys called it where users connect and that's going to be your computers your phones your wireless access points that kind of thing right at the edge of the network and that's where you typically have your uh your high density port counts you can have fixed switches you can have chassis based switches you can have stackable switches that kind of thing and again it's it's where your users will connect to the network to get to the rest of the network and if you're in a large enough facility where you have multiple access layer closets then you'll have what's called the distribution or the aggregation layer and that is if you can afford it at least two switches or would be two switches where all of your access layer switches uh connect dual and then if you have multiple sites and you have a centralized data center that kind of thing that's where you would have that core layer to aggregate those multiple access layers to give you connectivity between all sites so so you said something interesting there so you said from your access switches to your distribution switches you have uh dual basically like a dual home right like yes one goes to one one goes the other what about access switches that are connected to other axis switches known as like a daisy chain there it is is that a good or bad thing tim it it is a design yeah it is something you can do so so what dan's uh talking about at least what i've seen done before is when you may have an access layer switch in a in a data room in a network closet if you will that's connected to a distribution switch and you may have another part of the floor that you need to cover with another switch and for whatever reason you may not be able to get an uplink back to the distribution switch you may not have port count at the distribution switch to support it so to get around that you would essentially daisy chain or hang a another axis layer switch off of that first axis layer switch and and the goal there is really again just kind of coverage if you don't have another link that you can get back to that distribution layer or you just need more port count and it's not a stackable switch that kind of thing but yes i i've seen it done i will uh not admit to whether or not i have done it yeah i've definitely seen it so definitely done it too i'm proud of you i'm still at the uh whatever uh stage of uh that is denial that's that's where i'm at right now we want to talk it like we want to talk about the stuff that happens at the access layer like vlans poe like do we care how deep do we want to get into this no that's happening at the access layer that's good let's um first let's talk about and we kind of touched on that quick is what are the different kinds of switches that are there and we talked about fixed and then stackable switches so a fixed switch is your typical one rack unit uh 24 48 port you may have an uplink module to get higher speeds back to that distribution layer a stackable switch is is typically multiple uh multiple one rack unit switches that are connected together usually via cable in the back uh that connect them and that will logically make them a single switch and then i'll get into the third which is kind of a what's called a chassis switch which is a big back plane and then you have multiple line cards uh that jordan told us about uh there when you wrote that intro yeah so can we go in a little bit on stacking switches like what are some of the benefits of that and what are some of the cons if any definitely so stackable switches what that gets you is that that kind of affords you that benefit that we just talked about a minute ago in that if you needed more port count if you didn't uh use like a stacking technology then you would have to have the scenario that dan mentioned where you had the daisy chain switches off of each other and those are separate physical and logical switches so any any spanning tree uh caveats or faults will apply to that whereas if you stack them together using a stacking cable or a backplane cable uh in this case you you avoid that because they become treated as a single logical switch yeah did you say multi-chassis uh um oh we're gonna get there okay and uh well another thing too is like management right so if you have let's say six switches in a closet and you're they're daisy chained or let's even say they do have a home run back to the distribution you still have to log into each individual one of those switches whereas if you have a stack it's one management uh login versus uh six different logins yeah great point single uh single management plane single control plane as well if you're doing uh layer three switching so you're doing actual routing at that layer it happens all logically as a as a single switch now uh andy i do want to circle back to what you were bringing up what are some of the different things that can happen uh at the access layer and one of them was you mentioned poe why what is poe guys and and why do we need it power over ethernet right you can uh well what is it i mean what see i i'm realizing as we're looking at this like as a wan guy and now data center guy i've never been in this environment which might be why i'm so bad at design but um i guess like what phones right need power um and like security systems like cameras or not like um wireless cameras that's for sure wireless controllers too like are those access points need power too right yep uh power over ethernet will will give you the ability to uh power low power devices uh over pairs of the ethernet cable so you can power things like ip phones uh security cameras wireless access points and not need external power to power those devices now that's one thing from a design standpoint if you are going into a new facility you're building a new building that is definitely something you want to take into account when you're looking at what switches to purchase because especially like these new access points a lot of these access points come with 30 plus watt requirements and uh there's so many different switch models out there you can you can fall in a trap pretty easily and not be able to power some necessary devices yeah because not all switches are made the same right so right they have switches that have certain power limits on them like a certain wattage that that they can push out of the you know across all 48 ports and uh so you do have to take that into consideration because if you're gonna have a phone hanging off of every single port you know and or if you have like a like what tim was saying some ap that's doing 30 plus watts because i think phones are aren't they roughly like 15 watts or something it depends like some of the the real i don't want to call them low end but some of the low feature phones are are between like six and seven but yeah typically you know but that's another thing to watch out for dan if you get into those front desk phones with the side cars and everything that's yeah that's more and you may still need you may need poe for the kind of the main component of the phone and you may need external power for the sidecar it's just all different things you need to take into account yeah so uh one of our patreons was saying port watt and total switch watt that's you gotta consider that yeah and someone said uh i've seen this uh ubiquity they have lights that you can power over poe which i thought was interesting huh never would have thought i would be pulling a cable to a light to my switch connected buildings man skynet's taken over oh uh i'm probably going to butcher this but raino uh then clients that's a good one right there iot devices i would i would love to hear that i would love to hear the story of anybody who's doing i don't know if you guys are i haven't used leveraged thin clients that are that are poe i think that would be pretty cool yeah because you know if you were mad at the pers the user you could just bounce their port it's like what were you working on again well that's that's a good point dan that's another kind of benefit to uh things that are that use poe is that i guess it's a blessing and a curse is you as a as a network admin a network engineer have the ability to reboot those devices if they have issues you got a security camera that's having problems the anybody who can get into the switch can reboot that now so be careful that uh they don't take advantage of you so you're not getting woken up at two o'clock in the morning because somebody needs a camera rebooted and you gotta wake up the network admin to do that but hey uh so we talked about poe let's talk about some components on uh and this will apply to other layers as well but uh redundancy so typically when we talk about the access layer there's from a switch perspective there's not a whole lot of redundancy right because you've got your phone your computer your access point really only has one link to the access layer where you can gain some redundancy is at the power level a lot of switches coming out these days will have modular power supplies that you can run one or both and what's nice about those is in my experience now there are other things that can happen to a switch to cause it to fail but power is is a big part of it power supplies like to pop when they get older and even if you have a power supply pop if you've got a modular setup you can basically hot swap it right so hopefully you've got two and if one dies the switch stays up and you can replace the one at any time so that's that's really beneficial and something to think about yeah well i i love dual power supplies um because when i first started we didn't have any and as we upgraded our network and whatnot um we we started getting switches that had the dual power supplies or at least had the slot to put a second one in there and uh i one thing that is just crazy to me is like if you plug in one side and then you do some cable management on the other one and then you get that one plugged in and then you unplug the original one and do your cable management there and it's like mind-blowing because you could just sit there and switch between i am i am that guy dan i've done that same here and uh i i was just having a brain fart but traffic network helped me out i did want to bring up uh it and to caveat this i'm going to talk a lot about cisco stuff so i know other other vendors are going to have similar technologies but uh power stacking so we were talking about data stacking earlier power stacking is a thing too and that is where you have a a separate cable again it'll be in the back of the switch that can basically connect to multiple switches and they can provide power to each other so let's say you have a stack of three switches and you only have one power supply in each one if you have one power supply let's say in switch 2 that fails if they are power stacked so cabled together with that power stack cable then your switch can stay up now that's i'll caveat that with it depends on uh how much uh how much poe you're using you may have some devices that have to turn off because you've got to now power a switch with one less power supply so there are caveats there but it's definitely uh i think it's kind of a game changer it's helped me out on on multiple occasions when we've had power supplies go so i wanted to circle back to something you said tim earlier about daisy chaining why why exactly i mean i know that it's not good but what does it break like does it break spanning tree is it just that you're putting way too much traffic on those single links that are interconnecting them so i'll say i'll say bandwidth is is one concern another is that uh if you have a power supply failure in the upstream switch you're now taking out the one that sits behind it as well right okay anything hanging down yeah yeah so that's an issue uh spanning tree itself i guess as long as you're just single connecting everything you don't have you've got less to worry about but then you've also got that layer at the bottom where who knows what people are doing down there and can plug in whatever they want but yeah it's it's just kind of a management mess and it's it's just not very clean from what what i've seen i try to stack where necessary but you know sometimes you've got to get connectivity to an area that doesn't have any and you may not have an uplink or port count available up at that distribution layer switch so you know what do we as network engineers do we we just make it work right so kind of an off off topic on topic kind of whatever sideways question is how did you get so knowledgeable with design because you know full transparency here like i feel like design is definitely a weakness of mine um there wasn't a ton in the ccna which you know was my my big cert coming in and it's basically what we're talking about the three tier layer right there you know access distribution core they don't go like too deep into any of it i've never designed a network and the networks i've worked on were either you know isp or wan i really was just concerned with getting a router out on an edge you know on a circuit so like how did you have you designed networks was this is this just through studying and like how do you become a design knowledgeable engineer instead of so first off i i don't know if knowledge is the right word for the the crap i'm spewing right now but uh i i guess it's it's just experience andy you have typically been on the you know the data center side the lan side that kind of thing i i've been the opposite uh i have been in the enterprise campus uh for close to 10 years now so it's a lot of it's just doing it um seeing that environment what doesn't work yeah yeah and we've had sites that have come up uh in the last few years where i've gotten to kind of take the lead on what the design is going to look like how we're going to implement it that kind of thing so yeah it's it's really just opportunity and and yeah some of it comes from study but a lot of it's just getting in getting your hands dirty and there's a question too i mean does design change much like you've been working in this for 10 years and i mean i know that there's been shifts you know what is it like every 10 20 years we just change things right like i mean are we still doing access distribution and core i know the spine leaf is a thing but that's more data center we'll get to that later so like are most enterprises still doing the three-tier i will say yes three-tier is still there uh in in the campus to to aggregation to kind of connect back to the the central point of the core i will say that certain aspects are changing and we'll get into that with uh with moving into overlays and fabrics and and that kind of thing but i think the core access or sorry core distribution access or at least distribution access if we're talking a single physical building is is still going to be there whether you're doing traditional networking or fabrics and overlays for sure so now that we've covered the access layer kind of in somewhat depth let's let's talk the uplinks up to the distribution layer so dan already kind of highlighted it in that where you can where you can afford it you want to have redundancy upstream from the access layer so ideally in a in a campus building you would have uh two at least physical uh distribution switches and we'll get into whether they stay physically too or or logically to or logically one but physically you want to have connectivity to both of them from the access layer and what does or what can that connectivity from above layer one look like and that's a debate that i like to have often so if anybody wants to get into that let's let's get into it so in a traditional design back in the day we would always we were always taught that your layer 2 to layer 3 boundary was at your distribution layer and that you would have trunked vlans down to the access layer and you could have you know layer two in in multiple across multiple switches across multiple closets if you had the requirement for a device that had to move around that had to maintain its same layer 2 layer 3 connectivity that's how you would handle that typically now andy what uh what is a constraint there if you got layer 2 stretched everywhere what do we got to deal with uh spanning tree protocol no now before i i get on i'm stealing andy soapbox tonight before before i get on the soapbox and and hurdle myself off of it here in a minute spanning tree does a job of making sure that we don't have layer two loops without spanning tree frames could essentially be sent endlessly if there are unknown unicast frames if a broadcast packet comes in at layer two it has to flood it out all ports except the port it came in on and without spanning tree blocking redundant links you would take your network down constantly because your your switches would always have to process and cpu all of that traffic and know where to send it and constantly be changing and your mac tables get ins unstable unstable whatever stability issues why didn't they put a ttl in frames is there any reason because then i think that's a fantastic question and i don't know the answer to it i'm sure people smarter than me did it for a reason but you know and why haven't they done it since i mean i guess everything's just there and they can't change it now but yeah so spanning trees so before i i get into my my detrimental spanning tree bit spanning tree has a job and it does it very well where it can get complex is more to me on the the person that's doing the design and depending on who you have that's that's building out your network doing the design they may not fully understand spanning tree and and how you should try to strategically place your primary and secondary route bridges and it just adds a adds a lot of complexity so is this directed at me no no i'm the same way dude before i had a good grasp on on spanning tree dude i despised it i was afraid to do anything outside of uh you know a scheduled downtime maintenance window if i was gonna be adding switches because i didn't fully understand uh where my root bridge was and what it meant if i you know i thought it was whatever i plugged in it was gonna cause it to have to figure out its its new path and everything it freaked me out before i really got a good grasp on it so and i've screwed it up and i shared that with the off camera that's that's why i mentioned it i mean i i i i i built infrastructure no but it's there's you know there's truth in it i built infrastructure and i didn't even i i forgot to set a route like i just the thought never occurred to me because i deal with player two so infrequently and i had to build some and i'm like oh we'll just you know just put this all up and then ran in all kinds of problems because the route where was wasn't where it's supposed to be and then root guard screwed me all up and so yeah like i'm i'm that guy right who doesn't understand design well enough that i i exactly what you're saying if you don't understand it you don't know where to make those decisions you can get into trouble and i have a lot of truth there i mean shameless shameless plug to the show but there really is an art to designing spanning tree if you really have to take uh vlans and allow them to go across multiple access layer switches you really have to take some time and make sure you understand the design and at a high level design tips for for that is you typically have your primary and secondary root bridges at your distribution layer and set those with either strict priority levels you know in the increments of the 4096 or root primary root secondary and then your access layer switches would be responsible for blocking one of their uplinks so but if you have spanned layer two across you know pretty flat network across multiple switches that's where it gets dangerous because you have to make sure you're managing it well it's not that spanning tree is screwing up and doing something wrong you're just making your network complex and making it more difficult to to keep a loop free topology do you put all your vlans on all your access switches i've seen it done both ways some people only use what they need and then other you know all the vlans in that environment they want on all the switches is there like a best practice what i need yeah okay so if you need some in one place and then some in another place you don't put all them yeah right okay yeah like if you break if you break uh if you break it up by you know floors or something like that yeah this floor only has these vlans and like if you break up the floor in sections and whatnot maybe that section only has certain vlans and this section has a certain vlans so and that's something that's something else i want to talk about is that with spanning tree we have to maintain a loop free topology right so some links some redundant links have to go into a blocking state so you're essentially not using them so that's where with spanning tree in different vlans and root bridges if you're running your primary and secondary root bridges on your distribution layer that's where you can kind of get creative with for example odd vlan's distribution switch number one would be the root even vlan's distribution switch number two can be the root and by doing that your kind of poor man's load balancing uh and using both distribution layer switches because the alternative is that distribution switch number two is really just a a hot standby and isn't in use at all so you kind of have to figure out do i want just kind of that hot standby or do since i'm paying for that switch and paying for the uplinks i want to be able to use them both and that's where you can get kind of creative with with where you place your primary and secondary root bridges and and how you handle your your vlans but yeah dan and andy i'm with you both i i only like to create and trunk vlans uh where necessary and i saw somebody jordan typed the words vtp in the chat here yeah and i i had revolution numbers for a minute right yeah yeah i i don't i don't want to um i don't know if this is the place or not but is the new jam player three down to the access and then you just get rid of spanning tree i do want to want to cover that yes i i'm a big fan of that andy uh again not because i think spanning tree is bad spanning tree does what it has to do well you could use the links you're not leaving right right yeah so that's the big advantage the reason i appreciate layer 3 to the access layer in a in a traditional networking standpoint is one anytime i don't have to think about spanning tree i'm happy and in that case i don't have to and then two you're able to leverage those links natively because you can do equal costs multipath right right um but you're letting the routing protocol decide but you're letting the writing protocol you just you're just giving it a link and it decides right right so to that end where we're talking about being able to leverage so let's say you wanted you have a design requirement you have devices that have to be able to be mobile across different switches have to maintain the same vlan same layer 2 same layer three but you don't want to uh sacrifice bandwidth you wanna be able to use all the possible links there are some options around there and again i'm gonna be coming from a from a cisco standpoint but dan you maybe you can get into this you mentioned something a little bit earlier called a multi-chassis ether channel you want to kind of get into that yeah so like on now i'm not familiar with more of like the catalyst side so i think that's vss right yes yeah vss uh was the technology that i think was around um kind of in the catalyst 6500 catalyst 4500x and it's now stack wise virtual with the catalyst 9k and somebody can can correct me if i'm wrong i don't know what or if there are major differences between the two other than the name but they're essentially the same thing and it's it's treating your distribution layer switches separate physically but logically one yeah now what i'm actually more familiar with is uh the nexus using vpc virtual port channel um so i'm more i'm more used to that but basically what it is is you're taking two physical chassis and you're making them one logical chassis and so when you when you connect two or more links between these two uh physical switches to a single another single switch what it logically looks like is a switch to switch connection basically that's in a ether channel yep and that's exactly what it is so with the access layer you would configure you know two you would have two physical links but they would be bonded in a in a port channel so you could do um just straight manual mode on or you could do lacp or cisco you could do uh peg p that kind of thing but that that's how you can still have a layer two topology to the access layer but from access to distribution the access layer just sees that hey i got a port channel that looks like one switch to me i can use my my links in the port channel and that's how you can you can still get that bandwidth by maintaining a layer 2 topology now there's pros and cons to that and i would love to we probably don't have time to go through it here but i would love to compare and contrast and get your guys's opinions on on vss versus uh vpc and in the the pros and cons to each of those but there are some caveats that that you want to think about too because it's it's nice that you're managing one device it's a single con management plane it's a single control plane but you also kind of got to think about the other side of that too um go ahead dan yeah so that's one difference between like nexus vpc right it's two logical switches just that connection the vpc connection is the layers yeah uh because you still manage the two different devices yep so just real quick because i i want to i want to piss off roddy our old guest when you're doing routed links between access and distribution you guys are definitely using slash 31s right on this point to points andy i i i am a convert so ah pre pre recently i was all about the the slash 30s because that's what you were talking about that's what the book said right that's what the book says you might as well use four ips on a link that needs to i guess exactly because you have to andy you have to because you need broadcasts on point yeah you need a broadcast right you have to andy because there's so many places to talk to in that broadcast on the other side of the damn wire there's only one place to go that's the point so roddy that's for you anyway we all use slash 31s here at the art of network engineering from point to point links i am a convert to the slash 31s i've got uh something good something else that i got coming up and i actually just threw it in in eve within the last week and and yeah to plan it out just one ip after another man slash 31. i do find it funny though that when you you throw that in there on the on the cisco config and it uh it gives you some grief it's like hey you better be using just a point-to-point link or we're going to have problems yeah yeah warning you i do find that funny but uh sorry i didn't mean to bring us down that i do wish they were a little more savage on their on those responses like bro you're about to break stuff that would be cool who do you think you are yeah so again i'm talking purely traditional networking yeah i i like having routed access so layer three to the edge distribution access but you you have to make sure that if you're gonna go that route that you pun intended if you're gonna go that route that you don't have any of those uh what i'll call legacy applications technologies that require that layer two mobility because you're gonna have problems if if you have to support that can you can you elaborate quickly on that because i've heard that before w okay i've never asked because it sounds like one of those things i'm supposed to know but is it like certain applications need layer two throughout the day so so uh an example there's a yeah go ahead and go ahead tim i was just saying an example would be how an application's coded on uh that's running on multiple devices where they have to communicate with each other and they can only communicate on the same layer two they can't the traffic can't be routed um another that's the way the application is written it's just the layers yeah something so andy back in the day things didn't use actual ip yeah right so like uh you could have like sna right okay yeah and and so that's where you have to have something like dlsw to come in to actually route that traffic right but if you have your access switches at layer three now you're gonna have to have a router that can do dslw connected to that so it's like is it legacy support or there's like modern applications that need layer two legacy like i always thought i wouldn't say modern yeah i i mean there's a lot of legacy stuff out there yeah i would like to say legacy as well another example would be let's say you've got a really old device or a group of really old devices that have to move around plug into different access layer switches and they can only be configured with static ips they can't do dhcp so so how would you even know like would you have to talk to how would you know if your applications require layer two like if you're like okay we're gonna go to layer three to use all of our links and then you do that and then everything breaks and that's where that's where it gets dangerous and i think that's where a lot of people shy away from that and right the the cisco books uh the cisco press books they'll they'll say that you know layer three to the edge is is the golden child but adoption is difficult because of those those requirements people may not even know about and and they don't want to go layer three all the way because they don't want to get it all done and then something come in and say well that breaks my design that's where the the overlays and the fabrics come in where you can have layer three all the way to the edge from core to distribution edge and if you need to support those legacy applications you can run cisco's sd access or do lisp vxlan whatever on your own that there's other ways around that and i and i think that's kind of the future of what you're going to see more routed access designs with overlays and fabrics over the top now one thing too that i want to mention is like what are some of the cons of moving layer three to your access layer because i know the first one i'm thinking of is going to be your cost right because now you have to have layer three switches uh in every one of your closets and you know layer three switch is generally more expensive than a layer two switch yeah that's a really good point dan i i think other than the layer 2 dependency thing i i would say cost is is a big thing if you're an organization that has spent their entire time leveraging really low cost layer two very lightweight switches at the at the access layer and you all of a sudden want to go to a layer three access design then yeah that's your you may need a switch upgrade and not only a switch upgrade but but much potentially much more expensive switches that's a really good point and then do you introduce any kind of like overhead when it comes to your l3 design now um how do you mean so like okay you had to design out your stp right and and all that so now instead of spending that time on stp now you're just spending that time on an l3 so like if you're using ospf or you have grp yeah that is a good point it is something to to take into account because with a layer 3 access design you're going to be first you need to have that internal debate on whether you're going to use slash 30s or 31s but then you need to come up with that design of how you're going to ip things because you're using more layer three you're using more ip addresses now you're using technically to your point dan more routers because your access layer switches become routers if you're running ospf isis eigrp they become nodes in that in that cluster and that's where you know if we're talking purely ospf you can start playing around with different areas summarization so i mean you can add complexity there as well it doesn't necessarily mean if you're doing layer three to the access yeah i'm getting rid of spanning tree i have no complexity anymore that's that's not the case and with the with the overlays and fabrics that's something i want to touch on too because that sounds awesome right i can have layer 3 everywhere and i can extend my quote extend my layer 2 over layer 3 with with overlays and it and it looks great because cisco or whatever vendor just does it for me well to me that's not that's not getting rid of complexity that's that's really just masking or hiding it right it's still there it's just potentially under the covers so this this may or may not be a good segue to distribution layer but if you are running layer two you know between access and distribution you have to terminate layer three somewhere right like where do you put your sbis do they go on the distribution yes if you're doing the real traditional uh where the distribution layer is your l2 to l3 boundary then yeah that's that's where they would live and you nailed it andy it would be um at least in the cisco world it would be switch virtual interfaces and that's the main function of the distribution layer right beyond the physical redundancies that if you know one distribution falls over you still have links yeah on the axis on the other yep yeah it's to it's to aggregate your access layer switches and to get them connectivity to the rest of the organization get them up to the core to get to the rest of the organization and traffic network uh says utilizing stubs and and multi areas to reduce uh routing levels routing tables yeah i agree with that 100 i love uh ospf design i could talk about that for days uh and that's another reason where if you're getting into layer three to the access and you're all of a sudden technically bringing in more routers that's something you want to consider is how am i going to i p things how because you you don't want massive routing tables you're going to want to do some summarization so again the complexity doesn't go away with a routed access design there's there's still plenty of things to think about you have to watch what you say on the show tim because i just penciled you in for an ospf episode that you'll be hosting you just said i could talk about ospf for hours got it i didn't say it was correct new episode that that's one of my favorite things to to lab up is ospf that stuff's fun so i think we've kind of talked to the from a campus side the access and and distribution layer now where you would have that core layer to me is when you have multiple sites that have distribution layers and you need to connect them back to a central point for instance a data center to get to the rest of the compute infrastructure to get out to the internet that kind of thing and i i will kind of come out and say it is typically the the design there is that where while you can have potentially layer two from distribution to access or layer three from distribution access typically your design is gonna be layer three from distribution to core once you get to the core the core you just want to route packets as quickly as possible you're not going to have a whole lot of features extra features running security that kind of stuff you want that to be handled elsewhere your core is just getting packets from one point of the network to the other so you really want to stay away from vlans svis you know if you got to have multiple sub links you can run layer 3 sub interfaces but it's it's really trying to keep that a routed core as much as possible yeah i was going to say uh before i knew anything about like the three-tiered design and whatnot um i always thought like core that that's going to be where all the fun's at uh same and then once i started learning about it i was like oh actually that's not where the fun is it's you just put a couple of routes in there or set up your uh set up your routing protocol and then uh that's about it yeah i'm gonna make sure everything's super redundant and you you really and i'm with you today it's kind of counter-intuitive you think it's the core it's it's the big dog it should be doing all the things and really you want that thing to be as simple and redundant as possible mm-hmm yeah yeah you want to be able to yank the power on uh one of your cores and nothing blimp so this might sound ridiculous i've never really understood the purpose of the core i had somebody tell me once it's like the glue it kind of ties everything together is that an accurate statement i mean yes so i know in my environment behind the core all the applications so you know that's where the compartments are with all the stuff built and then that gets you out to the outside world and then you know everything seems to congregate in the core but like you said dan like it's all it's not a lot going on in the core it's just some routes it's fast switching so like do you just need the cord a certain size like you said earlier tim it's the amount of distribution like is it the amount of distribution switches you have determines if you need either a uh you know a core or a collapse core right like there's that whole decision to be made yeah that's definitely i think the way to think about it and it's it's all about scale and it's not to me it's not necessarily just distribution layer blocks like it at campus sites it's once you get into the data center as well because from a cisco perspective there's you know your access block your distribution block but once you get into the data center you can have an internet block that you know is is routers that specifically connect to the internet you can andy which you're familiar with the wan block where you have uh wand routers that that connect mpls to multiple sites and and other technologies as well you can have the the compute block that connects to your your data center servers you could even have a network services block if you want your um things like your security load balancer load balancer dns dhcp if you don't want that in the the regular server block that that could be separate too and so to me all of those different blocks would have kind of a distribution layer and they need to connect to something to be able to communicate with each other communicate to the other sites communicate to the internet and that's where the core comes in and it's really i think glue is a good word because it's you want something that can handle that level of traffic however you need to size it but it's really just switching packets at that point so it's running routing protocols but really you don't want it to do anything much fancier than that yeah i think one thing that helped me a lot was um jeff kish on cbt nuggets he did the encore series and um or i think they kind of mixed a bunch of them that did the encore series but he did at least the first part of it right and one thing he was talking about was you know network design and he was talking about this this uh basically what you're talking about tim these blocks right like you've got your campus block you've got your wan block you've got your internet block you've got your data center block and the core just really tied all of that back together and and that for his explanation of that which you pretty much you nailed it tim i think daniel no no no i think i'm not trying to like re-say what you said but visualizing it that way is what made it click in my head a lot easier now because i think about it dan i'm pretty sure what i just said came from kish's video yeah oh nice i'm almost positive so uh i definitely would recommend watching that he may have said the glue to me because i remember watching that video too studying for on corey we might just so yeah so i knew we were gonna spend or i should have known we would spend this much uh or this amount of time on uh our layer two to layer three debate at the campus but i do we don't have to spend a whole lot of time but i i want to at least touch on uh some some wireless infrastructure design just kind of at least at a high level there's uh multiple ways you can handle this the first is an autonomous design where you have access points that that are just that yeah i know dan i know what happened dan what happened having a bit of a clamped talk amongst yourself so what are we talking about here wireless design yeah we're talking about wireless infrastructure uh and it's really ap client to ap to the rest of the the network and the internet connectivity and kind of the old school school of thought is aps that are autonomous and they're just that they're they're managed on their own there's no like tied to the other keys there's no so does that mean no controller when you say yeah no no controller the ssid you literally log into the into the ap and you do your config yeah okay it is a nightmare yeah ask me how i know oh yeah that doesn't sound good so you can have the same ssid on multiple autonomous aps but they're really not working together at all they're they're just serving up the same network yeah you're setting up your your channels and all that stuff on each individual ap sounds like my house and it's awful yeah yeah try try to get more than like five ap it's just it's a nightmare it sounds bad and because of that that's kind of where the the lightweight model was born and that's where you have uh what they'll call the split mac architecture where you'll have an ap and ap will connect to the controller the ap is just that it's really lightweight there's not a whole lot of brains on the ap all data and control plane traffic in local mode of of lightweight mode has to go back to a centralized controller and that's where more infrastructure design comes in if you are a centralized company pretty much anything has to go in and out of a data center having one or multiple wlc wireless lane controllers wlc's in a data center makes sense another option if you don't want to trombone traffic all the way back to a data center you can have distributed wireless lane controllers but in local mode it's sending all of that data and control plane traffic somebody device a wants to talk to device b even if they're sitting on the same ap next to each other that traffic data traffic has to go back to the ap through the switches to the wireless lane controller and that's where it gets switched that's where the vlans live connects to uh exterior switches to connect to svis that kind of thing that's where traffic would get switched and routed and would have to come back to that same ap to that other client so there's pros and cons to that yeah and but you also have to consider your design when you do that uh but well because if you're because you're using cap wap at this point right and you're making basically the the user that is sitting on wireless is now sitting wherever the wireless lane controller is you know logically yes that's where they're sitting and so there's some design thought that you have to put into that do you want your wireless lane controller in your data center because now you don't have a separation like with a firewall or something like that between your users and your in your servers right so it's just some things to consider and that being said is kind of an enhancement to that is where cisco calls flex connect and that is kind of a hybrid lightweight design so with uh one of the options with flex connect is that you can tunnel all of your control plane traffic so all of the auto rf the ap is talking to each other to be able to say hey you should be on this channel i'm going to be on this channel power levels that kind of thing all of that control plane traffic still gets tunneled back to a controller but you can locally switch your traffic you can have vlans locally and and you can switch all that traffic locally so it's kind of kind of a hybrid design and then dan i wanted to see i probably shouldn't call you out because i think you have experience in this i shouldn't assume but um you have kind of a what's my safe word you uh tickle your fancy i think that's what it is okay so another model that i think is has kind of been coming more relevant these last few years is kind of a cloud-based model where yeah do you want to get into that because you probably have a lot more experience than i do yeah so my my experiences with uh cisco meraki right and so basically you set it up in the con in the uh in their dashboard and when you join a device to the network it'll pull its config from that dashboard right so now am i am i am i tunneling my traffic to the cloud my user traffic uh no no so it's really just that management plane configuring the devices is happening through that interface okay yeah i don't want to send my i don't want to send my users uh traffic up and back from the cloud bro i i have a serious design question for wireless and i apologize if it doesn't exactly fall in where it should here but it's for my house so i have a bunch of autonomous access points they're legit access points that i may or may not have procured from a certain isps cable truck but they're autonomous and they're great but it's so stupid and people say it's because of the iphone but i could be standing next to one of them in another room and it's still stuck to the one on the other side of the house with like one bar so my question i guess is and because i don't know enough about wireless i always thought well if i kind of try to do an enterprise level gig at the house like if i got legit access points tied into a centralized controller does that make the wireless experience any better so that my stupid phone isn't still connected to something on the other side of the house when i'm standing in front of one does that make sense so i i think there's two things there um one thing the first thing i'll talk about is that i've learned is that um roaming decisions while your aps and if you have controllers while they can try to influence clients it's really up to the client to determine when it's going to run okay so yeah when we get those kind of uh incidents that's one of the first things we we look at is is try to get the end user computing team involved hey let's let's check the the nic driver hope it's not 10 years old hope it's set for you know somewhat aggressive roaming that kind of thing but i i liked in those scenarios like to start with a client yeah now to your point about kind of a more enterprise thing where the aps are connected to a controller where i think that's beneficial is that what i call or what i call what cisco calls the radio resource management rrm and that's where your aps are are talking to each other to determine who should use what channel what power level each of them should run at so you're making sure that proper coverage but not yeah blasting each other out of the water right okay so it's not going to solve this iphone from just being sticky to whatever i i won't say i won't say it won't i i think it's i mean enterprises is gonna provide you that that benefit of your aps not being autonomous and being able to to try to work with each other instead of just being there it may help but it might be the client too right it could be a stinking phone yeah tim you you might know this um but like when you have band steering and what not turned on isn't it technically the ap just trying to well so it's still the client's choice right if it's going to steer but isn't it the ap basically saying like hey are you sure you want to use two point yeah yeah so and it does that like multiple times basically in in hopes that the client just says okay fine i'll use five is that yeah so there's band steering and then another one is uh on the cisco side you can do client load balancing and and that's where you got to be kind of careful because so band steering and client load balancing it's it's the ap or the controller trying to influence the client from band steering hey i want you on five gig but if the client says x amount of times no i'm two four no i'm two four no i'm two four then the ap and the controller is eventually going to let it on otherwise it'll never potentially never get on and yeah and load balancing uh is is a similar feature in that let's say you have two aps somewhat close to each other a client comes on tries to join ap number one ap number one may have 20 clients on it already ap number two that's in a somewhat close range good rssi may only have four clients on it the ap can try to influence the client say no back off try another one try another one but if the client says no dude i want i want to get on your ap the ap has to eventually let it on okay so the there's when it comes to connectivity there's a lot of things that the wireless infrastructure can do to try to influence the clients but it's ultimately the client's decision gotcha right that's a lot more wireless than i ever talk about sorry i pulled this down stretched them legs i can't wait for all that hate mail to hit aj's twitter account yeah well fellas i underestimated uh the amount of time i would stand on top of my soapbox talking layer two layer three to the access so andy you can have that soapbox back what i think we should do guys is is turn this into a bit of a series and in the next one i think we should dive into the data center maybe a little security uh we'll see how crazy we get what do you think yeah yeah 100 for sure how did you like being up on the soapbox it's nice right it's plenty of room it's yeah the the air is is nice up here i feel a little feisty too right yeah andy i could get used to this man i'm not gonna take it from you but i might borrow it from time to time you're welcome any time man it was really nice having you you guys uh got anything to to help finish this out i just want to thank you man i uh yeah i need to have more design discussions like this this is really helpful for me um i'm actually going to take the design um you know specialty after encore for my ccmp i'm right with you dude i'm doing the same thing yeah because i'd like to know more about it so the more discussions i have about this and you did a really good job at you know keeping me uh keeping me engaged with it so i appreciate it man it's good job yeah i think from uh from a podcast point of view uh you did really good steering this conversation tim uh we don't need daddy aj don't say that i don't you know when he watches the and listens to the unedited cut of this dan i don't think he's gonna agree with you yeah you're probably right tim was this the first art of network engineering podcast episode you hosted this was your baby right this was it i don't know i don't remember yesterday man i i think so you did a great job man i appreciate it guys this is this was fun awesome and with that thank you for joining us in design part one we will be back for design part two where we will dive further into the data center and some high level design tips so join us next time on the art of network engineering hey everyone this is aj if you like what you heard today then make sure you subscribe to our podcast and your favorite podcatcher smash that bell icon to get notified of all of our future episodes also follow us on twitter and instagram we are at art of net edge that's art of n-e-t-e-n-g you can also find us on the web at art of network engineering dot com where we post all of our show notes you can read blog articles from the co-hosts and guests and also a lot more news and info from the networking world thanks for listening you