I tend to do silly stuff with computer networks, and I thought it might be
worth writing about some of that stuff here. So I will start the possible
series by explaining here, what my home network and general infrastructure
looks like. It's not that special, but there are probably some interesting
quirks nonetheless.
My requirements and situation
Infrastructure around me now and in the past
I have already had my own network for about 8 years. I started at the
dormitory, where the uplink was mostly an eduroam WiFi connection. Now I live
in a flat with VDSL uplink providing only IPv4.
Both of these environments have several things in common: the uplink is not
friendly to having non-trivial IPv6 networks (currently there is none, on
dormitories there was SLAAC and blocked communication between clients), which
means the simplest thing is to still have IPv4-only network behind a NA(P)T.
Another common thing is that both environments have roommates. These pesky
creatures tend to demand internet connection, and I get to be the one to fix
that (now even more than before, because all we have is a single Cat 5 cable
from a modem-bridge with no user-facing functions). So my infra has to work, at
least somewhat, most of the time.
I also use the network and computers at my university, had a job as a sysadmin,
have a VPS and occasionally set up network at random events at remote parts of
Czechia. The last part will be interesting in a moment.
Everything is a server and a router
Most, if not all of my machines are routers. Yes, that includes smartphones.
(No, it does not include a Nokia 5110, unfortunately.)
That means that even my laptops, desktops and smartphones have a need for
dedicated subnets (two for each: one for virtual machines and containers, other
for random devices that get connected to them, either because I am working with
the connected device or because my machine is currently providing internet
connectivity to my other devices). And that includes a DHCP server – sometimes
the random device just asks for an IP address or netboot and you need to
provide that. (At least twice I have connected a device, didn't know what to
expect, looked into Wireshark and saw a DHCPDISCOVER message.)
And when I say router, I mean it should route (exchange routing information
with the rest of the network) at my sites, not merely forward (send packets
to the correct network interface) according to a static routing table. This is
needed, because as devices come and go and their dedicated networks appear and
disappear, it would be hard to make sure packets can correctly reach their
destination and not get lost or misforwarded. (This pretty much disqualifies
any off-the-shelf SOHO-grade routers for me, because they tend to not route,
they DHCP.)
(Almost) everything moves and meets
Having phones and laptops be on the move and sometimes meeting in the same
network segment is probably quite common. However, being a poor student at the
time, when both me and my roommate were at a distant event that needed network
equipment, the main router also went with us. There is nothing stopping me from
making the desktop (which is the least mobile part of my infrastructure, so it
usually stays home) substitute the router as the gateway.
And speaking of events: I ended up being the main sysadmin on some of them as
well, which among other things involved taking care of the group's laptops and
server between the events. Which means that I needed to have a good
approximation of the event's network at hand whenever I wanted to work on these
machines.
Other nice-to-have stuff
I have multiple machines that most of the time contain different data. So it's
useful to be able to connect to them and get my data. This is not limited to
when I'm geographically separated from them, but also whenever I need data from
one machine on another machine across the room.
Also, I don't like changing the configuration too much, so if I can make one
configuration to work in all cases (except like when I need an ad-hoc DHCP and
web server), I like that.
The actual configuration
Quite naturally, I have a main home network for my devices. This is where I
connect most of them, and while it has dynamic address configuration, I mostly
assign static addresses to my devices. If I have a long-term or
network-wide experiment (like deploying home.arpa. domain names), it
usually happens here.
Given the need for routing, there is also a routing subnet, which is intended
as a transit network to have the “ordinary” (stub) networks have fixed number
of gateways and not confuse routing-unaware devices. It runs OSPF .
I say intended, because it actually is not – by default, any device connected
to the routing subnet will find that the default route is through that subnet
and use its routing IP addres as source address for anything off-link. That is
not the correct behaviour, but given that these addresses are static, I don't
really mind.
Then all devices have the two helper networks: one for virtualisation (VMs,
containers and whatever running on that machine itself), one for auxiliary
connections (other devices that need to be temporarily connected).
And then there are the event subnets, which also serve as the network for my
roommate. This is the stuff that (i) needs to work, (ii) sometimes travels with
me and (iii) I try to not break (too often and for too long). And this is where
the trickiness comes in :-)
The event infrastructure
There are two event subnets – one for the organisers, one for the attendees –
but only the one for organisers is important most of the time (the attendee one
is provided on best effort basis, only has foreign devices and provides like
one local service which is also present in the organiser subnet).
My router-device on these events does not route, only serves as WiFi access
point. DHCP server is elsewhere, it has no idea of where the gateway for this
network is.
And I want to be able to take the router from home, make little to no changes
and have both roommates network and the event network work.
The migrating router
Naturally, I have a spare router (slower and less tunable MikroTik) to serve my
home when the main router is away. And while the main router is also the DHCP
server, DNS resolver and gateway when it's home, it does not do any of that for
the event and roommate's network by default. The DNS resolver and routing still
runs on it, but this network has external DHCP server that either tell the
client to use these features (at home), or doesn't.
And I am quite picky about DHCP servers (I want to refer a netbooting device to
the bootable file depending on whether it runs EFI, x86 BIOS or whatever else
in the firmware), so the main DHCP server is not actually runnig the MikroTik,
but it's another instance of ISC's dhcpd on my desktop, which is connected both
to my networks and the event ones.
Falling back
Because my main router occasionally goes travelling, the network needs to
function without it. There are several parts to that:
- The dynamic routing ensures that the present devices that route dynamicaly
can reach what they want.
- DHCP actually can give multiple routers for the given network, and iirc
both NetworkManager and dhcpcd do add multiple default routes in that case.
- Uplink is a PPPoE, which means that I can switch it to both routers and
configure them to use that uplink. It is not possible to use both at once,
but one will win the race and the other will either try periodically, or
retries with a simple nudge. (I don't think I can ensure the main router
wins the race, but I can reboot the routers in correct order if needed. And
the uplink is still slower than the slower router, so there is no bottleneck
anyway.)