2023 Internet Setup

Hanging out at anchor looking up at our cell antennas and Starlinks

As time marches on, so do our needs for internet reliability and speeds. I’ve been passing around our original post for a while now, but at this point virtually nothing on the boat remains from it, so it seemed like time for an entirely new post…

Routing and Cellular Modems

The core of our latest system is still Peplink-based, but we’ve moved ahead to the 5G device generation. We use the MBX Mini 5G as our primary appliance — it supports 2 wired WANs, 2 2.4/5G wifi connections for WAP or WiFi-as-WAN, and has 2 built-in 5G 4×4 cellular modems. Unlike the Transit line, it has a much more powerful processor that’s able to keep up with the bandwidth its modems can provide, even when using Speedfusion to bond multiple lines. We still want to maintain the three-cellular plan setup (Verizon, AT&T, and T-Mobile), so we also have added a MAX Adapter 5G to augment the two built-in modems in the MBX Mini. Note that the MBX Mini 5G is a very expensive appliance, and, since we bought it, Pepwave has added a BR2 Pro to its lineup, which is slightly over half the price, and theoretically has all the features that we care about from the MBX Mini.

The MBX Mini 5G and MAX Adapter 5G on the ceiling right below the antennas

The two units mount on the ceiling right under our radar/antenna mast, which is a folding unit to be lowerable to clear all the low bridges on the east coast. This setup complicates the need to run as-short-as-possible wires between the cell modems and the antennas, so I had to get some very short extensions to reach between the two. I also wanted to leave the units exposed both to be able to change out SIM cards (which happens annoyingly often as these different MVNOs all go the way of the dodo) and also because the MBX actually gets very hot, just like the Transit Duo that came before it, so this was a nice lazy way to keep it “cooled”.

On-Boat WiFi

Since we’re using the WiFi ports on the MBX for WiFi-as-WAN (bringing outside internet into the internal network), we need some solution for WiFi on the boat. I have CAT6 run all over the boat from the MBX to have reliable hard links to work computers and for some simpler devices without wifi, but also to power an access point. I have a very simple gigabit switch centrally located in the boat, and I POE inject one of the lines off that down to an access point located in the ceiling panel above the center hallway of the boat, about as “middle of the boat” as one can get.

I originally used the AP One Rugged for an access point, both on our earlier Meridian boat and again with the first-gen internet setup on the TrawlerCat, but kept having issues with connectivity dropping out on it approximately daily and needing to hard reboot it to recover connectivity. I had a pile of communications with support at Peplink, and they were completely flummoxed by what was going on. They sent multiple RMA replacements that all had the same problem, did enhanced debugging, had me run special internal firmware builds, and nothing helped.

At the end of the day, fighting with Peplink was less important than reliable internet, so I gave up and bought a simple Ubiquiti WiFi 6 Lite unit, put it in the exact same spot as the AP One had been, and we’ve never had a problem since then. A big downside is that I don’t get AP management through the single Peplink portal, but our WiFi always works, so …

Antennas

For cellular antennas, we have updated to the latest generation of Peplink stuff: the Maritime 40G, which is a 4×4 MIMO unit w/built-in GPS antenna. Steve@SeaBits posted a nice article back in March 2022 testing out a bunch of the next-generation antennas, and these seemed like a good fit for us. One tiny benefit of this is that new antenna I was able to remove the dedicated GPS receiver taking up space on our radar mount tower. But mostly, it’s 4×4 MIMO, so you can use the latest quad-band-aggregation for higher speeds and reliability, and has great omnidirectional gain. On the downsides, not only do you now have to run 5 wires from each antenna through the radar tower and into the boat, but they’re also huge. But they are working better in every way than our older Poynting units.

For WiFi-as-WAN, given how incredibly rare it is that we care about using external wifi these days, I’m still using two of the same old Poynting 496 units (literally the last remnants of the old setup), but we go months at a time in between when I feel like trying them out, being disappointed in the shore-based internet, and turning them off again. So they’re pretty useless at this point, given the amazingness of the rest of the system.

Cell Plans

As mentioned above, we have plans with all 3 of the major carriers in the US: Verizon, AT&T, and T-Mobile.

For Verizon, we lucked out years ago and got an account on the pre-paid plan before they realized they were losing money and stopped selling new ones, but we’re still grandfathered in and using it. It’s definitely lower priority than our cell phones — during high congestion times in busy areas, it slows to low single-digit mbit speed, while our phones (on Verizon) can still pull high speeds. But for 65$/mo, we can’t complain. If we have to end up moving to one of the MVNOs for Verizon, we’ll end up paying over twice that.

For TMobile, I’ve been using atthotspot.com for an unlimited plan since June 2022, after our last MVNO shut down. The guy who owns it has been great for support and I haven’t had a single burp in service.

For AT&T, our last MVNO actually shut down in early January, and I just switched over to atthotspot.com for AT&T as well last week. So far it’s been working great, and only dealing with one MVNO at a time is a nice consolidation. AT&T has been locking down a lot of 3rd party routing equipment on their networks, so we actually had problems with the modem in the MBX Mini getting shut down within a few hours of activating, but after several days in the MAX Adapter, it seems to still be working fine, so crossing our fingers here.

I also keep a Google Fi data-only sim in one of the MBX’s secondary SIM slots to let me switch to it in case of emergency and whenever we bring the boat into non-US waters (until we can find a local SIM). It’s a great cheap backup network, and we do the same thing in our cell phones (primary Verizon, secondary GoogleFi).

Note that I’m not paid anything by the atthotspot.com folks for linking them, they’re just serving me well.

Starlink

One of the biggest changes in our general connectivity has been the addition of Starlink. We first got a residential dish in February 2022 after being on the waitlist for a while, and immediately integrated it into our cellular setup. I got the pole mount for it, used some simple pole clamps to clamp it to a vertical railing support on our Juliette balcony, and ran the wire through the wall into the boat to the Peplink router’s WAN 1 port. It worked great, immediately, and gave us a fairly fast (usually >50mbit) connection that was always up. It struggled a bit when we were under way or moving around on anchor — any time the boat turned significantly it would go offline for seconds to minutes as it reassessed the state of the world and reconnected, but otherwise was fairly solid.

We tested it that way for long enough to be sure that we had functioning hardware, and then quickly did the common POE conversion to save on power. Since we have a 48V house battery system, it’s actually a large efficiency savings for us to ditch the factory router and power brick setup and just go straight to running Starlink off the 48V system rather than through the inverter to 110 and then through a power brick back to 48V, as well as powering an extra router that I had no use for. I used the Tycon POE injector method, since it lets you use a cheap and very small enclosed/protected device, and all you have to do is flip a couple of the pairs when you crimp your ethernet cables both in and out of it. So the dish is DC-powered and uses a fairly low 35-ish watts most of the time. Small price to pay for the level of internet it gives us.

Early experimenting with the two Starlink dishes. They can never agree where to point…

However, in the boonies, where we had weak cell signal and were starting to rely on Starlink for connectivity, when under way especially, it still had significant dropouts, which was impacting our work. I ended up buying a second (RV this time) dish setup, and setting it up in the other corner of the Juliette balcony, POE-converting it, and setting it up as a second WAN. Also, as soon as I was sure the hardware was working, did the motor-disabling conversion, so it always lays flat.

People had been finding that if you disabled the motors, the dish basically just “did the best it could” at all times, rather than always trying to reorient toward the northern pole before getting connectivity back. This was theoretically resulting in increased connectivity for those in motion/rocking around on the water. The logic was fairly sound, but being a science-ey person, I wanted some proof myself. So we did the A-B test of the two dishes — one motor-disabled, one not. While floating around on anchor in a nice open-sky bay in Maine, the dish with motors working had its usual ~50+ connection-drops a day, usually in the 2-5 second range, but some in the 10+ second range. However, the dish that I disabled motors on?

Well. Okay then. We continued the experiment for several more weeks, and consistently got the same results — either zero or very close to zero outages on the motor-disabled dish, and the normal dish had consistent outages, especially under any kind of boat movement. I eventually called the experiment a success, disabled the motors on the second dish, and then both dishes had the same virtually-zero-outages behavior. Great success!

For the rest of our summer in Maine, while cellular was pretty spotty most of the time, Starlink is what kept us happily working, despite being miles from civilization. Working from Maine in 2020 was very difficult, and put a big damper on the spots we were able to spend weekdays. But the summer of 2022 had no such problems — we went anywhere we wanted, whether it had cell signal or not, and had consistently usable internet for multiple simultaneous video calls. It’s truly a paradigm shift in connectivity.

SpeedFusion

The SpeedFusion control panel, showing the different links being bonded

The last critical piece to our current setup is Peplink’s WAN-bonding system called SpeedFusion. It basically takes whatever WANs are currently working (we have 5 possible ones — 3 cellular and 2 starlinks), and for every packet to or from the internet, sends copies of it down multiple WANs (usually 2, but sometimes more, depending on your setup) to the SpeedFusion server in a cloud data center. From there, it takes whichever copy of the packet gets there first, ignores the other copies, and passes it onto the destination. When the destination sends you data back, it does exactly the same thing, but in reverse — sends multiple copies down to your boat, and whichever WAN delivers it to you first, it uses that, and the straggler copies are ignored.

The result is that, at the cost of duplicating data down your multiple connections, you get lowest-common-denominator behavior for your connectivity. If one of your connections has a hiccup for a second or two, you have absolutely no idea, because one of your other connections is still working, and the copies of the data on that link are still communicating happily. When you have inherently-slightly-unreliable connections like cellular or satellite, bonding multiple of them together and smoothing it with SpeedFusion is utterly critical to maintain the appearance of a single reliable internet connection. Our video calls are more stable and consistent than most people on home internet, because of the inherent redundancy that SpeedFusion gives you.

For quite a while, we used SpeedFusion only for video calls — the router has an option to just automatically forward zoom/teams meetings to SpeedFusion, leaving your other traffic to pick a random WAN to go out, stable or not. However, at some point, I got tired of manually managing which WANs were stable or not — you’d try to load a webpage and get nothing, go into the router config, and start trying out disabling different connections and refreshing the page and see when it worked well. While manually twiddling bits makes you feel like a hacker and all, sometimes you just want your shit to work without constant fiddling. So eventually, I bought a larger swath of SpeedFusion data (you pay for SpeedFusion by the gigabyte of data that goes through their servers) and routed ALL data through our computers through it, just leaving media devices (TV for netflix, tablets, etc.) to play internet roulette off speedfusion, since those are what use so much of our bandwidth and needs reliability the least.

In the end, this new setup costs a bit more money in SF costs, but it means that I’m virtually never messing with the router anymore. Things just work. Which is really the goal. Living on a boat brings enough challenges. Sometimes you just want to grab a beer and open up YouTube and know that it’s going to work, without futzing with connectivity for 5 minutes first.

Conclusion

A happy 5 WANs, sitting in Baltimore, MD’s inner harbor

With all of this firepower in place, it’s an incredibly rare day that we have under 50 megabits of downlink at any time, are usually between 150 and 250mbits, and periodically exceed 350-400. It’s actually fairly incredibly how fast the technology in this area has advanced. If you’d told me 5 years ago that we’d be able to do this today, on a boat, anywhere, I’d have said it was at least 5 years too early. And yet, here we are.

An interesting note we’ve learned is what the different parts of our system are good at. Starlink has started getting slower and slower in populated areas as they’ve oversold the network. When we first got Starlink, we were in northern Florida, and easily cleared 100mbit constantly. As the year went on, by late summer, in populated areas, we were often down in 5-10mbit territory until after midnight. However, in populated areas you also end up with great cellular connectivity, so the two systems actually complement each other very nicely. When Starlink is slow, multiple cellular connections are usually working great. Then, out in the middle of nowhere, cellular is only passable in an emergency, but we have two Starlink dishes each pulling over 100 mbits with no one around to share it with.

It’s working out unbelievably well.

Posted on Categories Planning

3 thoughts on “2023 Internet Setup”

  1. Sounds like SpaceX should push a software update for dishy on a boat that just locks the dish vertical. Should be easy enough to have an option in the settings to do that.

    1. The problem is that they’ve made it very clear in the TOS that in-motion use of these dishes is prohibited, and they’ve dabbled with disabling the dishes under motion in various firmware releases. They released a more expensive in-motion dish that I’ll get around to picking up at some point, but I don’t have a good spot to mount it — it’s bigger than the old dishes and will require some custom solutioning, but with the existing dishes working so well I haven’t bothered yet. 😀

Leave a Reply