State of 3G/4G fallback on MikroTik?
We need to quote to a client for two routers capable of running a VPN between two sites - easy, that's a MikroTik. But they want a fallback to 3G/4G so that the site that falls back keeps Internet access AND so that the VPN keeps running. What's the state of MikroTik 3G/4G fallback? Last I looked it seemed very roll-your-own, and supported only a few very specific dongles... Ideally it would work like the (vastly more expensive) Merakis and just fail everything over to the secondary link if the primary fails, where "fail" would be either a ping test or interface down. That said, I'm OK with a solution that needs more work, as long as once done it is set-and-forget. Any pointers? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
Hey Karl, There are a few dongles that work OK, Optus prepaid ones work, Telstra prepaid ones work, as long as you have a USB port a lot of the USB dongles work and just create a LTE interface once connected. The easy way to use them is to just create a low cost route to go via your primary link and a higher cost one via your VPN, if the main link goes down then the route is unavailable so your traffic will go via the 4G It's pretty straight forward, I think there are even examples in the Mikrotik Wiki as well. Regards Paul -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Karl Auer Sent: Wednesday, 9 October 2019 5:44 PM To: MikroTik Public <public@talk.mikrotik.com.au> Subject: [MT-AU Public] State of 3G/4G fallback on MikroTik? We need to quote to a client for two routers capable of running a VPN between two sites - easy, that's a MikroTik. But they want a fallback to 3G/4G so that the site that falls back keeps Internet access AND so that the VPN keeps running. What's the state of MikroTik 3G/4G fallback? Last I looked it seemed very roll-your-own, and supported only a few very specific dongles... Ideally it would work like the (vastly more expensive) Merakis and just fail everything over to the secondary link if the primary fails, where "fail" would be either a ping test or interface down. That said, I'm OK with a solution that needs more work, as long as once done it is set-and-forget. Any pointers? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi, As usual with Mikrotik it is roll-your-own but you can do this: https://wiki.mikrotik.com/wiki/Advanced_Routing_Failover_without_Scripting I'd test the heck out of it first though. If the failover happens bear in mind the connections will drop (maybe not ones in the VPN if it connects again fast enough) as the default route will change and it'll be on a completely different network. I have wanted to try the following though: * Set up a Cloud Hosted Router on a VPS with oodles of speed and data. Everything, including the Internet for all the sites runs through this central node. * Have each external Mikrotik router run a VPN on the main (say NBN) and 4G link back to this router. If the main link goes down there is an alternate route to the VPS over 4G but all the sessions keep going albeit with maybe a small delay while the internal route changes and noticeable speed and latency change. * The VPN between the two sites would be kept persistent as well. You'd be running RFC1918 addresses on your 'internal' network and all internet traffic from this network would appear to come from the public IP address of the CHR using masquerade NAT. I am sure OSPF would be used to make the routes work properly. If you used IPv6 you could have public addresses for all the devices on the network. * See below - I hope my ASCII art survives. /--NBNVPN---\ /--NBN VPN---\ SiteA CHR SiteB \--4G VPN---/ | \--4G VPN----/ | Public | Internet I have seen some business grade ISPs offer such persistent data links using NBN and 4G and I figure they do something similar to the above. Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/> On Wed, 9 Oct 2019, at 17:43, Karl Auer wrote:
We need to quote to a client for two routers capable of running a VPN between two sites - easy, that's a MikroTik. But they want a fallback to 3G/4G so that the site that falls back keeps Internet access AND so that the VPN keeps running.
What's the state of MikroTik 3G/4G fallback? Last I looked it seemed very roll-your-own, and supported only a few very specific dongles...
Ideally it would work like the (vastly more expensive) Merakis and just fail everything over to the secondary link if the primary fails, where "fail" would be either a ping test or interface down.
That said, I'm OK with a solution that needs more work, as long as once done it is set-and-forget.
Any pointers?
Regards, K.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi, You will probably need 2 vpn's, one in each direction. Perhaps one having a lower distance than the other. As unless you try very hard, you won't get a routable IP address on your 4g/3g modem, so it needs to dial out to the other end. If when both ends fail over you still require connectivity, a hosted Mikrotik VPS can be used as mentioned in an earlier reply. ikev2 notionally (and sometimes actually) should stay connected when one ends IP address changes. Regards Roger From: Karl Auer <kauer@nullarbor.com.au> To: MikroTik Public <public@talk.mikrotik.com.au> Date sent: Wed, 09 Oct 2019 17:43:41 +1100 Subject: [MT-AU Public] State of 3G/4G fallback on MikroTik? Send reply to: MikroTik Australia Public List <public@talk.mikrotik.com.au> [ Double-click this line for list subscription options ] We need to quote to a client for two routers capable of running a VPN between two sites - easy, that's a MikroTik. But they want a fallback to 3G/4G so that the site that falls back keeps Internet access AND so that the VPN keeps running. What's the state of MikroTik 3G/4G fallback? Last I looked it seemed very roll-your-own, and supported only a few very specific dongles... Ideally it would work like the (vastly more expensive) Merakis and just fail everything over to the secondary link if the primary fails, where "fail" would be either a ping test or interface down. That said, I'm OK with a solution that needs more work, as long as once done it is set-and-forget. Any pointers? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au---------------------------- Roger Plant
On Wed, 2019-10-09 at 09:44 +0000, Roger Plant wrote:
You will probably need 2 vpn's, one in each direction.
All in all, what I am hearing is "it's hard and you will have to do it yourself". This is very different to the (did I mention vastly more expensive?) Meraki solution that just works, out of the box, with a few clicks. But from the sound of it, it's the only realistic game in town. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
Yep, and Meraki is a hosted system where they're VPN'd back to a hosted VPS somewhere, yeah? Which is why they're so expensive? On Wed, 9 Oct 2019 at 21:39, Karl Auer <kauer@nullarbor.com.au> wrote:
On Wed, 2019-10-09 at 09:44 +0000, Roger Plant wrote:
You will probably need 2 vpn's, one in each direction.
All in all, what I am hearing is "it's hard and you will have to do it yourself".
This is very different to the (did I mention vastly more expensive?) Meraki solution that just works, out of the box, with a few clicks. But from the sound of it, it's the only realistic game in town.
Regards, K.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
-- Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder
Not to mention they drop dead when you don't pay your (regularly-renewing) license. James On Thu, 10 Oct 2019, at 05:11, Damien Gardner Jnr wrote:
Yep, and Meraki is a hosted system where they're VPN'd back to a hosted VPS somewhere, yeah?
Which is why they're so expensive?
On Wed, 9 Oct 2019 at 21:39, Karl Auer <kauer@nullarbor.com.au> wrote:
On Wed, 2019-10-09 at 09:44 +0000, Roger Plant wrote:
You will probably need 2 vpn's, one in each direction.
All in all, what I am hearing is "it's hard and you will have to do it yourself".
This is very different to the (did I mention vastly more expensive?) Meraki solution that just works, out of the box, with a few clicks. But from the sound of it, it's the only realistic game in town.
Regards, K.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
--
Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Apart from it's default configuration (which I mostly like), there are very few single click functions on a Mikrotik, almost everything, especially complex things take a bit (lot) of time and effort and googling for something similar you can either use directly or modify to suit. But with the effort, you do often get a lot back. Regards Roger From: Karl Auer <kauer@nullarbor.com.au> To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Date sent: Wed, 09 Oct 2019 21:38:08 +1100 Subject: Re: [MT-AU Public] State of 3G/4G fallback on MikroTik? Send reply to: MikroTik Australia Public List <public@talk.mikrotik.com.au> [ Double-click this line for list subscription options ] On Wed, 2019-10-09 at 09:44 +0000, Roger Plant wrote:
You will probably need 2 vpn's, one in each direction.
All in all, what I am hearing is "it's hard and you will have to do it yourself". This is very different to the (did I mention vastly more expensive?) Meraki solution that just works, out of the box, with a few clicks. But from the sound of it, it's the only realistic game in town. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au---------------------------- Roger Plant
While I like the idea of point and click and a solution ready made I do much prefer the scripting. I have this same setup with a client's multi-site setup where both sites have 4G backup and failover of the VPN. Essentially the script uses netwatch and very frequently checks the other end of the connection and then makes adjustments to routes/interfaces as needed on up/down. Users connected to a terminal server might see the reconnect message for maybe a couple seconds when it fails over, if at all (rare that they see it because it's quite quick). At other sites which are just your standard 4G failover, it's a dongle/Teltonika next to the tik and a netwatch script for adjusting route and clearing NAT table. Additionally I use a dummy eoip interface that is enabled/disabled on primary link up/down so that SNMP monitoring via Nagios can throw an alert quickly to show a site is operating on failover. Matt
Just revisiting this has anyone been able to get VPN's working between 4G linked devices over IPv6? Does Telstra's IPv6 allocation for 4G devices allow servers to be run or do they firewall all inbound connections to 4G devices? Their IPv4 CGNAT effective does the same thing as inbound firewall. If IPv6 works then an intermediate VPS running CHR isn't needed. Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/> On Thu, 10 Oct 2019, at 16:37, Matt Hare wrote:
While I like the idea of point and click and a solution ready made I do much prefer the scripting. I have this same setup with a client's multi-site setup where both sites have 4G backup and failover of the VPN. Essentially the script uses netwatch and very frequently checks the other end of the connection and then makes adjustments to routes/interfaces as needed on up/down. Users connected to a terminal server might see the reconnect message for maybe a couple seconds when it fails over, if at all (rare that they see it because it's quite quick).
At other sites which are just your standard 4G failover, it's a dongle/Teltonika next to the tik and a netwatch script for adjusting route and clearing NAT table. Additionally I use a dummy eoip interface that is enabled/disabled on primary link up/down so that SNMP monitoring via Nagios can throw an alert quickly to show a site is operating on failover.
Matt _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
A few weeks ago I made a suggestion to Mikrotik to use their cloud to implement a VPN mode that is similar to Meraki's. The idea is that they'd wrap the IPSEC session in UDP and use Mikrotik cloud to coordinate hole punching between two Mikrotik devices that are both behind NAT like 4G. They seemed very receptive to the idea so perhaps it'll pop up in a ROS update. Here's hoping.... Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/>
Just to follow up, Mikrotik announced Zerotier support: https://forum.mikrotik.com/viewtopic.php?f=1&t=178063 This means hole punching VPNs which makes Mikrotik 4G <----> 4G Mikrotik doable now! Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/> On Fri, 24 Jan 2020, at 08:53, Jason Hecker wrote:
A few weeks ago I made a suggestion to Mikrotik to use their cloud to implement a VPN mode that is similar to Meraki's. The idea is that they'd wrap the IPSEC session in UDP and use Mikrotik cloud to coordinate hole punching between two Mikrotik devices that are both behind NAT like 4G. They seemed very receptive to the idea so perhaps it'll pop up in a ROS update. Here's hoping....
Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/>
I just set up this system exactly using 2x RB4011 and WAPR-LTE and a CHR. The main IKE2 VPN link is over the NBN connection but if any of the NBN GRE tunnels drop out after 10 seconds the inter-subnet route goes via 4G and the CHR. It works really well. The WAPR has a handy mode where it will pass the 4G connection through to a nominated MAC address (I used a VLAN for this) so a VLAN interface on the RB4011 actually does the DHCP session to Telstra - this makes organising things much easier. This AusNOG post discusses Telstra's new IPv6 trials with 4G. Hopefully in the long they might allow services to run on those addresses which would mean you could do 4G-4G without needed to get a CHR involved. http://lists.ausnog.net/pipermail/ausnog/2020-February/043869.html Regards, Jason Hecker <https://www.upandrunningtech.com.au/> <https://www.upandrunningtech.com.au/> On Wed, 9 Oct 2019, at 17:43, Karl Auer wrote:
We need to quote to a client for two routers capable of running a VPN between two sites - easy, that's a MikroTik. But they want a fallback to 3G/4G so that the site that falls back keeps Internet access AND so that the VPN keeps running.
What's the state of MikroTik 3G/4G fallback? Last I looked it seemed very roll-your-own, and supported only a few very specific dongles...
Ideally it would work like the (vastly more expensive) Merakis and just fail everything over to the secondary link if the primary fails, where "fail" would be either a ping test or interface down.
That said, I'm OK with a solution that needs more work, as long as once done it is set-and-forget.
Any pointers?
Regards, K.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
GPG fingerprint: 887A DA07 4DCC EE76 B413 27D4 C638 4189 6CF0 D556 Old fingerprint: 8454 EE43 6215 B6DD 1B4D 9D8D 984D 7BA1 7378 A38D
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
participants (7)
-
Damien Gardner Jnr -
James Hodgkinson -
Jason Hecker -
Karl Auer -
Matt Hare -
Paul Julian -
Roger Plant