Hi all, I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP. I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop. This all seems logical, except that all the guides say to do it this way. I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors. Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious... Regards, Philip
Hi Philip, All your intervening routers must know how to reach the other devices. This would mean you would need to distribute the BGP routes into OSPF or build BGP between all your devices. Even establishing MPLS does not necessarily do this as each of the intervening hops must know how to resolve the MPLS packets. You could look at establishing EoIP tunnels between the edge devices, and building iBGP neighbours over the EoIP tunnels. Thanks, Andrew On Fri, 23 Oct 2020 at 10:04, Philip Loenneker < Philip.Loenneker@tasmanet.com.au> wrote:
Hi all,
I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP.
I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop.
This all seems logical, except that all the guides say to do it this way.
I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors.
Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious...
Regards, Philip _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Thanks Andrew. That's what I suspected but was hoping I was missing some magic trick to make it use the recursive next-hop address instead of the destination address for determining the next hop through the switches. All the documentation says you shouldn't put your full route table in the IGP (OSPF in this case) so how is this meant to be achievable? -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Andrew Yager Sent: Friday, 23 October 2020 10:17 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] iBGP over OSPF Hi Philip, All your intervening routers must know how to reach the other devices. This would mean you would need to distribute the BGP routes into OSPF or build BGP between all your devices. Even establishing MPLS does not necessarily do this as each of the intervening hops must know how to resolve the MPLS packets. You could look at establishing EoIP tunnels between the edge devices, and building iBGP neighbours over the EoIP tunnels. Thanks, Andrew On Fri, 23 Oct 2020 at 10:04, Philip Loenneker < Philip.Loenneker@tasmanet.com.au> wrote:
Hi all,
I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP.
I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop.
This all seems logical, except that all the guides say to do it this way.
I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors.
Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious...
Regards, Philip _______________________________________________ Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk. mikrotik.com.au%2Fmailman%2Flistinfo%2Fpublic_talk.mikrotik.com.au& ;data=04%7C01%7Cphilip.loenneker%40tasmanet.com.au%7C458ff6a2ece342c52 0b308d876e0d159%7Cb53dc580ab7847208b30536f36d398ac%7C0%7C0%7C637390055 249463998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XIlPvRXipM7%2FLC6WDX6P tNRoR7mdEthM%2FF24EEPEQwQ%3D&reserved=0
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk.mikrot...
Further clarification: I've set next-hop-self on the iBGP peers, and intend to have a full mesh between the routers. However I tested it before the full mesh was complete. Here is a rough idea of what I've got: MT Router -- (ospf) -- 2x L3 switches -- (ospf) -- MT router MT routers advertise a loopback into OSPF and I am able to ping between MT routers. iBGP can establish and learn an abundance of routes - next-hop-self and update-source configured Route tables update and show the learned routes as valid with recursive next-hop of the loopback of the far-end MT router Traceroute to a subnet behind the far-end MT router stops at the first switch with "destination network unreachable" or equivalent because the switch itself has no route for the final destination Are there any special settings required for the switches to process the recursive-next-hop address instead of the packets actual destination IP? -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Philip Loenneker Sent: Friday, 23 October 2020 10:01 AM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] iBGP over OSPF Hi all, I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP. I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop. This all seems logical, except that all the guides say to do it this way. I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors. Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious... Regards, Philip _______________________________________________ Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk.mikrot...
Hey Philip, That's not really how recursive next hop works - That just means the router learning the route will still consider it valid if the next hop isn't directly attached. The intermediate L3 devices still need to have a route for the destination prefix. You can either: - Run iBGP on the intermediate switches - Connect the MT's together at layer 2 (either a normal vlan, EoIP, or something like GRE) - Run MPLS (on the intermediate switches too) - This will mean they look at the labels for forwarding (which will tell them to pass it on to the MT) rather than the L3 header. Hope that helps a bit. Cheers, Tim On Fri, 23 Oct 2020 at 10:21, Philip Loenneker < Philip.Loenneker@tasmanet.com.au> wrote:
Further clarification: I've set next-hop-self on the iBGP peers, and intend to have a full mesh between the routers. However I tested it before the full mesh was complete.
Here is a rough idea of what I've got:
MT Router -- (ospf) -- 2x L3 switches -- (ospf) -- MT router
MT routers advertise a loopback into OSPF and I am able to ping between MT routers. iBGP can establish and learn an abundance of routes - next-hop-self and update-source configured Route tables update and show the learned routes as valid with recursive next-hop of the loopback of the far-end MT router Traceroute to a subnet behind the far-end MT router stops at the first switch with "destination network unreachable" or equivalent because the switch itself has no route for the final destination
Are there any special settings required for the switches to process the recursive-next-hop address instead of the packets actual destination IP?
-----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Philip Loenneker Sent: Friday, 23 October 2020 10:01 AM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] iBGP over OSPF
Hi all,
I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP.
I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop.
This all seems logical, except that all the guides say to do it this way.
I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors.
Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious...
Regards, Philip _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk.mikrot...
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Thanks Tim. I think I understand why I misunderstood the examples now. They probably have the routers all L2 adjacent so they OSPF peer directly to each other, not through intermediate L3 devices. Then they don't need the IGP to have the full route table because they will look at the devices route table to make the next-hop decision. This makes the project far more complex than I was hoping for. I was hoping to avoid needing to stretch layer2 links between the devices. I'll have to re-assess my options. Thanks everyone who replied so quickly, I really appreciate it. -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Tim Sandy Sent: Friday, 23 October 2020 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] iBGP over OSPF Hey Philip, That's not really how recursive next hop works - That just means the router learning the route will still consider it valid if the next hop isn't directly attached. The intermediate L3 devices still need to have a route for the destination prefix. You can either: - Run iBGP on the intermediate switches - Connect the MT's together at layer 2 (either a normal vlan, EoIP, or something like GRE) - Run MPLS (on the intermediate switches too) - This will mean they look at the labels for forwarding (which will tell them to pass it on to the MT) rather than the L3 header. Hope that helps a bit. Cheers, Tim On Fri, 23 Oct 2020 at 10:21, Philip Loenneker < Philip.Loenneker@tasmanet.com.au> wrote:
Further clarification: I've set next-hop-self on the iBGP peers, and intend to have a full mesh between the routers. However I tested it before the full mesh was complete.
Here is a rough idea of what I've got:
MT Router -- (ospf) -- 2x L3 switches -- (ospf) -- MT router
MT routers advertise a loopback into OSPF and I am able to ping between MT routers. iBGP can establish and learn an abundance of routes - next-hop-self and update-source configured Route tables update and show the learned routes as valid with recursive next-hop of the loopback of the far-end MT router Traceroute to a subnet behind the far-end MT router stops at the first switch with "destination network unreachable" or equivalent because the switch itself has no route for the final destination
Are there any special settings required for the switches to process the recursive-next-hop address instead of the packets actual destination IP?
-----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Philip Loenneker Sent: Friday, 23 October 2020 10:01 AM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] iBGP over OSPF
Hi all,
I have multiple MikroTik routers connected to non-MikroTik switches with OSPFv2 and v3 enabled on routers and switches (isolated VRF) to distribute loopbacks and interconnect subnets. No other subnets are in the IGP.
I can establish iBGP peers between loopbacks and exchange all routes (showing recursive next hop), including default route, but I can't connect to subnets behind other routers. In a traceroute the switches report that there is no valid next hop.
This all seems logical, except that all the guides say to do it this way.
I've seen some guides say to run MPLS over the top to get reachability, but I don't want to have that, especially since some routers are CHR and have issues with MPLS on some hypervisors.
Does anyone have any advice on this? Do I need to have L2 as adjacency between all routers eg with a tunnel over the OSPF loopbacks, or MPLS, or should this work as the guides all suggest? I'm sure I'm missing something obvious...
Regards, Philip _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk. mikrotik.com.au%2Fmailman%2Flistinfo%2Fpublic_talk.mikrotik.com.au& ;data=04%7C01%7Cphilip.loenneker%40tasmanet.com.au%7Cbbacc0db066745ab9 a8708d876e2b6e7%7Cb53dc580ab7847208b30536f36d398ac%7C0%7C0%7C637390063 404095970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=i1moYddcuRs93ZQLRZj0rm Wr7aSvYhRuZS1NR1yzu7c%3D&reserved=0
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk. mikrotik.com.au%2Fmailman%2Flistinfo%2Fpublic_talk.mikrotik.com.au& ;data=04%7C01%7Cphilip.loenneker%40tasmanet.com.au%7Cbbacc0db066745ab9 a8708d876e2b6e7%7Cb53dc580ab7847208b30536f36d398ac%7C0%7C0%7C637390063 404095970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIi LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=i1moYddcuRs93ZQLRZj0rm Wr7aSvYhRuZS1NR1yzu7c%3D&reserved=0
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk.mikrot...
participants (3)
-
Andrew Yager -
Philip Loenneker -
Tim Sandy