Something that may be useful is you could have the same IP allocated in multiple VRFs. This could help with getting srcnat to do its thing before the change of VRF, if required - but it looks like you're sorted now anyway :) -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Monday, 30 January 2017 10:29 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] VRF - routing and default local services Hi Just for clarity ether1 - cross connect to another ccr no vrf spfplus1&2 lacp trunk vlan management vrf management vlan internet vrf internet /ip route add distance=250 gateway=10.32.80.1 routing-mark=Management add comment="Must be here for the VRF's to work" distance=251 gateway=192.168.0.2 add comment="TEMP FOR TESTING - dgw via " distance=250 gateway=2.7.3.6 pref-src=2.7.3.6 routing-mark=internet /ip route vrf add interfaces=Management routing-mark=Management add interfaces=public routing-mark=internet /ip firewall mangle add action=mark-connection chain=prerouting in-interface=Management new-connection-mark=Management passthrough=yes add action=mark-routing chain=output connection-mark=Management new-routing-mark=Management passthrough=yes # internet not ready add action=mark-connection chain=prerouting in-interface=internet new-connection-mark=internet passthrough=yes add action=mark-routing chain=output connection-mark=internet new-routing-mark=internet passthrough=yes add chain=output action=mark-routing new-routing-mark=internet dst-address=192.231.203.132/32 add chain=output action=mark-routing new-routing-mark=internet dst-address=211.29.132.139/32 then I setup my ntp client to those 2 addresses I can see via packet sniff, packets leaving the right interface, going via the right gw, but it has the wrong src ip. src is based upon the original dgw in main 192.168.0.1. so I add this add action=src-nat chain=srcnat src-address=192.168.0.1 to-addresses=2.7.3.6 out-interface=internet and this time it works arg... when I did this before it didn't src nat .... thanks ! Alex -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Philip Loenneker Sent: Monday, 30 January 2017 9:47 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] VRF - routing and default local services Ah, I see! The second routing rule I provided specified that you would need to create a mangle rule for it, but the question is how to identify the traffic. Would the router ever initiate traffic destined for any other network? If not, then this might work: /ip firewall mangle add action=mark-connection chain=output connection-state=new new-connection-mark=connection-Management passthrough=yes Any new connections that are initiated from the router should have their connection marked. As for using SNAT or whatever to specify the source IP, I haven't tried that. It should automatically choose the default source IP for the route that it matches... that might be easiest to configure. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Monday, 30 January 2017 9:33 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] VRF - routing and default local services Hi Yep I have that and got that working. My issue is with traffic originating on the box, ntp , dns ... I tried a mangle and snat but the snat didn't work nor did the mangle strangely. So my default main table route points to basically no where, with ssh inbound, the reply packets get marked and routed via the management vrf via a mangle rule. It's the ones that are initiated from the box ... dns don't really care can work around that. But ntp ? I will be playing with it more today Alex -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Philip Loenneker Sent: Monday, 30 January 2017 9:26 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] VRF - routing and default local services Hi Alex, My experience so far has been that the Mikrotik services (ssh, winbox, snmp, dns, ntp, etc) will listen on all VRFs, but will only respond on the "main" routing table. As long as you have an active non-blackhole route in the main routing table that covers the return traffic, the return traffic will be generated, but only on the "main" routing table. You need to have a Mangle rule for all return traffic, which Damien Gardner on this list previously showed can probably be most easily done by using Connection Marking. Perhaps try this: /ip firewall mangle ## Tag all new connections incoming on the Management VRF add action=mark-connection chain=input connection-state=new new-connection-mark=connection-Management passthrough=yes routing-table=vrf-Management ## Tag all new connections initiated by the router which should be on the Management VRF - YOU WILL NEED TO ADJUST THIS TO MEET YOUR REQUIREMENTS add action=mark-connection chain=output connection-state=new new-connection-mark=connection-Management passthrough=yes out-interface=etherX ## Move all traffic with the Connection Mark of Management to the Management VRF add action=mark-routing chain=output connection-mark= connection-Management new-routing-mark= vrf-Management passthrough=yes You will also need a route to cover all management traffic - doesn't have to be a default route, eg if your management network is 10.100.100.0/24, just have a route for that. I haven't fully test this... please let us know how you go. Regards, Philip Loenneker | Network Engineer | TasmaNet -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Saturday, 28 January 2017 6:24 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] VRF - routing and default local services Hi So how do I work with the local services .. DNS and NTP ??? Alex ________________________________________ From: Public [public-bounces@talk.mikrotik.com.au] on behalf of Alex Samad - Yieldbroker [Alex.Samad@yieldbroker.com] Sent: Friday, 27 January 2017 3:10 PM To: MikroTik Australia Public List Subject: Re: [MT-AU Public] VRF - routing and default local services Also one other question it was mentioned that if I enter a rule (or two ?) into /ip route vrf Route will disappear from the default route table. /ip route> export # /ip route add distance=250 gateway=10.32.80.1 routing-mark=Management add distance=251 gateway=192.168.0.2 /ip route vrf add interfaces=Management routing-mark=Management /ip route> print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 A S 0.0.0.0/0 10.32.80.1 250 1 ADC 10.32.80.0/24 10.32.80.72 Management 0 2 A S 0.0.0.0/0 192.168.0.2 251 3 ADC 192.168.0.0/24 192.168.0.1 ether1 0 So what do I need to add to vrf to make 0 A S 0.0.0.0/0 10.32.80.1 250 Disappear when printing the default table ? Or am I misunderstanding something Alex -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Friday, 27 January 2017 3:02 PM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] VRF - routing and default local services Hi Just continuing on my journey for multiple VRF. I have segregated of an interface for management. Using routing with vrf = Management and mangle rules to mark all packets / connections with vrf=Management. Caveat is that my default routing table must have a valid route. So when I tested telnet ccr on the management port ip from a box on the same vlan - management, I could see packets coming in and then nothing leaving Add in default route via a cross connect and suddenly packets start to flow back. Note I can send default to blackhole that doesn't work. Now my question is things like logging can I set the source address / interface . will setting the source set the interface ? Will packets pick up the mark if they have that source address - or do I need to add in a mangle that say's any with that source address has the vrf=Management Alex Alex Samad | Network And System Manager | Yieldbroker * +61 2 9994 2893 | ( +61 438 838 143 | * alex.samad@yieldbroker.com<mailto:alex.samad@yieldbroker.com> This email is confidential and intended for the addressee only. If you may have received this email in error please delete it and notify the sender immediately. Recipients should not forward, disclose, distribute or copy this e-mail or any attachments in whole or part without the express permission of the sender. Views expressed in this message are those of the individual sender, except where they are specifically stated to be those of Yieldbroker. Yieldbroker accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Yieldbroker can not guarantee the integrity of this communication and shall not be liable for e-mail which may be intercepted, corrupted, lost, spoofed, delayed, incomplete, or virus infected. _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au -- Message protected by MailGuard: e-mail anti-virus, anti-spam and content filtering.http://www.mailguard.com.au/mg Click here to report this message as spam: https://console.mailguard.com.au/ras/1Q8Yx1dlys/BtttE6CLbVF1p34JTDGua/0.2 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au -- Message protected by MailGuard: e-mail anti-virus, anti-spam and content filtering.http://www.mailguard.com.au/mg Click here to report this message as spam: https://console.mailguard.com.au/ras/1Qa7vjGt0Q/1SEROFaqfdZNcFCRmw8ycv/0.2 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au