Some may recall I now have a Wireguard VPN between a Starlink-connected site and another site. The other site has a static IP address, the Starlink site does not. Inevitably, someone now wants to connect from outside to a server in TCP/85 in the Starlink site. No problem, thinks I. They can connect to the static IP and I will ship their connection over the VPN to the other network. Just dstnat plus srcnat. In the following, xxx is the static IP address, and 192.168.103.184 is the address they want to reach in the Starlink-connected network. /ip filter nat add chain=dstnat action=dst-nat \ dst-address=xxx dst-port=85 protocol=tcp \ to-addresses=192.168.103.184 to-ports=85 \ in-interface=e1-uplink log=yes log-prefix="A" add chain=srcnat action=masquerade \ ???? out-interface=wg0 log=yes log-prefix="B" I can't figure out what I should put in the srcnat rule that will limit it so it only src-nats packets coming from the public internet. The obvious one, filtering on "in-interface=e1-uplink", isn't possible with srcnat. What am I missing? Or do I have to add "accept" rules for all the networks I don't want NATted by wg0? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
Hi Karl, Why not a "Wireguard" address list - with the source addresses of the far-end networks, and then on the src-nat rule, use the 'Source Address List' with the NOT option? - ie. Sourcenat everything NOT on this list. -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Karl Auer via Public Sent: Tuesday, November 7, 2023 11:56 AM To: MikroTik Public <public@talk.mikrotik.com.au> Cc: Karl Auer <kauer@nullarbor.com.au> Subject: [MT-AU Public] Puzzled why forwarding no work Some may recall I now have a Wireguard VPN between a Starlink-connected site and another site. The other site has a static IP address, the Starlink site does not. Inevitably, someone now wants to connect from outside to a server in TCP/85 in the Starlink site. No problem, thinks I. They can connect to the static IP and I will ship their connection over the VPN to the other network. Just dstnat plus srcnat. In the following, xxx is the static IP address, and 192.168.103.184 is the address they want to reach in the Starlink-connected network. /ip filter nat add chain=dstnat action=dst-nat \ dst-address=xxx dst-port=85 protocol=tcp \ to-addresses=192.168.103.184 to-ports=85 \ in-interface=e1-uplink log=yes log-prefix="A" add chain=srcnat action=masquerade \ ???? out-interface=wg0 log=yes log-prefix="B" I can't figure out what I should put in the srcnat rule that will limit it so it only src-nats packets coming from the public internet. The obvious one, filtering on "in-interface=e1-uplink", isn't possible with srcnat. What am I missing? Or do I have to add "accept" rules for all the networks I don't want NATted by wg0? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi, Don't masquerade at the remote end Just make sure at the starkink end the traffic that come down the WG tunnel, also goes back out the WG tunnel /routing table add fib name=wireguard1_Traffic # If you have port forwards you need to ensure they stick to the correct interfaces for the return traffic # port forwards from wireguard1, goes back out from wireguard1 /ip firewall mangle add action=mark-connection chain=forward in-interface= wireguard1 connection-state=new new-connection-mark=wireguard1_pfw passthrough=no disabled=no comment="pfw wireguard1, out wireguard1" /ip firewall mangle add action=mark-routing chain=prerouting in-interface=bridge-local connection-mark=wireguard1_pfw new-routing-mark= wireguard1_Traffic passthrough=no disabled=no Andy -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Karl Auer via Public Sent: Tuesday, November 7, 2023 8:56 AM To: MikroTik Public <public@talk.mikrotik.com.au> Cc: Karl Auer <kauer@nullarbor.com.au> Subject: [MT-AU Public] Puzzled why forwarding no work Some may recall I now have a Wireguard VPN between a Starlink-connected site and another site. The other site has a static IP address, the Starlink site does not. Inevitably, someone now wants to connect from outside to a server in TCP/85 in the Starlink site. No problem, thinks I. They can connect to the static IP and I will ship their connection over the VPN to the other network. Just dstnat plus srcnat. In the following, xxx is the static IP address, and 192.168.103.184 is the address they want to reach in the Starlink-connected network. /ip filter nat add chain=dstnat action=dst-nat \ dst-address=xxx dst-port=85 protocol=tcp \ to-addresses=192.168.103.184 to-ports=85 \ in-interface=e1-uplink log=yes log-prefix="A" add chain=srcnat action=masquerade \ ???? out-interface=wg0 log=yes log-prefix="B" I can't figure out what I should put in the srcnat rule that will limit it so it only src-nats packets coming from the public internet. The obvious one, filtering on "in-interface=e1-uplink", isn't possible with srcnat. What am I missing? Or do I have to add "accept" rules for all the networks I don't want NATted by wg0? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
On Tue, 2023-11-07 at 01:19 +0000, Andrew Oakeley wrote:
Don't masquerade at the remote end Just make sure at the starkink end the traffic that come down the WG tunnel, also goes back out the WG tunnel
Sounds good.
/routing table add fib name=wireguard1_Traffic
# If you have port forwards you need to ensure they stick to the correct interfaces for the return traffic # port forwards from wireguard1, goes back out from wireguard1 /ip firewall mangle add action=mark-connection chain=forward in- interface= wireguard1 connection-state=new new-connection- mark=wireguard1_pfw passthrough=no disabled=no comment="pfw wireguard1, out wireguard1" /ip firewall mangle add action=mark-routing chain=prerouting in- interface=bridge-local connection-mark=wireguard1_pfw new-routing- mark= wireguard1_Traffic passthrough=no disabled=no
This is new territory for me! What ties the inbound interface of the arriving packet to the outbound interface of the response packet? The first mangle rule marks the new connections arriving on the wireguard interface to be forwarded with a mark called "wireguard_pfw". I am guessing this causes all future packets that are part of that connection to be similarly marked? The second mangle rule adds a policy routing mark called "wireguard1_Traffic" to any packets marked with the "wireguard_pfw" mark IF they are in the pre-routing chain and they are arriving from the LAN (i.e. on bridge-local). The routing mark ties the packets to the FIB called "wireguard1_Traffic". What I still don't quite get is how these rules send a response packet on its way via wireguard1. I see the new FIB, I just don't get how it does what it presumably does :-) Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
On Tue, 2023-11-07 at 01:19 +0000, Andrew Oakeley wrote:
Don't masquerade at the remote end
Ah, here's the second reason why I thought masquerading was a good idea: I only allow the networks at each end of the wireguard link to go over that link. However, I could change the Starlink end to allow 0.0.0.0/0, I suppose. Probably best to be on site when I try that, rather than ssh-ed in over the very wireguard link I would be changing :-) Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
Could you do Source address list ! <list with internal subnets in> That should catch external IP’s but not mask the internal ones. If you have an easy internal subnet to define you could just use source address ! <internal subnet and netmask> In winbox click the small “checkbox” next to the field for the ! To negate the value. Regards Alexander Alexander Neilson Neilson Productions Limited 021 329 681 alexander@neilson.net.nz
On 7/11/2023, at 14:14, Karl Auer via Public <public@talk.mikrotik.com.au> wrote:
Some may recall I now have a Wireguard VPN between a Starlink-connected site and another site. The other site has a static IP address, the Starlink site does not.
Inevitably, someone now wants to connect from outside to a server in TCP/85 in the Starlink site. No problem, thinks I. They can connect to the static IP and I will ship their connection over the VPN to the other network. Just dstnat plus srcnat.
In the following, xxx is the static IP address, and 192.168.103.184 is the address they want to reach in the Starlink-connected network.
/ip filter nat
add chain=dstnat action=dst-nat \ dst-address=xxx dst-port=85 protocol=tcp \ to-addresses=192.168.103.184 to-ports=85 \ in-interface=e1-uplink log=yes log-prefix="A"
add chain=srcnat action=masquerade \ ???? out-interface=wg0 log=yes log-prefix="B"
I can't figure out what I should put in the srcnat rule that will limit it so it only src-nats packets coming from the public internet. The obvious one, filtering on "in-interface=e1-uplink", isn't possible with srcnat.
What am I missing? Or do I have to add "accept" rules for all the networks I don't want NATted by wg0?
Regards, K.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi, The dstnat rule is one way to enforce internet packets. If the mikrotik is the main gateway, you can choose source interface. If not you can have match source address to not (address list of lan addresses). To: MikroTik Public <public@talk.mikrotik.com.au> Date sent: Tue, 07 Nov 2023 11:55:42 +1100 Organization: Nullarbor Consulting pty Ltd Subject: [MT-AU Public] Puzzled why forwarding no work From: Karl Auer via Public <public@talk.mikrotik.com.au> Send reply to: MikroTik Australia Public List <public@talk.mikrotik.com.au> Copies to: Karl Auer <kauer@nullarbor.com.au> [ Double-click this line for list subscription options ] Some may recall I now have a Wireguard VPN between a Starlink-connected site and another site. The other site has a static IP address, the Starlink site does not. Inevitably, someone now wants to connect from outside to a server in TCP/85 in the Starlink site. No problem, thinks I. They can connect to the static IP and I will ship their connection over the VPN to the other network. Just dstnat plus srcnat. In the following, xxx is the static IP address, and 192.168.103.184 is the address they want to reach in the Starlink-connected network. /ip filter nat add chain=dstnat action=dst-nat \ dst-address=xxx dst-port=85 protocol=tcp \ to-addresses=192.168.103.184 to-ports=85 \ in-interface=e1-uplink log=yes log-prefix="A" add chain=srcnat action=masquerade \ ???? out-interface=wg0 log=yes log-prefix="B" I can't figure out what I should put in the srcnat rule that will limit it so it only src-nats packets coming from the public internet. The obvious one, filtering on "in-interface=e1-uplink", isn't possible with srcnat. What am I missing? Or do I have to add "accept" rules for all the networks I don't want NATted by wg0? Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au---------------------------- Roger Plant
On Tue, 2023-11-07 at 19:19 +1100, Roger Plant via Public wrote:
The dstnat rule is one way to enforce internet packets. If the mikrotik is the main gateway, you can choose source interface. If not you can have match source address to not (address list of lan addresses).
Thanks Roger. I've puzzled over the above for a while, and have to confess I don't understand your second and third sentences unless they both relate to dstnat. The dstnat rule is there to make sure that a packet coming in for a particular port gets sent to the right place - basic port-forwarding. And I have indeed chosen the source interface in the dstnat rule ("in- interface=e1-uplink"). As far as srcnat is concerned I have had a bit of a LOL moment; it's really not part of the solution. Progress has been made though, in my understanding if nothing else. I have allowed my own outside address as one of the remote end's allowed- addresses (0.0.0.0/0 will be needed at both ends eventually I guess). Sniffing shows that my dstnatted packets are now arriving at the remote end of the wireguard link, and also shows that replies are trying to get home over the remote end's uplink, instead of back over the wireguard link. So Andrew Oakley's suggestion re connection marking seems to be the last piece needed. If only I understood it :-) Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
Hi,
So Andrew Oakley's suggestion re connection marking seems to be the last piece needed. If only I understood it :-) This is the same principal as if you have a router doing Dual WAN. You need to ensure packets coming in one interface go back out the same interface. That's all it is.
A) Create a Routing table /routing table add fib name=wireguard1_Traffic B) Create a mangle rule to mark connections that come in and are forwarded in interface wireguard1. We mark these with the connection mark wireguard1_pfw /ip firewall mangle add action=mark-connection chain=forward in-interface= wireguard1 connection-state=new new-connection-mark=wireguard1_pfw passthrough=no disabled=no comment="pfw wireguard1, out wireguard1" C) Now when a packet come in through the interface bridge-local or whatever our bridge is, that already has the connection mark wireguard1_pfw, we add the routing mark wireguard1_Traffic to that packet. /ip firewall mangle add action=mark-routing chain=prerouting in-interface=bridge-local connection-mark=wireguard1_pfw new-routing-mark=wireguard1_Traffic passthrough=no disabled=no D) now I missed this out from my original post, you also need to tell the routing table what to do with such packets. Where X.X.X.X is the internal IP on the wireguard tunnel at the non Starlink end. So this is where the outbound packet from the starlink end goes so it goes back out the same way it came in /ip route add dst-address=0.0.0.0/0 gateway=X.X.X.X distance=1 routing-table= wireguard1_Traffic disabled=no comment=" wireguard1_Traffic" So for completeness A-D above is normally combined with 2 rules on the input/output chain also so packets on the input chain (i.e. destined for the router at the starlink end, not being forwarded through the router) also go back up the wireguard tunnel. # what comes from wireguard1, gets out from wireguard1 /ip firewall mangle add action=mark-connection chain=input in-interface=wireguard1 new-connection-mark=wireguard1_conn passthrough=yes disabled=no comment="in wireguard1,out wireguard1" /ip firewall mangle add action=mark-routing chain=output connection-mark=wireguard1_conn new-routing-mark=wireguard1 passthrough=no disabled=no So a complete set that deals with forwarding and input/output traffic at your starlink end would look like this. This might make more sense outlined in this fashion. - Where 192.168.31.1 is the remote (non-starlink end out the wireguard interface) - if you take a dummy router with # Define routing table and routes /routing table add fib name=wireguard1_Traffic /ip route add dst-address=0.0.0.0/0 gateway=192.168.31.1 distance=1 routing-table=wireguard1_Traffic disabled=no comment="Wireguard1" # Input/output rules # what comes from wireguard1, goes out from wireguard1 /ip firewall mangle add action=mark-connection chain=input in-interface=wireguard1 new-connection-mark=wireguard1_conn passthrough=yes disabled=no comment="in wireguard1,out wireguard1" /ip firewall mangle add action=mark-routing chain=output connection-mark=wireguard1_conn new-routing-mark=wireguard1 passthrough=no disabled=no # Forwarding rules # If you have port forwards (NAT) or simply forwarding through the router, you need to ensure packets stick to the correct interfaces for the return traffic # port forwards from wireguard1, goes back out from wireguard1 /ip firewall mangle add action=mark-connection chain=forward in-interface=wireguard1 connection-state=new new-connection-mark=wireguard1_pfw passthrough=no disabled=no comment="pfw wireguard1, out wireguard1" /ip firewall mangle add action=mark-routing chain=prerouting in-interface=bridge-local connection-mark=wireguard1_pfw new-routing-mark=wireguard1 passthrough=no disabled=no Final words. Fix the above with a) correct IP for the remote gateway on the wireguard interface b) name of the wireguard interface c) name of the lan interface Then put your remote starlink end router in safe mode and paste it in. Andrew PS - I am available for consulting. -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Karl Auer via Public Sent: Tuesday, November 7, 2023 10:07 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Karl Auer <kauer@nullarbor.com.au> Subject: Re: [MT-AU Public] Puzzled why forwarding no work On Tue, 2023-11-07 at 19:19 +1100, Roger Plant via Public wrote:
The dstnat rule is one way to enforce internet packets. If the mikrotik is the main gateway, you can choose source interface. If not you can have match source address to not (address list of lan addresses).
Thanks Roger. I've puzzled over the above for a while, and have to confess I don't understand your second and third sentences unless they both relate to dstnat. The dstnat rule is there to make sure that a packet coming in for a particular port gets sent to the right place - basic port-forwarding. And I have indeed chosen the source interface in the dstnat rule ("in- interface=e1-uplink"). As far as srcnat is concerned I have had a bit of a LOL moment; it's really not part of the solution. Progress has been made though, in my understanding if nothing else. I have allowed my own outside address as one of the remote end's allowed- addresses (0.0.0.0/0 will be needed at both ends eventually I guess). Sniffing shows that my dstnatted packets are now arriving at the remote end of the wireguard link, and also shows that replies are trying to get home over the remote end's uplink, instead of back over the wireguard link. So Andrew Oakley's suggestion re connection marking seems to be the last piece needed. If only I understood it :-) Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160 _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
On Tue, 2023-11-07 at 15:25 +0000, Andrew Oakeley wrote:
You need to ensure packets coming in one interface go back out the same interface. That's all it is.
Thanks heaps for that very detailed response. Step D was the main missing piece, it all made a lot more sense with that in there.
A-D above is normally combined with 2 rules on the input/output chain also so packets on the input chain (i.e. destined for the router at the starlink end, not being forwarded through the router) also go back up the wireguard tunnel.
The normal case for these two routers is that anything for the other router goes over the wireguard link via a very ordinary route entry. Everything else finds its own way home by being directly connected or default routed out the uplinks. So I *think* these two are not needed in my specific case. I like how little needs to be done on the non-Starlink end. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@nullarbor.com.au) work +61 2 64957435 http://www.nullarbor.com.au mobile +61 428 957160
So I *think* these two are not needed in my specific case.
2 use cases off the top of my head: Remote management VPN server Prob not needed for you Others may find useful Andy On 7 Nov 2023, at 23:58, Karl Auer via Public <public@talk.mikrotik.com.au> wrote: So I *think* these two are not needed in my specific case.
participants (5)
-
Alexander Neilson -
Andrew Oakeley -
Karl Auer -
Michael Junek -
Roger Plant