Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting
Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day. I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc. The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244) This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP As i understand it, I've followed this guide. https://forum.mikrotik.com/viewtopic.php?t=143620 For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3. My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2. At the moment the switch is administered over serial. Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon.
Hi, Initially on the CHR I would add a vlan interface to ether 2 with vlan ID 20 (perhaps call it vlan20) Attach the 10.10.20.200 to vlan20 On the switch, I would remove the line untagged 25 from the vlan 20 configuration. Does port 25 need an untagged entry for it to work (Sorry, I don't know) Then I assume you have access to the VM switch configuration stuff. You will probably need to play with this to allow the vlans you want through it. Also if the VM is running on a Windows host, you might need to look at the NIC driver configuration. (They usually drop VLan tags) Good Luck Regards Roger From: deadlift <666deadlift@gmail.com> Date sent: Tue, 12 May 2020 08:34:10 +1000 To: public@talk.mikrotik.com.au Subject: [MT-AU Public] Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting Send reply to: MikroTik Australia Public List <public@talk.mikrotik.com.au> Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day. I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc. The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244) This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP As i understand it, I've followed this guide. https://forum.mikrotik.com/viewtopic.php?t=143620 For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3. My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2. At the moment the switch is administered over serial. Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon. ---------------------------- Roger Plant
In order to use VLANs on ESX - you need to create a 'virtual network' with a VLAN ID of (ALL). That lets you use VLAN interfaces within CHR. If you don't want to do that then use a new 'etherX' on the CHR and then create an esx virtual network with the vlan ID of the VLAN - This does the VLAN manipulation for you. Use trunk ports on cisco as normal, and on the HP it should be: vlan <x> tagged 25 Or something like that.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of deadlift Sent: Tuesday, 12 May 2020 8:34 AM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting
Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day.
I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc.
The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244)
This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP
As i understand it, I've followed this guide. https://forum.mikrotik.com/viewtopic.php?t=143620
For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3.
My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2.
At the moment the switch is administered over serial.
Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon.
Hi Simon, Unfortunately your router configuration didn't come through as an attachment. Here are some things that may be useful, but note that my experience with Vmware is mainly as a consumer of VMs while another team manage the hosts. VMware host: You should have your host management (managing Vmware, vMotion, etc) on one physical interface, and the Guest VM networks (what you want the VMs to access) on a separate one. VMware Port Groups: Make sure the correct uplink port(s) are selected, with the NIC(s) or LAGs where it can pick up the VLANs. If you want one or more VLANs to be tagged on the VM, you must choose the VLAN type of Trunk on the Port Group and choose the VLANs you want to have available on that Port Group. You can specify all VLANs (1-4095 I think), but I would discourage that from a security point of view. If you want a single VLAN untagged, choose VLAN type Access. Switch: You should always have the VLANs tagged from the switch to the VMware host on the interface used for Guest VM networks. Other: If you happen to use VMANs, or QinQ, or whatever other various names there are for having VLANs inside VLANs, you must have them presenting to the VMware hosts with VLAN ethertype 0x8100. If you try to use 0x88a8, VMware will not pass the traffic through. If you do this, you need use-service-tag=no on the VLAN in the CHR. You can not hot-add NICs to the CHR (yet). Add a couple of spare NICs to the VM so that you can spin up extra connectivity without having to shutdown the VM. They can be disconnected in VMware until you need them. In our case, we tend to have ether1 with a common Management port-group that has a particular VLAN untagged. This way our VM team can build up the VM and the Network team can mac-telnet into the VM to configure it. I don't generally need to worry about console access or anything like that. Then we have one or more interfaces with VLAN Trunk configured to deliver the VLANs required for the CHR to do its job. Regards, Philip Loenneker | Senior Network Engineer | TasmaNet -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of Tim Warnock Sent: Tuesday, 12 May 2020 9:11 AM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting In order to use VLANs on ESX - you need to create a 'virtual network' with a VLAN ID of (ALL). That lets you use VLAN interfaces within CHR. If you don't want to do that then use a new 'etherX' on the CHR and then create an esx virtual network with the vlan ID of the VLAN - This does the VLAN manipulation for you. Use trunk ports on cisco as normal, and on the HP it should be: vlan <x> tagged 25 Or something like that.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of deadlift Sent: Tuesday, 12 May 2020 8:34 AM To: public@talk.mikrotik.com.au Subject: [MT-AU Public] Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting
Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day.
I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc.
The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244)
This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP
As i understand it, I've followed this guide. https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fforu m.mikrotik.com%2Fviewtopic.php%3Ft%3D143620&data=02%7C01%7Cphilip. loenneker%40tasmanet.com.au%7C820bd1e4b44144c6123908d7f600d0e5%7Cb53dc 580ab7847208b30536f36d398ac%7C0%7C0%7C637248355678340684&sdata=2Zy 2hucIQYdMnNqJ%2BIcSxiHw8Kid0gbfVABf5D1Ajrs%3D&reserved=0
For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3.
My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2.
At the moment the switch is administered over serial.
Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon.
Public mailing list Public@talk.mikrotik.com.au https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftalk.mikrot...
Hi all, Many, many thanks for this. I did try and put the ether2 NIC into promiscuous mode prior to the email, however adding the ALL Vlan to the NIC seems to have done the trick. I can ping either side of the management vlan from either device but no other vlans, yet. I haven't got any other links up yet as the HP is still in config mode at the moment. I was lamenting having to add all the routing to the host networking stack but so far this hasn't been the case. Since changing ONE setting on a NIC was all it took. Anyway I have re-attached cfgs as .txts, for anyone who is bored. I think they need a bit of scrutinizing, I'd say it's "working" more from good luck then good management. I'll be trying to implement Philip's advice once i have all vlans pinging as designed. After that i will get a document out for all eyes under a different thread with a view for contributions, then perhaps Mike can find a home for it somewhere. Simon. On Tue, May 12, 2020 at 8:34 AM deadlift <666deadlift@gmail.com> wrote:
Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day.
I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc.
The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244)
This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP
As i understand it, I've followed this guide. https://forum.mikrotik.com/viewtopic.php?t=143620
For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3.
My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2.
At the moment the switch is administered over serial.
Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon.
Hi Simon, I'm glad you’ve had some success. I have some additional thoughts now that I have seen your router configuration: VMware is very particular (by default) about the MAC addresses it sees on virtual NICs. Once you start bridging interfaces as you have done, you may have the router originate or forward traffic with a different MAC address than is assigned to the interface by VMware. To allow the VM to do this, you need to enable Promiscuous Mode on the port group, as well as Forged Transmits. This is also required if you want to set up VRRP or similar technologies on the VM, as the MAC address is different than the one allocated. If you don't need to use a Bridge, then I recommend that you avoid it, and have the VLANs directly under the interface they are on. This will avoid the above issue and negate the need to adjust those other settings, which I believe can have a performance impact on VMware. As a side note, I would also discourage you from having critical management subnets, especially OOB, routed by a virtual machine. If you lose connectivity to the VM for any reason at all, you will likely need access to those networks and the ability to restore services. In particular, if your VMware host management traffic is routed by a VM running on that host, you are going to cause yourself a lot of pain. You won't be able to start/reboot the VM without management, but can't get management until the VM is running. Regards, Philip Loenneker | Senior Network Engineer | TasmaNet -----Original Message----- From: Public <public-bounces@talk.mikrotik.com.au> On Behalf Of deadlift Sent: Tuesday, 12 May 2020 11:21 AM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] Assistance with CHR on ESX6, Vlan-ing and an HP2910 troubleshooting Hi all, Many, many thanks for this. I did try and put the ether2 NIC into promiscuous mode prior to the email, however adding the ALL Vlan to the NIC seems to have done the trick. I can ping either side of the management vlan from either device but no other vlans, yet. I haven't got any other links up yet as the HP is still in config mode at the moment. I was lamenting having to add all the routing to the host networking stack but so far this hasn't been the case. Since changing ONE setting on a NIC was all it took. Anyway I have re-attached cfgs as .txts, for anyone who is bored. I think they need a bit of scrutinizing, I'd say it's "working" more from good luck then good management. I'll be trying to implement Philip's advice once i have all vlans pinging as designed. After that i will get a document out for all eyes under a different thread with a view for contributions, then perhaps Mike can find a home for it somewhere. Simon. On Tue, May 12, 2020 at 8:34 AM deadlift <666deadlift@gmail.com> wrote:
Hi everyone, Either there is a fundamental lack of understanding on my part, (highly likely) on MT Vlan-ing/routing, or I'm just having a bad day.
I have spun up a CHR box with two VM Nics, one on (VMNetwork) ether1 for management on my local subnet 192.168.1.0/24 which has a dhcp client, use this for winbox mgmt, etc.
The other is direct ethernet connection to an HP Procurve 2910al, port 25 for now. VM Nic ether2 (LAN NIC for want of a better term) with Static IP on CHR 10.10.20.200/24 Hoping this can be the management Vlan network. (10.10.20.0/244)
This is where i start getting lost in nomenclature and it really starting to get the better of me. This "link" would be: Trunk in Cisco Trunk in MT (?) not sure Tagged in HP
As i understand it, I've followed this guide. https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fforu m.mikrotik.com%2Fviewtopic.php%3Ft%3D143620&data=02%7C01%7Cphilip. loenneker%40tasmanet.com.au%7Cf451d3e2dd4942acc09508d7f6130efc%7Cb53dc 580ab7847208b30536f36d398ac%7C0%7C0%7C637248434388293709&sdata=jOg bKakrd3vfDjVachrjsSobw%2FAa4cNDhXoi%2FfJmJwU%3D&reserved=0
For now, I do understand the 2910 will do layer 3 routing, i have chosen to keep with no ip routing on the switch. The reason for now is that most IP services are on the router. Moving routing may be beneficial but the current goal would like to get my devices correctly administered on layer 2 and 3.
My only signs of layer two life is a ping from the HP to the CHR ether2 10.10.20.200 is responsive, 10.10.20.1 is not. Also I cannot ping the Vlan IP on the HP (10.10.20.254/24) from the CHR either via the management vlan interface or ether2.
At the moment the switch is administered over serial.
Attached is the router and switch config for you enjoyment, please let me know where I've gone awry in your eyes and I will try and knock this over, then write up something for posterity. thanks Simon.
participants (4)
-
deadlift
-
Philip Loenneker
-
Roger Plant
-
Tim Warnock