Thanks Mike, will send you the supout, not sure if it'll be any use as it'll be after the VM has rebooted. Interestingly, if I drop the MTU on the other end to 1450 (rather than 1452), the issue goes away. (router has been up 25 minutes now) The really weird thing is that it seems specifically related to 103.235.52.0/24 (Would be interesting to work out if it's a specific IP in that range, but I don't have that much free time ;) ) - if I advertise another prefix out there, I can send as many large ping packets, or download gigs and gigs via HTTP, no problems at all The fragmentation is working correctly, as packets over 1452 without DF set work fine, and packets over 1452 with DF set come back with frag-needed. I'd have thought that the debian box couldn't actually send a 'large' packet anyway, as it's limited to 1500 on its ethernet interface. Cheers, DG On 27 March 2015 at 16:28, Mike Everest <mike@duxtel.com> wrote:
Hi Damien,
This is sounding more and more like a software bug to me...
If you'd like us to get involved, please send us a supout file of the router as close as possible to the crash event, then we can open a case with MikroTik for you.
The only other thing I can think of I if there is something odd going on between the routing platform and virtualisation network - like when the router loads it's routing table, something internal on the VM management network changes, thus causing some internals to play up. Can't say I've totally thought that though bubble through, though, just trying a 'creative' angle to this problem ;)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Damien Gardner Jnr Sent: Friday, 27 March 2015 2:09 PM To: MikroTik Australia Public List Subject: Re: [MT-AU Public] RouterOS rebooting over and over? Possibly BGP + ipipv6 tunnels?
Thanks, that didn't make a difference either.
It occurred to me that the subnet I'm having trouble with is the only one I was advertising which is in decent use - the others only have one or two IP's in use each at present.. So I started thinking maybe it's a bandwidth or packet flow issue.
Interestingly, I pushed the remote end MTU from 1452 to 1400.. And the router has now been online for 15 minutes without dying..?! Now the question is... Why is this end router dying when the remote end sends a large packet (or a packet goes missing?) IS the RouterOS ip4ipv6 tunnel implementation slightly buggy with lost packets? Maybe a memory leak overflow?
On 27 March 2015 at 13:48, Tim Warnock <timoid@timoid.org> wrote:
Ok try this:
/routing bgp network set disabled=yes 0 /routing bgp instance default set redistribute-connected=yes redistribute-static=yes
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Damien Gardner Jnr Sent: Friday, 27 March 2015 12:39 PM To: MikroTik Australia Public List Subject: Re: [MT-AU Public] RouterOS rebooting over and over? Possibly BGP + ipipv6 tunnels?
Yeah, iBGP Remote router is debian with quagga purely passing on the advertisements it receives (with the appropriate filters incase I fubar something at this end ;) ).
Disabling the accept input filter didn't help. Disabling the output filter did help, but I already knew that - sending another prefix works fine also.. turning off multihop didn't fix it either :(
Thanks,
Damien
On 27 March 2015 at 12:46, Tim Warnock <timoid@timoid.org> wrote:
Try disabling multihop as well. It is iBGP yeah?
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Damien Gardner Jnr Sent: Friday, 27 March 2015 11:41 AM To: MikroTik Australia Public List Subject: Re: [MT-AU Public] RouterOS rebooting over and over? Possibly BGP + ipipv6 tunnels?
Ill try disabling the in filter - it doesn't crash if i disable the accept out filter - it also doesn't crash if I'm sending a different prefix out.
On Friday, 27 March 2015, Tim Warnock <timoid@timoid.org> wrote:
> Try disabling lines 44,45 and 47 > > Does your VM crash then? > > > -----Original Message----- > > From: Public [mailto:public-bounces@talk.mikrotik.com.au <javascript:;>] > On Behalf Of > > Damien Gardner Jnr > > Sent: Friday, 27 March 2015 9:59 AM > > To: MikroTik Australia Public List > > Subject: Re: [MT-AU Public] RouterOS rebooting over and over? Possibly > > BGP + ipipv6 tunnels? > > > > There's no autosupout.rif being generated :( > > > > Sorry for the delay, I've been trying to work out what on > > earth is going > on. > > > > I've now started with a fresh (and 24hr licensed) 6.27 > > install, and > > absolute minimum config, and it's still shutting down. I've changed IP > > addresses locally, incase somehow the /28 from the /22 in question was > > conflicting. > > > > Config is here: http://pastebin.com/pTDS5JPM > > > > Within a minute or so of the bgp session coming up, the VM > > just shuts > > down. It's rather driving me nuts! > > > > Any thoughts? > > > > Thanks, > > > > Damien > > > > > > On 25 March 2015 at 21:21, Tim Warnock <timoid@timoid.org <javascript:;>> > wrote: > > > > > There should be a autosupout.rif if its crashing. > > > > > > Upload it to the rif viewer in your mikrotik.com account > > > page and > have a > > > look. > > > > > > But it sounds like a process is crashing. > > > > > > Do you have a diagram? > > > > > > > -----Original Message----- > > > > From: Public [mailto:public-bounces@talk.mikrotik.com.au > <javascript:;>] On Behalf > > Of > > > > Damien Gardner Jnr > > > > Sent: Wednesday, 25 March 2015 8:10 PM > > > > To: MikroTik Australia Public List > > > > Subject: Re: [MT-AU Public] RouterOS rebooting over and over? > Possibly > > > > BGP + ipipv6 tunnels? > > > > > > > > I did think that originally - so migrated it off the > > > > cluster and > onto a > > > > standalone xenserver head - and still the same issue. :\ > > > > > > > > I can advertise my second /22, not a problem at all At > > > > the moment I > have > > > > my secondary /22, a /24, and a v6 /48 advertised, and > > > > it's been up 14 > > > > hours. however within 2 minutes of advertising my > > > > primary /22 out > of it, > > > > it'll go down again. > > > > > > > > About to head to bed, but tomorrow I might try adding > > > > /24 > > advertisements > > > > on > > > > my primary transit's, and then do the same on this one - > > > > and see if > it's > > > > specifically to do with the /24 which also contains the > > > > /30's for the > > > ipv4 > > > > ip's on the v4 tunnels. Would be strange for that to be > > > > an issue, > but > > > > worth trying. -Though all four routers also do ibgp to > > > > each other > > > sending > > > > each other all prefixes so they also advertise the /22 > > > > from which > their > > > > local interfaces are in - the only difference is this > > > > one instead of > > > doing > > > > ebgp to an upstream is then doing ibgp to the US router (doing > multihop > > > and > > > > force-self), which in turn does ebgp to its upstream. > > > > > > > > On 25 March 2015 at 21:02, Stavros Patiniotis < > stavros@staff.esc.net.au <javascript:;>> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > The reboot, reboot shutdown is probably your HA setup > > > > > in xenserver. > > > > > > > > > > Does it matter which /22 you advertise? Ie can you advertise > either one > > > > > individually? > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Public > > > > > [mailto:public-bounces@talk.mikrotik.com.au > <javascript:;>] On Behalf > > Of > > > > > Damien Gardner Jnr > > > > > Sent: Wednesday, 25 March 2015 8:23 PM > > > > > To: MikroTik Australia Public List > > > > > Subject: [MT-AU Public] RouterOS rebooting over and over? Possibly > > BGP > > > + > > > > > ipipv6 tunnels? > > > > > > > > > > Hi Folks, > > > > > > > > > > Still waiting for my CCR to come back after dying, so > > > > > have > temporarily > > > > > cloned one of my RouterOS VM's and am running four of > > > > > them to > > > > accomplish > > > > > the > > > > > same as the CCR (and once the CCR comes back the plan > > > > > is to buy > three > > > > other > > > > > 'tik hardware routers, once I figure out exactly what > > > > > I want/need, > > > based on > > > > > the resources these VM's are using) > > > > > > > > > > Anyway, I seem to be having a weird problem with one > > > > > of the > routers. > > > it's > > > > > a > > > > > slightly strange config - I have ipipv6 tunnels > > > > > between the router > and > > > a > > > > > debian+quagga box in the US. Two tunnels on each of > > > > > debian+my upstream > > > > > debian+providers > > > > > (one for v4, one for v6, as linux does NOT want to run dual-stack > over > > > the > > > > > one tunnel). > > > > > > > > > > Since having issues, I've pulled it back to just the > > > > > two tunnels > > > between > > > > > the two routers - one for V4, one for V6. I then run > > > > > BGP over > both of > > > > > those > > > > > tunnels, and use that to advertise my prefixes in the US. > > > > > > > > > > It works well - provided I *don't* advertise my > > > > > primary /22. My > two v6 > > > > > /48's and my secondary /22 are advertising fine, and > > > > > work well. > > > However > > > > > within 2-3 minutes of advertising my primary /22, the router > reboots. > > > And > > > > > immediately reboots. And immediately reboots. And > > > > > then shuts down. > > > > I'm > > > > > not > > > > > sure if it's routerOS doing the shutdown, or xenserver detecting > the > > > > reboot > > > > > loop and forcing it down. > > > > > > > > > > Has anyone seen that before? There's nothing obvious > > > > > in the logs > right > > > > > before it reboots, and on boot it simply states that > > > > > the previous > > > shutdown > > > > > was unexpected. It doesn't generate a supout, so I'm guessing > it's not > > > > > hitting a watchdog or anything? > > > > > > > > > > It has me wondering if this is the same reason my > > > > > CCR1009 died? > As I > > > had > > > > > exactly the same config on the CCR (except it was > > > > > handling three > full > > > v4 + > > > > > v6 feeds, AND running the three pairs of ipipv6 > > > > > tunnels plus the > three > > > > > pairs > > > > > of send-only feeds to the US) > > > > > > > > > > Has me rather pulling my hair out.. Wanted to get my 'new' config > with > > > > > everything separate labbed up and running in > > > > > production before I go > > and > > > > > lash > > > > > out on all new 'tik hardware - but if 'tik isn't going > > > > > to do one of > > > them, > > > > > then I lose the awesomeness of winbox, and may as well > > > > > go back to > > cisco > > > > > gear > > > > > for all of it :-p > > > > > > > > > > Thanks, > > > > > > > > > > Damien > > > > > > > > > > -- > > > > > > > > > > Damien Gardner Jnr > > > > > VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net > > > > > <javascript:;> - http://www.rendrag.net/ > > > > > -- > > > > > We rode on the winds of the rising storm, We ran to > > > > > the sounds of > > > > thunder. > > > > > We danced among the lightning bolts, and tore the > > > > > world asunder > > > > >
> > > > > Public mailing list > > > > > Public@talk.mikrotik.com.au <javascript:;> > > > > > > > > >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
> > > > > > > > > > > > > > >
> > > > > Public mailing list > > > > > Public@talk.mikrotik.com.au <javascript:;> > > > > > > > > >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
> > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Damien Gardner Jnr > > > > VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net > > > > <javascript:;> - http://www.rendrag.net/ > > > > -- > > > > We rode on the winds of the rising storm, We ran to the > > > > sounds of thunder. > > > > We danced among the lightning bolts, and tore the world > > > > asunder
> > > > Public mailing list > > > > Public@talk.mikrotik.com.au <javascript:;> > > > > > >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.co m.au
> > > > > > _______________________________________________ > > > Public mailing list > > > Public@talk.mikrotik.com.au <javascript:;> > > > >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
> > > > > > > > > > > -- > > > > Damien Gardner Jnr > > VK2TDG. Dip EE. GradIEAust > > rendrag@rendrag.net <javascript:;> - > > http://www.rendrag.net/ > > -- > > We rode on the winds of the rising storm, We ran to the > > sounds of thunder. > > We danced among the lightning bolts, and tore the world > > asunder
> > Public mailing list > > Public@talk.mikrotik.com.au <javascript:;> > >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.co m.au
> > _______________________________________________ > Public mailing list > Public@talk.mikrotik.com.au <javascript:;> >
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
>
--
Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.co m.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
--
Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.co m.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
--
Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
-- Damien Gardner Jnr VK2TDG. Dip EE. GradIEAust rendrag@rendrag.net - http://www.rendrag.net/ -- We rode on the winds of the rising storm, We ran to the sounds of thunder. We danced among the lightning bolts, and tore the world asunder