Hi I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :) I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s A Not sure if this dup or not .. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Tuesday, 20 October 2015 2:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28 - they claim enormous performance boost (like up to 15 times faster!) in high traffic situation. Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-) Cheers! Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au Sent from my smart phone
Out of interest, is it the same for IPv6? -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 4:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86 Hi I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :) I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s A Not sure if this dup or not .. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Tuesday, 20 October 2015 2:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28 - they claim enormous performance boost (like up to 15 times faster!) in high traffic situation. Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-) Cheers! Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hmm, I didn't test ipv6, but I would presume so. A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Tim Warnock Sent: Tuesday, 20 October 2015 6:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Out of interest, is it the same for IPv6? -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 4:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86 Hi I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :) I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s A Not sure if this dup or not .. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Tuesday, 20 October 2015 2:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28 - they claim enormous performance boost (like up to 15 times faster!) in high traffic situation. Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-) Cheers! Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
I am enjoying reading this thread an I like the numbers you are seeing out of single thread sessions on the x86 units. I also have had some frustration with the CCR and single thread TCP and single core BGP however I manage that the best I can. The issues I have had in the past with x86 hardware (i cant find the data at the moment) has always been once you get into larger session counts, the x86 PCIe/bus/north/south bridge just cant cope and you start bogging machines down. This used to be a big issue and one of the reasons I moved to the CCR over x86. This is not something I have tested recently and may be fixed, but thought that while we are all looking at single and 8-12 thread performance, should really look at what happens when you put lots of big session counts into these things as well before you commit to one over another. My 2 cents. Thanks Daniel On Wed, Oct 21, 2015 at 8:52 AM, Alex Samad - Yieldbroker < Alex.Samad@yieldbroker.com> wrote:
Hmm, I didn't test ipv6, but I would presume so.
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Tim Warnock Sent: Tuesday, 20 October 2015 6:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Out of interest, is it the same for IPv6?
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 4:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi Like both :) I use my ccr for the core and VM on Dell ESXi hosts for environmental router/firewall. I have worked around my 1G tcp limit where I can. And for the price ... can't really go wrong.. A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Daniel Hoffman Sent: Wednesday, 21 October 2015 9:02 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 I am enjoying reading this thread an I like the numbers you are seeing out of single thread sessions on the x86 units. I also have had some frustration with the CCR and single thread TCP and single core BGP however I manage that the best I can. The issues I have had in the past with x86 hardware (i cant find the data at the moment) has always been once you get into larger session counts, the x86 PCIe/bus/north/south bridge just cant cope and you start bogging machines down. This used to be a big issue and one of the reasons I moved to the CCR over x86. This is not something I have tested recently and may be fixed, but thought that while we are all looking at single and 8-12 thread performance, should really look at what happens when you put lots of big session counts into these things as well before you commit to one over another. My 2 cents. Thanks Daniel On Wed, Oct 21, 2015 at 8:52 AM, Alex Samad - Yieldbroker < Alex.Samad@yieldbroker.com> wrote:
Hmm, I didn't test ipv6, but I would presume so.
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Tim Warnock Sent: Tuesday, 20 October 2015 6:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Out of interest, is it the same for IPv6?
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 4:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
IPv4 needs to be re check summed as the packets pass through (IE: due to ttl decreasing). IPv6 doesn't have such a checksum. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 7:52 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Hmm, I didn't test ipv6, but I would presume so. A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Tim Warnock Sent: Tuesday, 20 October 2015 6:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Out of interest, is it the same for IPv6? -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 4:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86 Hi I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :) I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s A Not sure if this dup or not .. -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Tuesday, 20 October 2015 2:46 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28 - they claim enormous performance boost (like up to 15 times faster!) in high traffic situation. Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-) Cheers! Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi Alex, Sorry to harp on with the same suggestion over and over :-} I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it. Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-) Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi No problem ... I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F 3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M Client the same point at server. Iperf -c <ip address> -I 10 -w32m I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr For some extra testing I add in C so it was vm to vm on different host. A to c via powerconnect A to c via ccr Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu. I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email. I think I tried to change the queue type but that didn't help. This was a while back maybe 18+ months . Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb.. A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Hi Alex, Sorry to harp on with the same suggestion over and over :-} I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it. Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-) Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and
tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce
Hi Alex, That is helpful, BUT: 1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states. You are welcome to send it to support@duxtel.com to keep it out of the general list :-} Cheers, Mike. they the
issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and
tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce
Hi I haven't tested in that time... If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing. Alex -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Hi Alex, That is helpful, BUT: 1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states. You are welcome to send it to support@duxtel.com to keep it out of the general list :-} Cheers, Mike. they the
issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi Alex, What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-} We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself. What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;) But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating... So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-) Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi If I get a chance to test I will do. You just need s system support file ? A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Hi Alex, What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-} We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself. What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;) But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating... So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-) Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite
that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so
Thanks! :) We need: a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue. Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-) Cheers! Mike. possible that
we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
Question were you able to get above 1Gb single tcp stream on the CCR.
When I was testing on my ccr36 I could never get a single tcp stream
above
1Gb/s
A
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Alex, By any chance were you using bonding/lacp on the links from your CCR's to your switches ? On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
> Question were you able to get above 1Gb single tcp stream on > the CCR.
>
> When I was testing on my ccr36 I could never get a single tcp > stream
above
1Gb/s
>
> A
>
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hmmm, yes... My sfp+ ports are LACP'ed into a stacked 10G switch and I have vlans off that A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Thursday, 22 October 2015 7:31 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Alex, By any chance were you using bonding/lacp on the links from your CCR's to your switches ? On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of
Alex Samad - Yieldbroker
Sent: Tuesday, 20 October 2015 2:36 PM
To: terry@skymesh.net.au; MikroTik Australia Public List
<public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
Hi
So was that a yes or no
http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this is a
thread I started, I have read other threads about it. Seems to be
based around the fact the CCR code base it not multi threaded another
example being the BGP
as for http://forum.mikrotik.com/viewtopic.php?t=62377 not sure what I
was supposed to get from that, those where not CCR's
sorry maybe im missing something ?
A
-----Original Message-----
From: Terry Sweetser (SkyMesh) [mailto:terry+mikrotik@skymesh.net.au]
Sent: Tuesday, 20 October 2015 1:28 PM
To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; MikroTik
Australia Public List <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
---
On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
> Question were you able to get above 1Gb single tcp stream > on the CCR.
>
> When I was testing on my ccr36 I could never get a single > tcp stream
above
1Gb/s
>
> A
>
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Try without the bond. It _should_ go over the 1gbit limit you previously hit. On 23/10/2015 11:02 am, "Alex Samad - Yieldbroker" < Alex.Samad@yieldbroker.com> wrote:
Hmmm, yes...
My sfp+ ports are LACP'ed into a stacked 10G switch and I have vlans off that
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Thursday, 22 October 2015 7:31 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Alex,
By any chance were you using bonding/lacp on the links from your CCR's to your switches ?
On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
> -----Original Message-----
> From: Public [mailto:public-bounces@talk.mikrotik.com.au] On > Behalf Of
> Alex Samad - Yieldbroker
> Sent: Tuesday, 20 October 2015 2:36 PM
> To: terry@skymesh.net.au; MikroTik Australia Public List
> <public@talk.mikrotik.com.au>
> Subject: Re: [MT-AU Public] X86
>
> Hi
>
> So was that a yes or no
>
> http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this > is a
> thread I started, I have read other threads about it. Seems > to be
> based around the fact the CCR code base it not multi threaded > another
> example being the BGP
>
>
> as for http://forum.mikrotik.com/viewtopic.php?t=62377 not > sure what I
> was supposed to get from that, those where not CCR's
>
> sorry maybe im missing something ?
>
> A
>
> -----Original Message-----
> From: Terry Sweetser (SkyMesh) > [mailto:terry+mikrotik@skymesh.net.au]
> Sent: Tuesday, 20 October 2015 1:28 PM
> To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; > MikroTik
> Australia Public List <public@talk.mikrotik.com.au>
> Subject: Re: [MT-AU Public] X86
>
> http://forum.mikrotik.com/viewtopic.php?t=62377
>
> ---
> http://about.me/terry.sweetser
>
> On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
> > Question were you able to get above 1Gb single tcp stream > > on the CCR.
> >
> > When I was testing on my ccr36 I could never get a single > > tcp stream
above
> 1Gb/s
> >
> > A
> >
>
>
> _______________________________________________
> Public mailing list
> Public@talk.mikrotik.com.au
> http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
> au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi I could, it would involve a lot of work, and I lose my redundancy. Are you suggesting the LACP / Bonding is limiting the traffic ? And why only for single tcp streams and not udp traffic ? Could I instead convert the bond from active / active to active passive would that help ?? A -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Friday, 23 October 2015 7:05 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Try without the bond. It _should_ go over the 1gbit limit you previously hit. On 23/10/2015 11:02 am, "Alex Samad - Yieldbroker" < Alex.Samad@yieldbroker.com> wrote:
Hmmm, yes...
My sfp+ ports are LACP'ed into a stacked 10G switch and I have vlans off that
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Thursday, 22 October 2015 7:31 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Alex,
By any chance were you using bonding/lacp on the links from your CCR's to your switches ?
On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Tuesday, 20 October 2015 5:43 PM To: public@talk.mikrotik.com.au Subject: Re: [MT-AU Public] X86
Hi
I have to admit it was a while ago. Also I looked through the slides from Melbourne they mentioned it would be fixed in 7 ... soon :)
I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the CCR in the middle and it dropped to 1Gb/s. if I started 8 parralel tcp streams I go 8Gb/s
A
Not sure if this dup or not ..
-----Original Message-----
From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest
Sent: Tuesday, 20 October 2015 2:46 PM
To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au>
Subject: Re: [MT-AU Public] X86
What routerOS version shows that behaviour? I know that there has been a heap of work done on multi-core queue architecture since around 6.20 to 6.28
- they claim enormous performance boost (like up to 15 times faster!) in high traffic situation.
Also, make sure that your testing methodology does not introduce some limitation. Btest, for example, is limited to 1 cpu, so if your btest client pegs a CPU, it is not necessarily indicative of a limitation with routing ;-)
Cheers!
Mike.
> -----Original Message-----
> From: Public [mailto:public-bounces@talk.mikrotik.com.au] > On Behalf Of
> Alex Samad - Yieldbroker
> Sent: Tuesday, 20 October 2015 2:36 PM
> To: terry@skymesh.net.au; MikroTik Australia Public List
> <public@talk.mikrotik.com.au>
> Subject: Re: [MT-AU Public] X86
>
> Hi
>
> So was that a yes or no
>
> http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this > is a
> thread I started, I have read other threads about it. Seems > to be
> based around the fact the CCR code base it not multi > threaded another
> example being the BGP
>
>
> as for http://forum.mikrotik.com/viewtopic.php?t=62377 not > sure what I
> was supposed to get from that, those where not CCR's
>
> sorry maybe im missing something ?
>
> A
>
> -----Original Message-----
> From: Terry Sweetser (SkyMesh) > [mailto:terry+mikrotik@skymesh.net.au]
> Sent: Tuesday, 20 October 2015 1:28 PM
> To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; > MikroTik
> Australia Public List <public@talk.mikrotik.com.au>
> Subject: Re: [MT-AU Public] X86
>
> http://forum.mikrotik.com/viewtopic.php?t=62377
>
> ---
> http://about.me/terry.sweetser
>
> On 17/10/15 06:45, Alex Samad - Yieldbroker wrote:
> > Question were you able to get above 1Gb single tcp stream > > on the CCR.
> >
> > When I was testing on my ccr36 I could never get a single > > tcp stream
above
> 1Gb/s
> >
> > A
> >
>
>
> _______________________________________________
> Public mailing list
> Public@talk.mikrotik.com.au
> http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
> au
_______________________________________________
Public mailing list
Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
Sent from my smart phone _______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi Alex, It may be.. For us we experienced similar issues on 6.5, 6.19 and 6.27 with bonding. On Sat, Oct 24, 2015 at 9:39 AM, Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com> wrote:
Hi
I could, it would involve a lot of work, and I lose my redundancy. Are you suggesting the LACP / Bonding is limiting the traffic ? And why only for single tcp streams and not udp traffic ?
Could I instead convert the bond from active / active to active passive would that help ??
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Friday, 23 October 2015 7:05 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Try without the bond.
It _should_ go over the 1gbit limit you previously hit. On 23/10/2015 11:02 am, "Alex Samad - Yieldbroker" < Alex.Samad@yieldbroker.com> wrote:
Hmmm, yes...
My sfp+ ports are LACP'ed into a stacked 10G switch and I have vlans off that
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Thursday, 22 October 2015 7:31 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Alex,
By any chance were you using bonding/lacp on the links from your CCR's to your switches ?
On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
> -----Original Message----- > From: Public [mailto:public-bounces@talk.mikrotik.com.au] On > Behalf Of Alex Samad - Yieldbroker > Sent: Tuesday, 20 October 2015 5:43 PM > To: public@talk.mikrotik.com.au > Subject: Re: [MT-AU Public] X86 > > Hi > > > > I have to admit it was a while ago. Also I looked through the > slides from Melbourne they mentioned it would be fixed in 7 ... > soon :) > > > > I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the > CCR in the middle and it dropped to 1Gb/s. if I started 8 > parralel tcp streams I go 8Gb/s > > > > A > > Not sure if this dup or not .. > > > > -----Original Message----- > > From: Public [mailto:public-bounces@talk.mikrotik.com.au] On > Behalf Of Mike Everest > > Sent: Tuesday, 20 October 2015 2:46 PM > > To: 'MikroTik Australia Public List' > <public@talk.mikrotik.com.au> > > Subject: Re: [MT-AU Public] X86 > > > > What routerOS version shows that behaviour? I know that there > has been a heap of work done on multi-core queue architecture > since around 6.20 to > 6.28 > > - they claim enormous performance boost (like up to 15 times > faster!) in high > traffic situation. > > > > Also, make sure that your testing methodology does not > introduce some limitation. Btest, for example, is limited to > 1 cpu, so if your btest client pegs > a CPU, it is not necessarily indicative of a limitation with > routing > ;-) > > > > Cheers! > > > > Mike. > > > > > -----Original Message----- > > > From: Public [mailto:public-bounces@talk.mikrotik.com.au] > > On Behalf Of > > > Alex Samad - Yieldbroker > > > Sent: Tuesday, 20 October 2015 2:36 PM > > > To: terry@skymesh.net.au; MikroTik Australia Public List > > > <public@talk.mikrotik.com.au> > > > Subject: Re: [MT-AU Public] X86 > > > > > > Hi > > > > > > So was that a yes or no > > > > > > http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << this > > is a > > > thread I started, I have read other threads about it. Seems > > to be > > > based around the fact the CCR code base it not multi > > threaded another > > > example being the BGP > > > > > > > > > as for http://forum.mikrotik.com/viewtopic.php?t=62377 not > > sure what I > > > was supposed to get from that, those where not CCR's > > > > > > sorry maybe im missing something ? > > > > > > A > > > > > > -----Original Message----- > > > From: Terry Sweetser (SkyMesh) > > [mailto:terry+mikrotik@skymesh.net.au] > > > Sent: Tuesday, 20 October 2015 1:28 PM > > > To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; > > MikroTik > > > Australia Public List <public@talk.mikrotik.com.au> > > > Subject: Re: [MT-AU Public] X86 > > > > > > http://forum.mikrotik.com/viewtopic.php?t=62377 > > > > > > --- > > > http://about.me/terry.sweetser > > > > > > On 17/10/15 06:45, Alex Samad - Yieldbroker wrote: > > > > Question were you able to get above 1Gb single tcp stream > > > on the CCR. > > > > > > > > When I was testing on my ccr36 I could never get a single > > > tcp stream > > above > > > 1Gb/s > > > > > > > > A > > > > > > > > > > > > > _______________________________________________ > > > Public mailing list > > > Public@talk.mikrotik.com.au > > > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > > > au > > > > > > _______________________________________________ > > Public mailing list > > Public@talk.mikrotik.com.au > > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > au > > Sent from my smart phone > _______________________________________________ > Public mailing list > Public@talk.mikrotik.com.au > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
Hi Okay, so your saying you can push more than 1Gb/s in a single stream through a CCR and you setup didn't include bonded interfaces. Alex -----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Wednesday, 28 October 2015 12:39 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86 Hi Alex, It may be.. For us we experienced similar issues on 6.5, 6.19 and 6.27 with bonding. On Sat, Oct 24, 2015 at 9:39 AM, Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com> wrote:
Hi
I could, it would involve a lot of work, and I lose my redundancy. Are you suggesting the LACP / Bonding is limiting the traffic ? And why only for single tcp streams and not udp traffic ?
Could I instead convert the bond from active / active to active passive would that help ??
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Friday, 23 October 2015 7:05 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Try without the bond.
It _should_ go over the 1gbit limit you previously hit. On 23/10/2015 11:02 am, "Alex Samad - Yieldbroker" < Alex.Samad@yieldbroker.com> wrote:
Hmmm, yes...
My sfp+ ports are LACP'ed into a stacked 10G switch and I have vlans off that
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Andrew Thrift Sent: Thursday, 22 October 2015 7:31 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Alex,
By any chance were you using bonding/lacp on the links from your CCR's to your switches ?
On Thu, Oct 22, 2015 at 4:12 PM, Mike Everest <mike@duxtel.com> wrote:
Thanks! :)
We need:
a) description of test set-up (what devices are involved, OS type and versions, hardware/interface manufacturer, model and driver version) and how they connect together b) description of your test procedure (what steps are taken - in what order - to observe the behaviour) c) supout.rif from all devices involved
It's a lot to ask, I know, but it's what we need to capture the attention of MikroTik engineers and get them on side to resolve the issue.
Bottom line is this: CCR is designed to route 10Gbps on all interfaces at the same time - so if it doesn't then it does not work to specification, and needs to be fixed! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Thursday, 22 October 2015 12:21 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
If I get a chance to test I will do. You just need s system support file ?
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Thursday, 22 October 2015 12:09 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
What I'm saying is that (according to MT engineering team) there is no inherent reason why CCR would not forward more than 1gbps of traffic :-}
We don't have any systems on hand that have 10Gbe interfaces, so I can't run the test myself.
What I /do/ know is that each time we have heard reports like that (and we have heard about it a few times! ;), we have never been able to confirm it with thorough test methodology. Each time we have heard this report, we have asked for more details but not received any further clarification - sometimes, the next report says that the problem has gone away :-j
Other times, the problem is related to use of btest tool in routerOS - which is limited to one CPU and can easily chew up 100% of that cpu core before reaching much more than 1 gbps ;)
But I have heard of this issue enough times that I consider it quite possible that there is some kind of problem in there worth investigating...
So I would really like to find a case of the problem actually happening so that we can investigate properly. If you are willing, then we are ready and able to assist! :-)
Cheers!
Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 5:51 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
I haven't tested in that time...
If you saying you have a ccr 1036 and you can push > 1Gb tcp stream through, then I will organise some time to re org my setup for testing.
Alex
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 4:43 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
That is helpful, BUT:
1) you say it was 18+ months ago - does it still happen now? 2) we need supout files from all devices in the mix so that we can understand the configuration states.
You are welcome to send it to support@duxtel.com to keep it out of the general list :-}
Cheers, Mike.
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Alex Samad - Yieldbroker Sent: Wednesday, 21 October 2015 2:13 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi
No problem ...
I had R620 Dell server ESXi 5.5i 4 x 10G NIC 2 vswitches 2 nic in each vswitch active active .. 10G dell powerconnect 8024F
3 vm .. Centos 6.5 at the time vmnext3 Vm-a & vm-b on the same esx host Vm-c was on another host via 8km of dark fibre - 10G network
Setup server iperf print every 10sec and windows size of 32Meg Iperf -s -I 10 -w32M
Client the same point at server. Iperf -c <ip address> -I 10 -w32m
I ran A to b (so same vmware host A to b via l3 on the powerconnect A to b via the ccr
For some extra testing I add in C so it was vm to vm on different host.
A to c via powerconnect A to c via ccr
Always the single stream of tcp was limited to 1G, cpu showed high usage on 1 cpu.
I am pretty sure this has come up in the forums multiple times and I am pretty sure support has acknowledge this to me via email.
I think I tried to change the queue type but that didn't help.
This was a while back maybe 18+ months .
Now if you start 2 of these or get iperf to do some in parallel the total gets up to 9.8Gb. And UDP just pushes up to 9.8Gb..
A
-----Original Message----- From: Public [mailto:public-bounces@talk.mikrotik.com.au] On Behalf Of Mike Everest Sent: Wednesday, 21 October 2015 1:30 PM To: 'MikroTik Australia Public List' <public@talk.mikrotik.com.au> Subject: Re: [MT-AU Public] X86
Hi Alex,
Sorry to harp on with the same suggestion over and over :-}
I have discussed this behaviour report at some length with MikroTik and they tell me that there is no reason why that behaviour should occur. They are very keen to try to reproduce the issue and investigate further for a fix, but they have not been able to reproduce the problem, and so they can't even begin to address it.
Therefore, if anyone at all can provide us with some steps to reproduce the issue (send it to support@duxtel.com) then we'll be very pleased to take it up with MikroTik for review and comment :-)
Cheers! Mike.
> -----Original Message----- > From: Public [mailto:public-bounces@talk.mikrotik.com.au] On > Behalf Of Alex Samad - Yieldbroker > Sent: Tuesday, 20 October 2015 5:43 PM > To: public@talk.mikrotik.com.au > Subject: Re: [MT-AU Public] X86 > > Hi > > > > I have to admit it was a while ago. Also I looked through > the slides from Melbourne they mentioned it would be fixed in 7 ... > soon :) > > > > I had 2 VM's which were able to iperf 9.8Gb/s tcp. Place the > CCR in the middle and it dropped to 1Gb/s. if I started 8 > parralel tcp streams I go 8Gb/s > > > > A > > Not sure if this dup or not .. > > > > -----Original Message----- > > From: Public [mailto:public-bounces@talk.mikrotik.com.au] On > Behalf Of Mike Everest > > Sent: Tuesday, 20 October 2015 2:46 PM > > To: 'MikroTik Australia Public List' > <public@talk.mikrotik.com.au> > > Subject: Re: [MT-AU Public] X86 > > > > What routerOS version shows that behaviour? I know that > there has been a heap of work done on multi-core queue > architecture since around 6.20 to > 6.28 > > - they claim enormous performance boost (like up to 15 times > faster!) in high > traffic situation. > > > > Also, make sure that your testing methodology does not > introduce some limitation. Btest, for example, is limited to > 1 cpu, so if your btest client pegs > a CPU, it is not necessarily indicative of a limitation with > routing > ;-) > > > > Cheers! > > > > Mike. > > > > > -----Original Message----- > > > From: Public [mailto:public-bounces@talk.mikrotik.com.au] > > On Behalf Of > > > Alex Samad - Yieldbroker > > > Sent: Tuesday, 20 October 2015 2:36 PM > > > To: terry@skymesh.net.au; MikroTik Australia Public List > > > <public@talk.mikrotik.com.au> > > > Subject: Re: [MT-AU Public] X86 > > > > > > Hi > > > > > > So was that a yes or no > > > > > > http://forum.mikrotik.com/viewtopic.php?f=2&t=82359 << > > this is a > > > thread I started, I have read other threads about it. > > Seems to be > > > based around the fact the CCR code base it not multi > > threaded another > > > example being the BGP > > > > > > > > > as for http://forum.mikrotik.com/viewtopic.php?t=62377 not > > sure what I > > > was supposed to get from that, those where not CCR's > > > > > > sorry maybe im missing something ? > > > > > > A > > > > > > -----Original Message----- > > > From: Terry Sweetser (SkyMesh) > > [mailto:terry+mikrotik@skymesh.net.au] > > > Sent: Tuesday, 20 October 2015 1:28 PM > > > To: Alex Samad - Yieldbroker <Alex.Samad@yieldbroker.com>; > > MikroTik > > > Australia Public List <public@talk.mikrotik.com.au> > > > Subject: Re: [MT-AU Public] X86 > > > > > > http://forum.mikrotik.com/viewtopic.php?t=62377 > > > > > > --- > > > http://about.me/terry.sweetser > > > > > > On 17/10/15 06:45, Alex Samad - Yieldbroker wrote: > > > > Question were you able to get above 1Gb single tcp > > > stream on the CCR. > > > > > > > > When I was testing on my ccr36 I could never get a > > > single tcp stream > > above > > > 1Gb/s > > > > > > > > A > > > > > > > > > > > > > _______________________________________________ > > > Public mailing list > > > Public@talk.mikrotik.com.au > > > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > > > au > > > > > > _______________________________________________ > > Public mailing list > > Public@talk.mikrotik.com.au > > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > au > > Sent from my smart phone > _______________________________________________ > Public mailing list > Public@talk.mikrotik.com.au > http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. > au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au
http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.
au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikroti k .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikroti k .com . au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik. c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik. c om .au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com. au
_______________________________________________ Public mailing list Public@talk.mikrotik.com.au http://talk.mikrotik.com.au/mailman/listinfo/public_talk.mikrotik.com.au
participants (5)
-
Alex Samad - Yieldbroker -
Andrew Thrift -
Daniel Hoffman -
Mike Everest -
Tim Warnock