January 26th, 2005, 04:13
After verifying that may raid card is supported in OBSD I decided to move off of win2k3, and use OBSD + Samba for all my network storage sharing stuff. I moved all my data, about 360Gb of movies, music and documents over to a spare server, and installed OBSD 3.6 on the prime server. When I initally moved the data from win2k3 to my temp OBSD box, I was really frustrated with the file transfer speed. I have a Intel Pro 1000GB (SX) Server card in each system. Both boxes are connected back to back via fiber cable. The network is tiny using just 2 ips on the whole subnet ( With the default MTU setting on both machines max thruput topped out at about 3mb (3000kb). which is correct for ether net running a default MTU of 1500 and having a full-dux connection speed. My buddy told me to set up jumbo frames and give that a spin to decrease file transfer times. So i did the ifconfig <addr> mtu 9000 on the OBSD box. On the win2k3 box things got dicey, as the default windows drivers do not support jumbo frames. So i had to download another set of drivers directly from Intel. After doing this and setting jumbo frames to intel's preset or 9016, I tried slinging a few fat packets to the OBSD box. It worked no fragments just 8000 byte packets coming into the OBSD box. So then I fire up sftp to start transfering files and I acually get LESS bandwidth than before. So I chalk it up to stupid windows and set everything back to the default MTU of 1500, slog thru 3 days of sftp transfers, wipe the win2k3 box in install OBSD. This time I use ifconfig on both servers to set the MTU to 9032 (added 32 for header overhead). Slung a few packets back and forth to make sure no fragmentation occured and started my scp.
Max download is now 3.5mb. What am I doing wrong? These are Gigabit fiber network adapters. I can see no reason why they should not be able to get at least 250mb using this config. Any suggestions or ideas would be greatly appreciated.
Oh and heres what the ifconfig says:

address: 00:d0:b7:82:27:27
media: Ethernet 1000baseSX full-duplex
status: active
inet netmask 0xfffffffc broadcast
inet6 fe80::2d0:b7ff:fe82:2727%em0 prefixlen 64 scopeid 0x2

address: 00:03:47:25:35:50
media: Ethernet 1000baseSX full-duplex
status: active
inet netmask 0xfffffffc broadcast
inet6 fe80::203:47ff:fe25:3550%em0 prefixlen 64 scopeid 0x1

And here is what the netstat -rn says:

SERVER- link#2 UC 2 0 - em0 0:3:47:25:35:50 UHLc 2 1031554 - em0 link#2 UHLc 2 67 - em0

CLIENT- link#1 UC 2 0 - em0 0:d0:b7:82:27:27 UHLc 2 1112362 - em0 0:3:47:25:35:50 UHLc 0 8 - lo0

Thanks in advance.

January 26th, 2005, 15:38
Huh this guy shows a similar problem:

No replies...

This post says Jumobo frames on em(4) are now reliable:

I'll keep looking around but that's all I can see right now.

January 26th, 2005, 15:47
Thanks E.

January 26th, 2005, 15:56
Here's Ryan McBride's answer:

January 27th, 2005, 07:29
Yeah I saw this when I was snooping around the Net too. But I am using 3.6 so I wonder if the issue still exists. I am too tired to load both machines up with SuSE 9.0 so I guess it'll have to wait from me to test later. I'll keep you all informed.

February 6th, 2005, 21:26
So does that mean that non broadcom chipsets can do large frames reliably and speedily?

February 12th, 2005, 00:21
Change you jumbo size to 9216 that's my switch is set to by default. I played them today and did not have an issue.

March 3rd, 2005, 14:47
I'll try this tonight. Thanks.

March 4th, 2005, 10:18
You're transferring files using sftp, which isn't the fastest way to transfer files and definitely not the best way to judge network performance since it adds your computer's encryption performance into the mix, plus I seem to recall there being other issues with sftp that slows down transfers as well - check the mailing list.

If you just want to see how fast your network is, use a tool other than sftp.

March 9th, 2005, 00:37
Check. I was just using sftp to do a basic test. Not really a benchmark. I am making the changes to the box right now. And I'll get a benchmarking app to see the real speed soon.

March 23rd, 2005, 15:59
I set the packet size to 9216 and had some serious issues. I think the problem is a WindowsXP + OpenBSD problem. I plan on doing further tests but I need to get the system up and running for a party I have next weekend.