molotov
September 20th, 2004, 19:51
As 5.3 is approaching and eagarly anticipated by us SMP users, I figured the src summarys might be nice too see
FreeBSD cvs-src summary for 13/09/04 to 20/09/04
++++++++++++++++++++++++++++++++++++++++++++++++
This is a regular weekly summary of FreeBSD's cutting-edge development.
It is intended to help the FreeBSD community keep up with the fast-paced
work going on in FreeBSD-CURRENT by distilling the deluge of data from
the CVS mailing list into a (hopefully) easy-to-read newsletter. This
newsletter is marked up in reStructuredText_, so any odd punctuation
that you see is likely intended for the reST parser.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
You can get old summaries, and an HTML version of this one, at
http://www.xl0.org/FreeBSD/. Please send any comments to Mark Johnston
(mark at xl0.org).
If you would like to get the summary without subscribing to current@,
please send mail to freebsd-cvs-summary-subscribe@lists.enderunix.org.
Thanks to Omer Faruk Sen and EnderUNIX for hosting this list.
For Lukasz Dudek and Szymon Roczniak's Polish translations of these
summaries, which may lag the English ones slightly, please see
http://mocart.pinco.pl/FreeBSD/.
.. contents::
============
New features
============
Netgraph implementation of Netflow v5
-------------------------------------
Gleb Smirnoff (glebius) added an implementation of Netflow v5 as a
Netgraph node. Netflow is a specification that allows routers to export
network accounting data in "flow" format to remote monitoring tools
like ntop_, FlowScan_, or Cisco's FlowCollector_. This development was
supported by the ISPs Bestcom and Rinet.
.. _ntop: http://www.ntop.org/
.. _FlowScan: http://www.caida.org/tools/utilities/flowscan/
.. _FlowCollector: http://www.cisco.com/univercd/cc/td/doc/pcat/24.htm
http://www.freebsd.org/cgi/mid.cgi?200409162024.i8GKONv3048588
BIND upgraded to version 9.3.0rc4
---------------------------------
Tom Rhodes (trhodes) imported BIND_ version 9.3.0rc4. This is the first
import of the 9.x series of BIND, which represents a complete rewrite
since version 8.
.. _BIND: http://www.isc.org/sw/bind/
http://www.freebsd.org/cgi/mid.cgi?200409190130.i8J1UPZN059043
===============
Notable changes
===============
Rework of GEOM Vinum code
-------------------------
Lukas Ertl (le) heavily modified the GEOM Vinum code, for doing RAID
under GEOM, to change how I/O is handled. This should fix recent data
corruption problems and improve performance.
http://www.freebsd.org/cgi/mid.cgi?200409181344.i8IDiiOL033004
=================
Discussion topics
=================
Supporting (and testing) really big disks
-----------------------------------------
Pawel Jakub Dawidek (pjd) reworked the md code, which supports swap-backed
simulated disks, to be 64-bit clean, as well as cleaning up and touching
up the code. He noted, "This work is a part of 'BigDisk' project:
http://www.freebsd.org/projects/bigdisk". He also gave a couple of
examples::
ragepxe:root:~# df -h
Filesystem Size Used Avail Capacity
Mounted on
[...]
/dev/md0 15T 249M 14T 0% /mnt
/dev/md1 11T 128K 10T 0% /mnt2
ragepxe:root:~# mdconfig -l -u 0
md0 swap 15T
ragepxe:root:~# mdconfig -l -u 1
md1 vnode 11T
Giorgos Keramidas (keramida) replied, "I think most of us were already
jealous enough by seeing the first post :-P"
Pawel answered, "Jealous about what?:) You know, that you can setup 15TB
of md(4) swap-backed device on 512MB swap partition, right?"
Ceri Davies (ceri) wondered, "What happens if you try to fill them?"
Giorgos responded enigmatically, "The best thing that happens is that lots
of nasal demons start flying out of your nose. You don't really want to
know the worst ;-)"
Scott Long (scottl) also replied, saying, "The advantage of what Pawel
has done is that you can simulate very large disks for the purpose of
debugging other utlities [ . . . ]."
Ceri clarified, "Don't misunderstand me, I'm appreciative of that. My
question stemmed from curiosity alone[.]"
Scott replied, "No offense taken. See
http://www.freebsd.org/projects/bigdisk for more details."
Pawel also added to Scott's first reply, noting, "It is still impossible
to detach such big md(4) device if it is filled. System panics somewhere
in swap code, but I don't think it is important for our use."
Robert Watson (rwatson) responded to Ceri's initial question, explaining,
"Bewm."
Brian Fundakowski Feldman (green) followed up, "Doesn't that have mostly
to do with vm_page_grab() and sf_buf_alloc() and vm_pager_get_pages() not
having any sort of error handling?"
Robert elaborated, "There are probably numerous sorts of errors, not least
that once you fill your swap space, you're going to be out of swap, and
file systems are written to assume that disks don't run out of space after
declaring they have the space (i.e., that they really are disks)."
Poul-Henning Kamp (phk) replied to Robert's previous post, saying, "Uhm,
be aware that we are limited also to the size of individual swap disks.
31 bits of bytes/sectors/pages, not quite sure which."
Scott clarified, "32 bits of pages, unless there is somewhere in the chain
that the unsigned nature of the word is not being honored."
Pawel also replied, "Pages. It gives as maximum swap-backed device of
16TB. I tested it with ~15TB devices, so there is no signed/unsigned
problem (which will limit us to 8TB)."
Poul-Henning responded, "I belive there is a limit both for the total
swapspce (16TB sounds likely) but also a limit for each swapdevice as far
as I know."
Brooks Davis (brooks) asked, "Given my quick read of the sunlabel format,
there would be some value in 16TB (17 would be sufficent) virtual disks.
Can we do a RAID0 of these to get something that big?"
Scott explained, "The 16TB limit comes [ . . . ] from using 4k pages for
swap backing and only being about to address these pages using 32 bit
numbers. The important part of the I/O path have already been converted
(or are in the process of being converted) to use 64 bit numbers."
Pawel also answered Brooks, noting, "You mean setup raid0 with md(4)
device to get more than 16TB? Sure, at least with gstripe[.]" He gave an
example of a 1.9 petabyte simulated disk using 128 mds in RAID 0.
Brooks followed up, "Just what I was looking for. Very cool!"
http://www.freebsd.org/cgi/mid.cgi?200409162132.i8GLWDpF050910
=================
Committer changes
=================
Murray Stokely (murray) introduced Xin LI (delphij) as a new src
committer. He will be working on Chinese (Simplified) documentation,
importing bugfixes from DragonflyBSD, and continuing to fix bugs in both
the source code and documentation.
http://www.freebsd.org/cgi/mid.cgi?200409141919.i8EJJLcO027263
===================
Important bug fixes
===================
Vulnerabilities in CVS fixed
----------------------------
Jacques Vidrine (nectar) fixed several vulnerabilities in
CVS: CAN-2004-0414_, CAN-2004-0416_, CAN-2004-0417_, CAN-2004-0418_,
CAN-2004-0778_, and others. These fixes were also made on the 4.10, 4.9,
4.8, and 5.2 security branches.
.. _CAN-2004-0414:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0414
.. _CAN-2004-0416:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0416
.. _CAN-2004-0417:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0417
.. _CAN-2004-0418:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0418
.. _CAN-2004-0778:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0778
http://www.freebsd.org/cgi/mid.cgi?200409192226.i8JMQNKC096007
===============
Other bug fixes
===============
Andre Oppermann (andre) applied a patch from Max Laier (mlaier) to PF, the
OpenBSD packet filter. The patch fixes the rdr (redirect) command when IP
fast forwarding (net.inet.ip.fastforwarding) is enabled. This closes `PR
71652`_. The fix was also merged to 5.x.
.. _`PR 71652`: http://www.freebsd.org/cgi/query-pr.cgi?pr=71652
http://www.freebsd.org/cgi/mid.cgi?200409131701.i8DH1rds076962
Brooks Davis (brooks) fixed a problem with USB keyboards being ignored
during the initial system installation. This closes `PR 71443`_. This
change also means that if a PS/2 keyboard is not connected while the
system boots to the installer, one will not be detected if it is connected
later.
.. _`PR 71443`: http://www.freebsd.org/cgi/query-pr.cgi?pr=71443
http://www.freebsd.org/cgi/mid.cgi?200409140401.i8E41vU0001768
Tim J. Robbins (tjr) fixed a problem in uniq that was causing empty lines
not to be counted when the -c argument was used.
http://www.freebsd.org/cgi/mid.cgi?200409141201.i8EC1JkQ015084
Diomidis Spinellis (dds) fixed crontab so that it will recognize edits
that are done within a single clock tick. This can happen when using
crontab over a slow connection or with a script. This commit fixes `PR
22612`_.
.. _`PR 22612`: http://www.freebsd.org/cgi/query-pr.cgi?pr=22612
http://www.freebsd.org/cgi/mid.cgi?200409141901.i8EJ1JH6026822
Bill Paul (wpaul) committed a patch to the bge driver to suppress spurious
link change events. Link changes were being generated when ifconfig was
run on the interface, as well as when applications like dhclient and
pppoed were run.
FreeBSD cvs-src summary for 13/09/04 to 20/09/04
++++++++++++++++++++++++++++++++++++++++++++++++
This is a regular weekly summary of FreeBSD's cutting-edge development.
It is intended to help the FreeBSD community keep up with the fast-paced
work going on in FreeBSD-CURRENT by distilling the deluge of data from
the CVS mailing list into a (hopefully) easy-to-read newsletter. This
newsletter is marked up in reStructuredText_, so any odd punctuation
that you see is likely intended for the reST parser.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
You can get old summaries, and an HTML version of this one, at
http://www.xl0.org/FreeBSD/. Please send any comments to Mark Johnston
(mark at xl0.org).
If you would like to get the summary without subscribing to current@,
please send mail to freebsd-cvs-summary-subscribe@lists.enderunix.org.
Thanks to Omer Faruk Sen and EnderUNIX for hosting this list.
For Lukasz Dudek and Szymon Roczniak's Polish translations of these
summaries, which may lag the English ones slightly, please see
http://mocart.pinco.pl/FreeBSD/.
.. contents::
============
New features
============
Netgraph implementation of Netflow v5
-------------------------------------
Gleb Smirnoff (glebius) added an implementation of Netflow v5 as a
Netgraph node. Netflow is a specification that allows routers to export
network accounting data in "flow" format to remote monitoring tools
like ntop_, FlowScan_, or Cisco's FlowCollector_. This development was
supported by the ISPs Bestcom and Rinet.
.. _ntop: http://www.ntop.org/
.. _FlowScan: http://www.caida.org/tools/utilities/flowscan/
.. _FlowCollector: http://www.cisco.com/univercd/cc/td/doc/pcat/24.htm
http://www.freebsd.org/cgi/mid.cgi?200409162024.i8GKONv3048588
BIND upgraded to version 9.3.0rc4
---------------------------------
Tom Rhodes (trhodes) imported BIND_ version 9.3.0rc4. This is the first
import of the 9.x series of BIND, which represents a complete rewrite
since version 8.
.. _BIND: http://www.isc.org/sw/bind/
http://www.freebsd.org/cgi/mid.cgi?200409190130.i8J1UPZN059043
===============
Notable changes
===============
Rework of GEOM Vinum code
-------------------------
Lukas Ertl (le) heavily modified the GEOM Vinum code, for doing RAID
under GEOM, to change how I/O is handled. This should fix recent data
corruption problems and improve performance.
http://www.freebsd.org/cgi/mid.cgi?200409181344.i8IDiiOL033004
=================
Discussion topics
=================
Supporting (and testing) really big disks
-----------------------------------------
Pawel Jakub Dawidek (pjd) reworked the md code, which supports swap-backed
simulated disks, to be 64-bit clean, as well as cleaning up and touching
up the code. He noted, "This work is a part of 'BigDisk' project:
http://www.freebsd.org/projects/bigdisk". He also gave a couple of
examples::
ragepxe:root:~# df -h
Filesystem Size Used Avail Capacity
Mounted on
[...]
/dev/md0 15T 249M 14T 0% /mnt
/dev/md1 11T 128K 10T 0% /mnt2
ragepxe:root:~# mdconfig -l -u 0
md0 swap 15T
ragepxe:root:~# mdconfig -l -u 1
md1 vnode 11T
Giorgos Keramidas (keramida) replied, "I think most of us were already
jealous enough by seeing the first post :-P"
Pawel answered, "Jealous about what?:) You know, that you can setup 15TB
of md(4) swap-backed device on 512MB swap partition, right?"
Ceri Davies (ceri) wondered, "What happens if you try to fill them?"
Giorgos responded enigmatically, "The best thing that happens is that lots
of nasal demons start flying out of your nose. You don't really want to
know the worst ;-)"
Scott Long (scottl) also replied, saying, "The advantage of what Pawel
has done is that you can simulate very large disks for the purpose of
debugging other utlities [ . . . ]."
Ceri clarified, "Don't misunderstand me, I'm appreciative of that. My
question stemmed from curiosity alone[.]"
Scott replied, "No offense taken. See
http://www.freebsd.org/projects/bigdisk for more details."
Pawel also added to Scott's first reply, noting, "It is still impossible
to detach such big md(4) device if it is filled. System panics somewhere
in swap code, but I don't think it is important for our use."
Robert Watson (rwatson) responded to Ceri's initial question, explaining,
"Bewm."
Brian Fundakowski Feldman (green) followed up, "Doesn't that have mostly
to do with vm_page_grab() and sf_buf_alloc() and vm_pager_get_pages() not
having any sort of error handling?"
Robert elaborated, "There are probably numerous sorts of errors, not least
that once you fill your swap space, you're going to be out of swap, and
file systems are written to assume that disks don't run out of space after
declaring they have the space (i.e., that they really are disks)."
Poul-Henning Kamp (phk) replied to Robert's previous post, saying, "Uhm,
be aware that we are limited also to the size of individual swap disks.
31 bits of bytes/sectors/pages, not quite sure which."
Scott clarified, "32 bits of pages, unless there is somewhere in the chain
that the unsigned nature of the word is not being honored."
Pawel also replied, "Pages. It gives as maximum swap-backed device of
16TB. I tested it with ~15TB devices, so there is no signed/unsigned
problem (which will limit us to 8TB)."
Poul-Henning responded, "I belive there is a limit both for the total
swapspce (16TB sounds likely) but also a limit for each swapdevice as far
as I know."
Brooks Davis (brooks) asked, "Given my quick read of the sunlabel format,
there would be some value in 16TB (17 would be sufficent) virtual disks.
Can we do a RAID0 of these to get something that big?"
Scott explained, "The 16TB limit comes [ . . . ] from using 4k pages for
swap backing and only being about to address these pages using 32 bit
numbers. The important part of the I/O path have already been converted
(or are in the process of being converted) to use 64 bit numbers."
Pawel also answered Brooks, noting, "You mean setup raid0 with md(4)
device to get more than 16TB? Sure, at least with gstripe[.]" He gave an
example of a 1.9 petabyte simulated disk using 128 mds in RAID 0.
Brooks followed up, "Just what I was looking for. Very cool!"
http://www.freebsd.org/cgi/mid.cgi?200409162132.i8GLWDpF050910
=================
Committer changes
=================
Murray Stokely (murray) introduced Xin LI (delphij) as a new src
committer. He will be working on Chinese (Simplified) documentation,
importing bugfixes from DragonflyBSD, and continuing to fix bugs in both
the source code and documentation.
http://www.freebsd.org/cgi/mid.cgi?200409141919.i8EJJLcO027263
===================
Important bug fixes
===================
Vulnerabilities in CVS fixed
----------------------------
Jacques Vidrine (nectar) fixed several vulnerabilities in
CVS: CAN-2004-0414_, CAN-2004-0416_, CAN-2004-0417_, CAN-2004-0418_,
CAN-2004-0778_, and others. These fixes were also made on the 4.10, 4.9,
4.8, and 5.2 security branches.
.. _CAN-2004-0414:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0414
.. _CAN-2004-0416:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0416
.. _CAN-2004-0417:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0417
.. _CAN-2004-0418:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0418
.. _CAN-2004-0778:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CAN-2004-0778
http://www.freebsd.org/cgi/mid.cgi?200409192226.i8JMQNKC096007
===============
Other bug fixes
===============
Andre Oppermann (andre) applied a patch from Max Laier (mlaier) to PF, the
OpenBSD packet filter. The patch fixes the rdr (redirect) command when IP
fast forwarding (net.inet.ip.fastforwarding) is enabled. This closes `PR
71652`_. The fix was also merged to 5.x.
.. _`PR 71652`: http://www.freebsd.org/cgi/query-pr.cgi?pr=71652
http://www.freebsd.org/cgi/mid.cgi?200409131701.i8DH1rds076962
Brooks Davis (brooks) fixed a problem with USB keyboards being ignored
during the initial system installation. This closes `PR 71443`_. This
change also means that if a PS/2 keyboard is not connected while the
system boots to the installer, one will not be detected if it is connected
later.
.. _`PR 71443`: http://www.freebsd.org/cgi/query-pr.cgi?pr=71443
http://www.freebsd.org/cgi/mid.cgi?200409140401.i8E41vU0001768
Tim J. Robbins (tjr) fixed a problem in uniq that was causing empty lines
not to be counted when the -c argument was used.
http://www.freebsd.org/cgi/mid.cgi?200409141201.i8EC1JkQ015084
Diomidis Spinellis (dds) fixed crontab so that it will recognize edits
that are done within a single clock tick. This can happen when using
crontab over a slow connection or with a script. This commit fixes `PR
22612`_.
.. _`PR 22612`: http://www.freebsd.org/cgi/query-pr.cgi?pr=22612
http://www.freebsd.org/cgi/mid.cgi?200409141901.i8EJ1JH6026822
Bill Paul (wpaul) committed a patch to the bge driver to suppress spurious
link change events. Link changes were being generated when ifconfig was
run on the interface, as well as when applications like dhclient and
pppoed were run.