[Aries-III]

Hey,
Noticed that there are no posts in this forum in the past 30 days :-/... not sure if I'll get an answer, but I'm plain-out STUCK.

I have a (so far) working pf.conf, and I have all the altq stuff I thought I needed in there, and it seems to read well, but it isn't working correctly! The internet works fine, and there are no errors, but the queue's aren't wanting to work correctly other than the SYN/ACK prioritization queue (which is a good sign nonetheless).

So, I will post my pf.conf in the hopes that someone can tell me why the queues don't wanna work the way that I'm intending. Just hope that my intentions are understandable from the code :P
/edit Code removed due to an obvious screw-up I made!! Will fix and see if it works, then if not, I'll repost it :P

/edit again... ARGH nope didn't work --so here is my pf.conf :|

################################################## #########################
# OpenBSD PF Ruleset
#
# Network is 192.168.1.x
#
# Main computers:
# ---Main(Win2K): 192.168.1.16 (High Priority-Game box ;P)
# ---Main(*nix) : 192.168.1.17 (Medium Priority)
# ---WebServer : 192.168.1.15 (Lowest priority [in whole network])
#
# Note: I should probably install another NIC and put the server on it by
# itself. Many sites seem to say this.
#
# All other computers should be low priority (I have roommates that HOG)
#
################################################## ##########################

################################################## ########
# Macro Definitions:

loop= "lo0"
ext_if= "{ tun0 }"
int_if= "rl0"
int_net= "192.168.1.0/24"
unfiltered= "{ lo0, rl0 }"

#modulate state=keep state with high quality random ISNs
tcp_opts = "flags S/SA modulate state"
udp_opts = "keep state"

main_comp= "192.168.1.16"
unix_comp= "192.168.1.17"
server= "192.168.1.15"
# low_comps= "192.168.1.xx" How do I specify ip-ranges?

# Web Services offered by 192.168.1.15 (ports):
services= "{ 80, 443, 20:21, 25, 8888, 32006 }"

# /Macro Defs
#############################

################################################## ########
# Network Options:

set block-policy drop
#set loginterface $ext_if

#
#############################

################################################## ########
# Table Definitions:

# table containing all IP addresses assigned to the firewall
table <firewall> const { self }

# Networks that aren't assigned and/or aren't allowed to be routed on the internet
table <reserved> persist { \
0.0.0.0/7, 2.0.0.0/8, 5.0.0.0/8, 7.0.0.0/8, 10.0.0.0/8, 23.0.0.0/8, \
27.0.0.0/8, 31.0.0.0/8, 36.0.0.0/7, 39.0.0.0/8, 41.0.0.0/8, 42.0.0.0/8, \
49.0.0.0/8, 50.0.0.0/8, 58.0.0.0/7, 70.0.0.0/7, 72.0.0.0/5, 83.0.0.0/8, \
84.0.0.0/6, 88.0.0.0/5, 96.0.0.0/3, 169.254.0.0/16, 172.16.0.0/12, \
173.0.0.0/8, 174.0.0.0/7, 176.0.0.0/5, 184.0.0.0/6, 189.0.0.0/8, \
190.0.0.0/8, 192.0.2.0/24, 192.168.0.0/16, 197.0.0.0/8, 198.18.0.0/15, \
223.0.0.0/8, 224.0.0.0/3, 255.255.255.255 \
}

# Evil advertizing spam
table <doubleclick> persist { \
216.73.80.0/20, 204.253.104.0/24, 205.138.3.0/24, 208.184.29.0/24, \
206.65.183.0/24 \
}

table <mediaforce> persist { \
2.23.190.0/24, 65.247.105.0/24, 65.215.137.0/24, 208.251.137.0/24 \
}

table <main_comps> persist { 192.168.1.16, 192.168.1.17 }

table <them> persist { 192.168.1.0/24, ! $main_comp, ! $unix_comp, ! $server }

# /Table Defs
#############################

################################################## ########
# Packet-Scrubbing

# In-bound scrubbing
scrub in on $ext_if all \
no-df min-ttl 100 max-mss 1492 fragment reassemble

# Out-bound scrubbing
#
scrub out on $ext_if all \
max-mss 1492 fragment reassemble random-id no-df

# For NFS
scrub in on $int_if all no-df
scrub out on $int_if all no-df

# /Scrub
#############################

################################################## ########
# Queues - Priorities (AKA: Keeping me on top!!)

# Will use CBQ 'cause it seems to be the easiest. I need
# to use 3 queue's--one for me, one for the webserver,
# and one for the rest of the people on the network.
# Note: Using 'RED' alot since it seems to be the best
# option, from what I was reading (don't see why it
# is not used on all the configs I've seen)... maybe
# this isn't a good thing to use *ALL* the time???

# Incoming queue
altq on $int_if cbq bandwidth 2.8Mb queue { std_in, fast_in }
queue std_in bandwidth 90% cbq(default, borrow) { me_in, thm_in, srv_in }
queue me_in bandwidth 60% priority 3 cbq(red, borrow)
queue thm_in bandwidth 30% priority 2 cbq(red, borrow)
queue srv_in bandwidth 10% priority 1 cbq(red, borrow)
queue fast_in bandwidth 10% priority 5 cbq(red, borrow)

# Outgoing queue
altq on $ext_if cbq bandwidth 280Kb queue { std_out, fast_out }
queue std_out bandwidth 90% cbq(default, red, borrow) { me_out, thm_out, srv_out }
queue me_out bandwidth 60% priority 3 cbq(red, borrow)
queue thm_out bandwidth 30% priority 2 cbq(red, borrow)
queue srv_out bandwidth 10% priority 1 cbq(red, borrow)
queue fast_out bandwidth 10% priority 5 cbq(red, borrow)

# /Queue
#############################

################################################## ########
# NAT - "Port Forwarding"

# NAT
nat on $ext_if inet from $int_net to any -> $ext_if

# Port Redirection

# Webserver (This rule is probably WAY too loose!)
rdr on $ext_if proto { tcp, udp } from any to $ext_if port $services -> $server

# GnuTella on main computer.
rdr on $ext_if proto { tcp, udp } from any to $ext_if port 18006 -> $main_comp port 18006

# That freaky thing to let internal network get to server:
rdr on $int_if proto tcp from 192.168.1.0/24 to $ext_if port 80 -> \
127.0.0.1 port 5000

# /NAT
#############################

################################################## ########
# Packet Filtering "Security" and Queue Assignment

# Dont bug loopback/internal network:
pass quick on $unfiltered all

# Default policy
block in log quick from no-route to any
block in log on $ext_if

block return in quick log on $ext_if inet proto tcp from any to $ext_if port 113
block return-icmp(10) in quick log on $int_if to { <verisign>, <doubleclick>, <mediaforce> }
block in quick on $ext_if from { <verisign>, <doubleclick>, <mediaforce> }
block return-icmp(10) in log on $int_if to <reserved>
block in log on $ext_if from <reserved>

# I dont see me using inet6 yet...
#block all inet6

# Kill spoofing
antispoof quick for $int_if inet

# Allow SSH *ONLY* from my main box:
block return in quick on $int_if proto tcp from ! <main_comps> \
to $int_if port ssh flags S/SA

# Normal pass rules and Queue Assignment:

# Outgoing - Prioritize connection-creation:
pass out quick on $ext_if inet proto tcp from any to any flags S/SA keep state queue fast_out

# Pass rules for ME-queue
pass in on $ext_if inet proto { tcp, udp } from any to <main_comps> port 18006 flags S/SA keep state queue me_in
pass in on $ext_if inet proto icmp from any to <main_comps> icmp-type { echorep, echoreq, timex, unreach } queue me_in
pass in on $ext_if inet proto tcp from any to <main_comps> queue me_in
pass out on $ext_if inet proto { tcp, udp, icmp } from <main_comps> to any keep state queue me_out

# Pass rules for Server-queue
pass in on $ext_if inet proto tcp from any to any port $services flags S/SA keep state queue fast_in
pass in on $ext_if inet proto icmp from any to $server icmp-type { echorep, echoreq, timex, unreach } queue srv_in
pass in on $ext_if inet proto tcp from any to $server queue srv_in
pass out on $ext_if inet proto { tcp, udp, icmp } from $server to any keep state queue srv_out

# Pass rules for the rest of the idiots on the network
pass in on $ext_if inet proto icmp from any to <them> icmp-type { echorep, echoreq, timex, unreach } queue thm_in
pass in on $ext_if inet proto tcp from any to <them> queue thm_in
pass out on $ext_if inet proto { tcp, udp, icmp } from <them> to any keep state queue thm_out


# allow ssh connections in on the external interface as long as they're
# NOT destined for the firewall (i.e., they're destined for a machine on
# the local network). log the initial packet so that we can later tell
# who is trying to connect. use the tcp syn proxy to proxy the connection.
pass in log on $ext_if proto tcp from any to ! <firewall> \
port ssh flags S/SA synproxy state queue srv_in

# /Packet Filters
#############################




#######################---End of File---#######################



Here is the output of "pfctl -vvs queue"


queue root_rl0 bandwidth 2.80Mb priority 0 cbq( wrr root ) {std_in, fast_in}
[ pkts: 2236 bytes: 1704120 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue std_in bandwidth 2.52Mb cbq( red borrow default ) {me_in, thm_in, srv_in}
[ pkts: 2236 bytes: 1704120 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue me_in bandwidth 1.51Mb priority 3 cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue thm_in bandwidth 756Kb priority 2 cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue srv_in bandwidth 252Kb cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue fast_in bandwidth 280Kb priority 5 cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue root_tun0 bandwidth 280Kb priority 0 cbq( wrr root ) {std_out, fast_out}
[ pkts: 2232 bytes: 279509 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue std_out bandwidth 252Kb cbq( red borrow default ) {me_out, thm_out, srv_out}
[ pkts: 910 bytes: 54721 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue me_out bandwidth 151.20Kb priority 3 cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue thm_out bandwidth 75.60Kb priority 2 cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue srv_out bandwidth 25.20Kb cbq( red borrow )
[ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 0 suspends: 0 ]
queue fast_out bandwidth 28Kb priority 5 cbq( red borrow )
[ pkts: 1322 bytes: 224788 dropped pkts: 0 bytes: 0 ]
[ qlength: 0/ 50 borrows: 544 suspends: 0 ]


It seems that the main queues are working, but the child queues are not... and I can't figure out for the life of me why they aren't working.

[molotov]

Sorry, I don't know, but I read it (:

[Aries-III]

Well, least it was read :P The question would be if you understand what I'm trying to do :P

Searched around for 3 days on google, and there are lots of examples out there, but the ones I could understand were *really* old and outdated, so it was kinda hard to get much out of them. I went around in the pf.conf example files people had up here and there and tried to sort out th logic in those instead. Which has me on the internet, but without the priorities I wanted. :confused:

So, real question is if there are any good, up to date sites that aren't written in uber-geek or some crazy 1337-speak (saw a FEW of those). If you know of any... I'm all ears :biggrin:

[Kernel_Killer]

Have you tried any of the working examples people have posted? You might try repleacing your code with some, and trying to find the error in your pf.conf. We have a lot of posts that would most likely help you out, and more will show up to help answer your question.

Next time, you might want to try a different name for your post. I could see where that name might push some people away. Just some advice.

[elmore]

It's sorta late so I don;t have a whole lot of time to look at this atm but....
I'll look at it tomorrow.

In the meantime, I have an extensive example of cbq posted, it works very well. Look at it and compare it to yours, if you find a fix please post the solution so I won;t waste a bunch of time looking through your conf file trying to figure out something that you fixed. ;)

See here:

http://screamingelectron.org/~elmore/pf/pf.egress.cbq

[Vile]

I am not sure if this is your problem or not but I have read in several places that RED is not the wonder tool it sounds to be and has actually been recommended not to use it by, I think Henning Brauer, who is one of the PF developers and incidentally operates an ISP -- meaning I would generally trust his judgements in these kinds of matters. I took thought RED sounded like a no-brainer but have since opted not to use it. My altq runs well but I am not using cbq but priq instead. I think priq is easier to use but cbq is more flexible.

[Aries-III]

Well,
The main thing I'm wanting to do it prioritize both incoming an doutgoing traffic so that I have enough bandwidth both ways to do my thing, the roomies have enough to do thier thing, and the server box can still serve pages and junk.
My main gripe with the roommate is that he uses shareaza to download everything under the sun all at the same time... so he munches the hell out of the bandwidth, often creaming the upstream with the gnutella connections.
Another problem I've picked up on is that even when I tweaked around with the rules is that apparently shareaza is cheating the firewall... its making a shitload of syn/ack's to get the connections, which is making the fast_in queue kinda pointless.
Figure if I can just get the rules to work right as far as assigning th epackets to the right queue instead of stuffing them all in the default (root) queue, then I can make two 'fast_in' queues, and assign him his own :P

As far as I can figure it, the queues are decent, and the rules *should* put the traffic into the queue at the end of the rule, but it seems that the IPS aren't getting parsed right or something... for instance:

pass out quick on $ext_if proto tcp from <main_comps> to any flags S/SA queue ( me_out, fast_out )

To me, this rule means, "Pass the packet in through to roommates if it is coming from anywhere on the 'net to one of thier computers, and assign it to the 'them_in' queue"

Sadly, the firewall isn't even picking up on the rule whatsoever. Just to test it out, I unplugged all the computers from the switch other than this machine I'm on now and of course the firewall... if I put in a default rule to block everything "block all" (which I think is a good default, myself), the firewall blocks everything...period. None of the rules I wrote for 'me_in/out' are even firing at all :(

I've been toying around with it a tad bit more... at least one side of the queues seems to be working half of the time now... but still not how it should :/ -- Seems half the packets are beign stuffed in 'them_in'...

I'll see if I can find a place to stick a copy of my ruleset as I go so that I don't clog up this post :P

######## Put pf.conf in my journal page. Hope you guys can see it. Will tweak and put code/results in there.

[Strog]

One error I see is that you put your internal interface in the unfiltered table and then passed quick on the unfiltered table. Quick means no more processing on the internal interface. :wink:

I'll look some more when I get off work to see if I can find anything else in there that might need a tweak.

Personally I like priq at home. It's easy to prioritize the traffic without worrying about how to divide the bandwidth. I'd just put the bandwith hogs into the default queue and then bump up the priority on "legit" traffic. The bandwidth hogs can have all the bandwidth until traffic for the other machines comes along and bumps it.

[Aries-III]

Hrmmm... Might be worth a shot to use the priq queue then.

If the pass in on external interface line is the one I'm thinking about that has the S/SA flag, I think I yanked that off of one of the guides I was reading that said something about SYN/ACK (connection request) packets, so that if someone is taking a dump on the outbound traffic it would 'normalize' the traffic. Considering I yanked it from a site I think they had the 'quick' option on :P Actually if I remember correctly, the whole ruleset was 'quick', which is something I didn't go for.

I also am keeping updated version of my pf.conf file in my 'journal' area. The current version seems to *sometimes* put bandwidth in the queues, but not all the time :/

[Strog]

The problem is that rl0 is defined in 2 macros. The unfiltered macro gets a quick pass and that's why queues that should be binding to rl0 aren't working. You are probably only seeing queue traffic when it's against the external interface only.

You have to remember even though you can queue off either direction on an interface, ALTQ only is working on the outbound side of the interface. The means traffic coming in from the internet is being queued at the oubound side of the internal NIC and vice versa.

[Aries-III]

Ahhhhh, okay :P I'll see fi I can rewrite some of this stuff... Going to try to use priq as well.

I didn't even notice the rule with unfiltered!! So simple... probably why I overlooked it!! (gets back to scriptin the firewall, lookin like a sheeeeep)

[Aries-III]

Hey,
Got a new pf.conf up as well as the output from 'pfctl -vvs queue'.

I edited out the one rule you talked about, Strog. What I don't get is why its still working as before.... I the rule now reads:
###### Default filter policy
block in on $ext_if all tag block_all

# Dont bug loopback network:
pass quick on $loop all

# Don't bug internal network
pass on $int_if from $int_if:network to $int_if:network keep state tag int_net

# Kill spoofing
block drop in log quick on $ext_if from <reserved> to any
block drop out log quick on $ext_if from any to <reserved>

I also did some reading on priq, and decided against it. Still going for CBQ because even though I want a priority on the bandwidth, I also don't want to suck the roomies dry... want to garauntee them at least a *cough* somewhat decent connection :P

Main desire to queue the outgoing packets is due to the roomies un-ceasing gnutella p2p stuff. It eats up the outgoing bandwidth mainly due to the sheer amount of connections it is trying to establish... which no doubt are syn/ack's to boot...

Got the incoming queue to work for the most part, but the outgoin queue is still fubar. :mad:

Here is a link to my journal iwth th new pf.conf: http://www.screamingelectron.org/forum/journal.php?do=showjournal&j=14

[Aries-III]

(Edited cause I'm an idiot and had it backwards!!)

Hey, it seems that the network is more responsive with RED off... thanks for the input! Averaging about 20KB/s better on downloads, and the queues 'even out' a *whole* lot faster. With red it sometimes took 5 minutes for them to equalize, but without it seems to pan out at *most* around 1-2 minutes.

Just wish I could get it to queue the outbound traffic... still tweaking away at the rules, but maybe my logic and PF's aren't agreeing.... :confused:

I'm open to any punches thrown my way due to me being a complete dufus... Strog already caught me with one :biggrin:. Knowing my luck I keep missing some simple and stupid thing....

P.S. Oh, another thing I was wondering! I've noticed a LOT of guides use rules with interfaces set up with parenthesis around them... but when I try to use them, PF barfs and says there is a syntax error? <i.e. ($ext_if)> Also, "set loginterface xxx" pops a syntax error as well... maybe my install of pf is broke?? I'm using the 3.7 default install... haven't tried 'ports' yet. Maybe my version of pf is old, or is somehow fubar (bad ftp link, or something)?