[clug] traffic shaping help

[Alex Satrapa]

Before reading any further, please consume the skeptic's requisite 
grain of salt. I'm about to spew forth a screed based on assumptions 
and half-knowledge (the half that makes me dangerous but not useful). 
So, proceed with caution.

Kim Holburn wrote:
> I have a program that can tell me if an IP number has used up their 
> daily quota of download bytes.  I want to throttle the bandwidth seen 
> by that IP for a certain period, so they are still connected and can 
> access the internet but their bandwidth is severely limited for that 
> period.

Throttling can only really be done from the end that's sending the 
data, you can only 'police' (ie: drop overflow) on ingress. By the time 
it's come over to your end, it's already been sent. However, you can 
have a little control at your end and drop packets from connections to 
simulate a lower-speed router being congested. This action is called 
"policing" - you're "policing" the flow of traffic by making the excess 
disappear.

The way I have concocted to work around this was to set up an IMQ 
device (InterMediate Queue), route all incoming traffic through that 
device, and set up a bunch of HTB qdiscs to rate limit the traffic. The 
IMQ is a virtual device much like the loopback interface. Except that 
it's different. I'm sure others will be more than happy to bring me up 
to scratch on "what is an IMQ device" :)

Using the IMQ device allows you to *shape* traffic on egress through it 
(remember, you can only *police* on ingress). Thus for any single 
connection (or "flow", I think is the routing terminology) you'd have 
packets flowing thus:

   Internet ---> pppX ---> imq0 ---> ethX ---> client IP

I'm not sure, but I think it would be possible to police on ingress 
into the imq, but why would you, when you've already policed traffic 
rates on ingress through the "physical" interface?

The idea with my imagined IMQ setup being that you can set up the HTBs 
to let packets through at a particular "share" of the total bandwidth, 
with a fixed "ceiling". The HTBs would be pre-allocated for each 
machine at "normal" and "shaped" speeds. The classifiers would dump 
stuff into the HTBs as required.  When a particular machine has 
exceeded it's "quota", the classifiers get modified to classify packets 
destined for that machine into the "shaped" HTB instead of the "normal" 
bucket. I imagine it would be something like:

Alex's Imaginary IMQ Setup (only working from the LARTC[1] notes):
> tc qdisc add dev imq0 root handle 1: htb default 10
>
> tc class add dev imq0 parent 1: classid 1:1 htb rate 256kbit
>
> tc class add dev imq0 parent 1:1 classid 1:10 htb rate 64kbit ceil 
> 200kbit
> tc class add dev imq0 parent 1:1 classid 1:20 htb rate 64kbit ceil 
> 200kbit
> tc class add dev imq0 parent 1:1 classid 1:30 htb rate 20kbit ceil 
> 80kbit
> tc class add dev imq0 parent 1:1 classid 1:40 htb rate 20kbit ceil 
> 80kbit
>
> tc qdisc add dev imq0 parent 1:10 handle 10: sfq
> tc qdisc add dev imq0 parent 1:20 handle 20: sfq
> tc qdisc add dev imq0 parent 1:30 handle 30: sfq
> tc qdisc add dev imq0 parent 1:40 handle 40: sfq
>
> tc filter add dev imq0 parent 1:1 protocol ip prio 1 u32 match \
>                 ip dst <mum and dad's machine> flowid 1:10
> tc filter add dev imq0 parent 1:1 protocol ip prio 1 u32 match \
>                 ip dst <leech's machine> flowid 1:20
>
> iptables -t mangle -A PREROUTING -i ppp0 -j IMQ --todev 0
> ip link set imq0 up

Then when someone has exceeded their quota, delete the repective filter 
and replace it with one that redirects to 1:[24]0 instead (or, rather, 
add the new filter then delete the old one so you don't have stray 
traffic wandering into the wrong queue). I haven't yet read the notes 
enough to understand how you go about dropping packets that have 
exceeded the allowed bandwidth in one of those HTB queues.

At this stage I haven't really read up on imq yet, so I don't know 
exactly how stuff gets routed in and out of it. IIRC, IMQ is one of the 
things that's possible thanks to Linux's amazing advanced routing 
capabilities.

For a tutorial about HTB, try the HTB page: 
http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm

What I don't know yet is whether a ECN-enabled Linux router will start 
marking the appropriate ECN bits and under what conditions it would do 
so. I also don't know whether a Linux router would end up emitting 
source quench at all or under what conditions.  I would imagine that a 
graceful progression would be ECN as queues start to fill up, source 
quench when queues are almost full, and dropping packets only when 
there is no option.

Regards
Alex
[1] LARTC: http://www.lartc.org/ - the Linux Advanced Routing and 
Traffic Control site.

ALL programs are poems, it's just that not all programmers are poets.
   -- Jonathan Guthrie