Network rate limiting with TC

| 分类 Linux  | 标签 TC 

Linux上,我们可以对网络设备设置一定的队列规则(Queueing Disciplines),从而影响数据的发送行为,比如reschedule, delay或者drop。从而间接达到限速的目的。

Classless Queueing Disciplines


内核使用struct Qdisc来表示队列规则:

struct Qdisc {
	int 			(*enqueue)(struct sk_buff *skb, struct Qdisc *dev); ///skb -> queue
	struct sk_buff *	(*dequeue)(struct Qdisc *dev);
	unsigned int		flags;
	const struct Qdisc_ops	*ops;

struct netdev_queue {
 * read mostly part
	struct net_device	*dev;
	struct Qdisc		*qdisc;



# tc qdisc show dev eth1
qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

pfifo_fast是使用简单的FIFO策略来管理skb,同时,具有简单的优先级(不受用户态工具控制)。有3个链表用于管理skb,band 0band 1band 2。内核根据skb->priority计算skb的band值,然后将skb放入对应的链表。在发送时,只要band 0有packet,就不会处理band 1,依然次类推。

struct Qdisc_ops pfifo_fast_ops __read_mostly = {
	.id		=	"pfifo_fast",
	.priv_size	=	sizeof(struct pfifo_fast_priv),
	.enqueue	=	pfifo_fast_enqueue,
	.dequeue	=	pfifo_fast_dequeue,
	.peek		=	pfifo_fast_peek,
	.init		=	pfifo_fast_init,
	.reset		=	pfifo_fast_reset,
	.dump		=	pfifo_fast_dump,
	.owner		=	THIS_MODULE,


 * Private data for a pfifo_fast scheduler containing:
 * 	- queues for the three band
 * 	- bitmap indicating which of the bands contain skbs
struct pfifo_fast_priv {
	u32 bitmap;
	struct sk_buff_head q[PFIFO_FAST_BANDS]; ///3个skb list

Token Bucket Filter(tbf)


TBF is very precise, network- and processor friendly. It should be your first choice if you simply want to slow an interface down!


The TBF implementation consists of a buffer (bucket), constantly filled by some virtual pieces of

information called tokens, at a specific rate (token rate). The most important parameter of the

bucket is its size, that is the number of tokens it can store.

Each arriving token collects one incoming data packet from the data queue and is then deleted from the bucket.


## 将eth1的出口带宽限制为1024kbit/s
# tc qdisc add dev eth1 root tbf rate 1024kbit latency 50ms burst 800000

# iperf -c 10.x.x.x -t 20
Client connecting to 10.x.x.x, TCP port 5001
TCP window size: 21.9 KByte (default)
[  3] local 10.y.y.y port 39434 connected with 10.x.x.x port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-21.0 sec  3.00 MBytes  1.20 Mbits/sec

# tc -s -d qdisc ls dev eth1 
qdisc tbf 8006: root refcnt 2 rate 1024Kbit burst 800000b/1 mpu 0b lat 50.0ms 
 Sent 10607570 bytes 12342 pkt (dropped 847, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  • 几个参数:

(1) limit or latency

Limit is the number of bytes that can be queued waiting for tokens to become available. You can

also specify this the other way around by setting the latency parameter, which specifies the

maximum amount of time a packet can sit in the TBF. The latter calculation takes into account the

size of the bucket, the rate and possibly the peakrate (if set).


static int tbf_parse_opt(struct qdisc_util *qu, int argc, char **argv, struct nlmsghdr *n)
	if (opt.limit == 0) {
		///latency时间内能够传输的大小 + buffer的大小,opt.rate.rate是转化后的以byte单位
		double lim = opt.rate.rate*(double)latency/TIME_UNITS_PER_SEC + buffer;
		if (opt.peakrate.rate) {
			double lim2 = opt.peakrate.rate*(double)latency/TIME_UNITS_PER_SEC + mtu;
			if (lim2 < lim)
				lim = lim2;
		opt.limit = lim;

(2) burst/buffer/maxburst

Size of the bucket, in bytes. This is the maximum amount of bytes that tokens can be available for

instantaneously. In general, larger shaping rates require a larger buffer. For 10mbit/s on Intel,

you need at least 10kbyte buffer if you want to reach your configured rate!

If your buffer is too small, packets may be dropped because more tokens arrive per timer tick than fit in your bucket.

buffer的大小,token的最大瞬时值。如果一个packet的大小超过这个值,就会被丢掉。必须 >= MTU/B。

(3) rate


static struct Qdisc_ops tbf_qdisc_ops __read_mostly = {
	.next		=	NULL,
	.cl_ops		=	&tbf_class_ops,
	.id		=	"tbf",
	.priv_size	=	sizeof(struct tbf_sched_data),
	.enqueue	=	tbf_enqueue,
	.dequeue	=	tbf_dequeue,
	.peek		=	qdisc_peek_dequeued,
	.drop		=	tbf_drop,
	.init		=	tbf_init,
	.reset		=	tbf_reset,
	.destroy	=	tbf_destroy,
	.change		=	tbf_change,
	.dump		=	tbf_dump,
	.owner		=	THIS_MODULE,

Stochastic Fairness Queueing(SFQ)


# tc qdisc add dev ppp0 root sfq perturb 10


上一篇     下一篇