to test the impact of rate limiting when applied to TCP traffic

• source: 192.168.73.2 (INFN), destination: 192.65.183.20 (CERN, Monarc
  test workstation)
• CAR configured at the ingress router (input direction)
• limited traffic: TCP, background traffic: UDP or TCP
• no WFQ configured in the network

TEST 1

• unlimited UDP traffic at 2 Mbps (200 pack/sec, 1000 by per packet)
• variable number of rate limited TCP sessions, number in the range [1, 20]
• rate limit of TCP traffic = 1 Mbps
• conform action: mark with precedence 5, exceed action: mark with
  precedence 3

REMARKS

• normal and exceeded burst sizes do not impact the throughput, no relevant
  improvement in throughput when the sizes increase
• rate limit does not provide bandwidth guarantees to the TCP streams:
  TCP aggregate throughput is around 120 Kbps (instead of 1 Mbps),
  independently of the number of TCP sessions. For 20 session it's about
  250 Kbps. Exceeding traffic (as reported by the router with command
  sh int rate_limit
  is almost null
• aggregate TCP traffic is the same after removing CAR, which means that
  TCP suffers from packet loss at the ingress interface and the congestion
  algorithm prevents TCP from grabbing more capacity.
• Conclusion: CAR prevents exceeding traffic from overcoming a given
  bandwidth threshold, but does not provide bandwidth guarantees to TCP
  streams. CAR is suitable to prevent best-effort traffic from getting
  more bandwidth than a given threshold. A problem in best-effort traffic
  characterization may arise in production, since CAR rate-limits traffic
  defined through access-lists. The right approach is the opposite: rate limiting UDP or, generally
  speaking best-effort traffic.

TEST 2

• rate limited UDP traffic, bandwidth = 100 Kbps
• unlimited TCP traffic

REMARKS

• TCP throughput ~ 1.32 Mbps
• effective rate limiting when applied to UDP
• what happens if the exceed action is different from drop (when applied to
  UDP traffic)?: probably TCP sessions are not protected from UDP
  NOTE: verify this

TEST 3

GOAL: how rate limited TCP traffic behaves if background traffic is
TCP (and not UDP as in TEST 1)?

• 1 rate limited TCP session, limit = 200 Kbps)
• conform action: mark with precedence 5, exceed action: drop
• 1 unlimited TCP session

REMARKS

• limited TCP throughput: 140 Kbps
• unlimited TCP throughput: 1.39 Mbps
• rate limiting is effective in this case because the rate limit is set
  to a relatively small value (small compared to the line rate), bandwidth
  distribution is *not* effective anymore for larger values (see
  the following test)

TEST 4

GOAL: verify bandwidth guarantees for TCP sessions when rate limits are
relatively large

• 1 rate limited TCP session, limit = 1296 Mbps (limits are multiples
  of 8 Kbps)

REMARKS

• rate limited TCP throughput : 0.69 Mbps
• unlimited TCP throughput: 0.94 Mbps
• the rate limited TCP throughput (the stream which should get most of the
  bandwidth) gets less than the unlimited throughput
• result confirmed by several runs
• CAR does not provide "committed" rates, just limits. CAR for example
  does not prevent TCP best-effort streams from impacting "rate limited" TCP
  streams, in fact, in this case the rate limited throughput is much less
  than the target. In this case the rate limited stream is not able to get
  the maximum. This is true only if the rate limit is small compared to the
  line rate (see previous test).
• Conclusion: in some cases CAR at the ingress of a domain and WFQ in
  the core do not suffice to implement VLL (they do not suffice in the
  the worst case, when background traffic is only UDP), a different
  scheme of feature deployment (WFQ, WRED, CAR) must be worked out.

TEST 5

GOAL: verify traffic isolation when the whole line capacity is
"sliced" into serveral rate limits

• 1 TCP stream (A): rate limit = 1 Mbps
• 1 TCP stream (B): rate limit = 292 Kbps
• 1 TCP stream (C): rate limit = 496 Kbps
• Exceed action: drop (in each case)
• deployment of modified netperf session which deploys specific TCP ports:
  mod_netperf -r 60000 -p 30000
  mod_netserver -r 60000 -p 30000
  -r 60000: TCP port for data exchange (in the packets)
  -p 30000: TCP port for comunication between netperf and netserver
• router configuration example:
  !
  interface FastEthernet0/0
  description test LAN
  ip address 192.168.72.1 255.255.255.0 secondary
  ip address 192.168.73.1 255.255.255.0
  no ip directed-broadcast rate-limit input access-group 167 1000000 8000 8000 conform-action transmit exceed-action drop
  rate-limit input access-group 168 296000 8000 8000 conform-action transmit exceed-action drop
  rate-limit input access-group 169 496000 8000 8000 conform-action transmit exceed-action drop
  full-duplex
  !
  access-list 167 permit tcp host 192.168.73.2 eq 60000 192.65.183.0 0.0.0.255 eq 60000
  access-list 167 deny ip any any
  access-list 168 permit tcp host 192.168.73.2 eq 60001 host 192.65.183.20 eq 60001
  access-list 168 deny ip any any
  access-list 169 permit tcp host 192.168.73.2 eq 60002 host 192.65.183.20 eq 60002
  access-list 169 deny ip any any

  qos#sh in faste0/0 rate-limit
  FastEthernet0/0 test LAN
  Input

  matches: access-group 167
  params: 1000000 bps, 8000 limit, 8000 extended limit
  conformed 32772 packets, 49403674 bytes; action: transmit
  exceeded 853 packets, 1289202 bytes; action: drop
  last packet: 4140468ms ago, current burst: 0 bytes
  last cleared 01:16:36 ago, conformed 85000 bps, exceeded 2000 bps

  matches: access-group 168
  params: 296000 bps, 8000 limit, 8000 extended limit
  conformed 8672 packets, 13071954 bytes; action: transmit
  exceeded 898 packets, 1359572 bytes; action: drop
  last packet: 4139236ms ago, current burst: 5124 bytes
  last cleared 01:16:36 ago, conformed 22000 bps, exceeded 2000 bps

  matches: access-group 169
  params: 496000 bps, 8000 limit, 8000 extended limit
  conformed 16705 packets, 25181276 bytes; action: transmit
  exceeded 1049 packets, 1587066 bytes; action: drop
  last packet: 4139152ms ago, current burst: 226 bytes
  last cleared 01:16:36 ago, conformed 43000 bps, exceeded 2000 bps

REMARKS

• throughput stream A: 0.83 Mbps, conform: 865 Kbps, exceed: 22 Kbps
• throughput stream B: 0.21 Mbps, conform: 228 Kbps, exceed: 23 Kbps
• throughput stream C: 0.42 Mbps, conform: 440 Kbps, exceed: 27 Kbps
• Conclusion: proper bandwidth guarantees are achieved if the *whole* bandwidth
  is distributed to different traffic classes. This is a sort of traffic engineering which is difficult
  to be deployed in production. In fact: how can we specify best-effort traffic
  through access lists? (best-effort must be rate limited according to this
  phylosopy and CAR requires access lists to define traffic filters)