Comparison of short-lived TCP streams performance with and without ECN

Performance comparison of short-lived TCP streams with and without ECN
for different TCP stack implementations

Goal: to estimate the gain introduced by ECN in case of short-lived TCP streams, typical in interactive traffic. The estimation takes into account also the type of TCP stack implementation, in particular if the TCP stack is SACK-capable or not.
Performance of the short-lived TCP session is evaluated as the average number of trnsactions succesfully completed in 1 sec.
Equipment:

chain of intermediate ecn-capable routers (Cisco 7500 and C7200), IOS: 12.1(4)E (experimental version) - only first router is ecn-capable in this test
pool of Linux hosts, v. 2.4.9, SACK-capable
pool of Solaris 2.7 hosts, non SACK-capable

Test description:

Tolopology
Parameters
- amount of data to be exchanged: [5 Kby, 10 Kby, 20 Kby]
- mark probability: [1/50, 1/5, 1/20]
- TCP stack of reference streams (background traffic is always non SACK-capable):
  - Linux ECN-capable and SACK-capable
  - Linux non ECN-capable and SACK-capable
  - Solaris non ECN-capable and non-SACK capable
Traffic profile
- 20 TCP iperf sessions (long-lived): background traffic, no ECN, no SACK
- 1 TCP reference stream generated by netperf run in request/response mode between Linux machined or Solaris
- test duration: 3 min

Summary:

great benefit in performance introduced by SACK, while minor improvement with ECN

Comments:

Test 1: Figure 1 compares the number of successfully completed TCP transactions - for an amount of data exchanged equal to 5 Kby - in three TCP stack configurations: without ECN and without SACK, without ECN and with SACK, with ECN and SACK.
The purple and green lines correspond to two scenarios that differ by the type of ECN support. In this case, we see that the improvement in number of transactions over time with ECN is considerable, while if ECN is not supported, the support of SACK does not imply any considerable change in performance. This is confirmed by what described in RFC 2884.

Figure 1: comparison of TCP performance (number of completed transactions over time) in three different TCP stack configurations. The amount of data exchanged in the three cases is constant and equal to 5 Kby.
Test 2 and 3: Test 2 and 3 are very similar to what described in Test 1, with the difference in amoutn of data exchanged. In test 2 for each transaction we exchange 10 Kby of data, while in test 3 we exchange 20 kby per session.
Figure 2 and 3 illustrate the corresponding results, and show that in this case the gain introduced by ECN is negligible, while the greatest gain is introduced by SACK.
This is accoding to our intuition, in fact, in case of short-lived transactions like in Test 1, the probability of having more than 1 packet loss is very low, and consequently the benefit of having selective retransmission is very low, on the other hand within a longer sequence of packets (test 2 and test 3), the probability of having a bursty loss is higher.

Figure 2: comparison of TCP performance (number of completed transactions over time) in three different TCP stack configurations. The amount of data exchanged in the three cases is constant and equal to 10 Kby.

Figure 3: comparison of TCP performance (number of completed transactions over time) in three different TCP stack configurations. The amount of data exchanged in the three cases is constant and equal to 20 Kby.

S.Alessandrini, T.Ferrari, 22 March 2002