RE: [ECOS] ethernet performance <TCPIP guru question>

["David Webster" <dwebster at lanergy dot com>]

Hi,
I've also started looking at the network performance of our eCos based
product recently using ttcp to provide performance figures. I get the
following figures:
	Ttcp Client:	2.1 Mbps (ie eCos mainly tx'ing data)
	Ttcp Server:	4.0 Mbps (ie eCos mainly rx'ing data)

The inconsistency and some of the poor performance can be put down to
our Ethernet hardware interface - we're using a Realtek 8019 which we
have to access thru a slow 16bit fifo.

But there are others areas of the network stack that seem to introduce
inefficiencies, namely the number of memcpy's that occur in the stack.
I've noticed the following happen when I perform a socket write with an
8K buffer:

	sosend() - allocates 4 2K mbuf's and calls uiomove() to memcpy()
the data from the buffer supplied in write to the mbuf. 

	Each of these 2K mbufs is passed to tcp_output(). Tcp_output()
creates 2 TCP segments for this data by allocating 2 mbufs and calling
m_copydata() to memcpy() the data from the original mbuf into the mbuf
to be used by the TCP segment. This results in two Ethernet frames, one
with 1448 and the other with 600 bytes of data, for each of the 2K
mbufs.

So there are two memcpy's within the TCP stack.
Has anybody looked at providing a socket write function that actually
provides the data already in an mbuf and bypasses uiomove()?
Equally providing a function that performed the memcpy in addition to
calculating the checksum might also provide an efficiency improvement.

There's a similar number of memcpy's on a socket read - but I've not
really looked at this path - I image that a read that could bypass
uiomove and pass the mbuf with the received data would also be more
efficient.

I've also noticed that by default the tcp_do_rfc1323 flag (in
tcp_subr.c) is set. This results in TCP using the Window Scale and
TimeStamp options. The timestamp option in particular is sent in every
segment and "wastes" 12 bytes of each Ethernet frame and adds additional
overhead?. My understanding of rfc1323 is that the timestamp option only
really helps over Long Fat Pipe connections and I'm not sure it's really
necessary on 100 Mbps Ethernet - definitely not on 10 mbps. So disabling
that might help a (tiddly) bit.

I don't have any answers to these issues - but I think I've highlighted
some of the performance bottlenecks that need to addressed if/when more
performance is needed.


	Dave Webster 




-----Original Message-----
From: ecos-discuss-owner@sources.redhat.com
[mailto:ecos-discuss-owner@sources.redhat.com] On Behalf Of Stephen
Polkowski
Sent: 25 April 2002 01:35
To: eocs
Subject: [ECOS] ethernet performance <TCPIP guru question>

Hello,

	I've been working on a simple ftp server for ECOS.  I have it up
and working on the intel pc using the 82559 ethernet card.  I'm also 
using redboot to load my application over the ethernet.

	When I ftp a file to my server application I seeing some really
slow ethernet speeds.  I have a 100Mb ethernet running on a 2Ghz Pentium

  IV.  So, my system should be fast enough to handle the data.  I'm 
getting about 65k bytes a second transfer rate.  I should be get getting

something close to 10M bytes a second.  What's the problem?

         I did a tcpdump from my Linux machine.  I've noticed two
things.
First, the TCP window size in ECOS isn't sliding during the transfer. 
This might be confusing the sender, I'm not sure.  Also, I see a burst 
of data packets and ACKS from ECOS which appear good.  Then all of a 
sudden, ECOS starts to ACK to the same packet over and over.  After a
few more acks, the sequence resumes.  Is this normal?  Does this explain

the slow network performance?

Here's the dump of the burst sequence running into the ACK sequence.

Thanks.

1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1507369:1508817(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1508817:1510265(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1510265:1511713(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1511713:1513161(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: P 1513161:1514609(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1514609:1516057(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1516057:1517505(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1517505:1518953(1448) ack 1 win 5792
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1518953:1520401(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1520401:1521849(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1521849:1523297(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.2.55961 > 1.1.1.3.ftp-data: P 1513161:1514609(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.2.55961 > 1.1.1.3.ftp-data: . 1523297:1524745(1448) ack 1 win 5792
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1513161 win 18824
1.1.1.3.ftp-data > 1.1.1.2.55961: . ack 1523297 win 8688

-- 
Stephen Polkowski
Centaur Technology
Austin, TX
(512) 418-5730


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss