[ECOS] Re: Dual Ethernet interfaces on i386/pc?

["Stephen Morgan" <smorgan at almaden dot ibm dot com>]

Well, I'm going to answer my own question.

In <devs/eth/intel/i82559/current/include/i82559_info.h> 
MAX_RX_DESCRIPTORS and MAX_TX_DESCRIPTORS are set to 128 each, for each 
i82559 (Ethernet) PCI adapter.

In <devs/eth/intel/i82559/current/src/if_82559.c>, memory is allocated in 
PCI space for that many receive buffers, each of maximum receive packet 
size, or 1,536 bytes, and that many receive frame descriptors (RFDs), each 
of 16 bytes (in procedure InitRxRing).  Moreover, space is also allocated 
for the transmit control blocks (TCBs), each of 16 bytes, and that many 
transmit buffers, each of maximum transmit packet size, or 1,536 bytes (in 
procedure InitTxRing).

The total number of bytes of PCI space allocated for send and receive 
buffers and control blocks is thus 794,264.  In 
<hal/i386/pc/current/include/pkgconf/mlt_1386_pc_*.h>, the total size of 
the PCI window is 0x64000, or 409,600 bytes.  Thus, adding a second 
Ethernet interface causes the receive and transmit buffers and control 
blocks to extend beyond the end of the PCI window when the default number 
of receive and transmit packets is used.

A reasonable solution would be to change the default maximum number of 
receive and transmit descriptors from 128 to 64.  When we did so, the 
problem we had been seeing immediately went away.  A more general solution 
would be to divide the maximum number of receive and transmit descriptors 
by the number of 82559 interfaces.

(This problem, its explanation, and its solution were all due to my 
colleague, Lance Russell.)

Stephen P. Morgan

Internal: Stephen Morgan/Almaden/IBM@IBMUS
Business: smorgan@almaden.ibm.com
Personal: stephen_p_morgan@hotmail.com
Lifetime: smorgan@alum.mit.edu