June 28, 2007

Overview

The following changes are introduced in this release:

Items Specific to LightParser2:

• Support for receiving GIGE, POS, LAPS and SFPDP packets.
• Completed implementation of 8B10B mini-packet processing for optical GIGE.
• Modified page table, channel descriptors and DMA sizes.
• Developmental support for SDH parsing.

Items Specific to LightParser:

• Support for receiving HDLC message channels.

Items Specific to DPL3:

• New API function to determine expansion FPGA type.

General Items:

• Implemented exclusive access to EEROM and other serial bus resources.
• Added wrap-around channel buffer mapping.
• Modified criteria for channel buffer overflow and underflow detection.
• Updates and fixes for the device drivers.
• Updates to the API library functions.
• Modified utility programs.
• Modified diagnostic programs.
• New and modified demonstration programs.
• Component versions for this release.
• Known problems.

Note:

This release includes new FPGA configurations for LightParser2 boards. Run the lp2_flashup program to update the appropriate FPGA configurations.

Support for Receiving GIGE, POS, LAPS and SFPDP packets

A new FPGA configuration type and software for LightParser2 boards provide packet processing on the board. The current design supports receiving the following types of packets:
- Ethernet packets extracted from optical and electrical GIGE signals
- POS and LAPS packets extracted from optical SDH (STM-4 and STM-16)
- Serial-FPDP packets from optical SFPDP signals

The FPGA configuration type is called lp2-packet. New boards will begin shipping with this FPGA configuration included in the MMC. To add it to existing MMCs, run the program:

    lp2_flashup -N packet dplt0

To make the packet configuration the default FPGA, run the program:

    lp2_init -D packet dplt0

Packet streams can be received by the host from each of the four ports (optical ports A and B and electrical Ethernet ports A and B). The board can also act as a media converter to route packets from incoming ports to outgoing ports although software support for this feature is not yet completed.

Host applications can receive packets using a new set of API functions with a new type of channel handle named, CAC_PACKET_CHANNEL. Note, this is similar to but not the same as the channel handle used to reading 8B10B mini-packets. The API functions for reading packet channels include:

cac_packet_chanopen		open a packet channel, returning a pointer to a newly created CAC_PACKET_CHANNEL handle.
cac_packet_chanclose		close the packet channel and relinquish the associated resources.
cac_packet_chanenable		enable the packet channel to begin receiving data.
cac_packet_sync		look for and sync to packet headers prior to reading packets from the channel.
cac_packet_read		read a packet from a packet channel storing the data and header information in buffers provided by the caller.
cac_packet_next		locate next packet a packet channel stream buffer storing pointers to the header and payload.
cac_packet_chanflush		request a flush of any packet data waiting for delivery or notification of availability.
cac_autoflush_interval		set the interval for automatic packet flushing done by the hardware.

A demonstration program named lp2_packetrec is provided as an example of using the various packet reading capabilities. It is a functional program that may be used to test packet processing systems.

LightParser2: Completed Implementation of 8B10B Mini-packet Processing for Optical GIGE

Support for simultaneously receiving GIGE 8B10B "mini packets" from both optical ports is added to the LightParser2 STD FPGA, library functions and utility programs. The ability to suppress the inclusion of Ethernet IDLE packets from the decoded 8b10b mini-packets is added which helps reduce system load during reception of small and sparse packets.

A new function, cac_8b10b_modes, is added to the library to control and select the various operational modes of the 8b10b decoders and mini-packet encoders. The modes include enabling 8B10B decoding, idle packet suppression, and enabling trickle packets.

Several new functions and a new channel type are added to the API for receiving GIGE mini-packets and converting them into Ethernet packets. A new data type structure, named CAC_MINIPKT_CHANNEL, is introduced. It contains the underlying data channel for receiving the 8b10b data as well as various state and status information.

The new API functions include:

cac_gigeminipkt_chanopen		open an 8B10B mini-packet channel, returning a pointer to a newly created CAC_MINIPKT_CHANNEL handle.
cac_gigeminipkt_chanclose		close the 8B10B mini-packet channel and relinquish the associated resources.
cac_gigeminipkt_read		read an Ethernet packet from a 8B10B mini-packet channel storing the data and header information in buffers provided by the caller.

The packet processing (converting the 8B10B mini packets into Ethernet packet data) is performed by a separate library of functions, now included in the distribution and build of the LightParser2 ChanAPI library. This separate library is called libminipkt.a and it must be linked separately by applications that utilize the new "cac_gigeminipkt_" functions.

A typical compile command might look like:

    gcc -I/usr/cac/include -o lp2_gigerec lp2_gigerec.c -L/usr/cac/lib -llp2 -lminipkt

This is the command used to compile the GIGE packet receiving demo program, lp2_gige8b10b, included in the demo directory of the software distribution.

LightParser2: Modified Page Table, Channel Descriptors and DMA Sizes

The page table is modified to decrease the channel buffer page size to 4 Kbytes to avoid allocation problems when system DMA memory becomes fragmented. The number of page table entries is increased to 256K to allow for up to 1GB of total channel buffer space per board.

The location of the page table is moved from block RAM in the FPGA to the on-board SDRAM. Doing so allowed for the increased number of page table entries and freed up a significant amount of block RAM in the FPGA which is required for other purposes. Problems with SDRAM access encountered in previous versions have been resolved.

The channel descriptor allocation and mapping is modified to support the increased number of stream IDs that are possible for parsed E1 streams. A side-effect of the new allocation is that only one channel may be opened per stream.

The buffer size field in the channel descriptors is expanded to allow for the increased number of buffer pages required for large buffers with the smaller page size.

The DMA block size for level 1 streams (e.g. E1) is changed from 512 bytes to 128 bytes. This allows for more efficient use of the DMA controller when large numbers of level 1 streams are in use.

LightParser2: Developmental Support for SDH Parsing

The software in this release includes preliminary modifications to support development of SDH parsing capability for LightParser2 boards. The parser configuration and control functions are included in the API library and a preliminary parser configuration program is included.

However, the design of FPGA configurations and additional software changes supporting the capability are not yet completed. When available, the LightParser2 will be able to parse SDH signals and extract PDH streams such as E1 and E3. The parsing engine will initially support 4 STM-1 input signals. The SDH groomer is used to select the STM-1 (or STM-0) signals to be parsed, from the available incoming SDH signals from optical ports A and B.

LightParser: Support for Receiving HDLC Message Channels

A new Virtex FPGA configuration type and software for LightParser boards provide HDLC message processing. The current design supports receiving HDLC messages from up to 4 DS3 signals and/or up to 96 E1 signals. E1 HDLC messages can be extracted from any group of consecutively bonded time slots but only one HDLC extraction per E1 stream is supported.

The FPGA configuration type is called dplp-virtex-hdlc. To load the HDLC configuration into the board's flash memory for the Virtex FPGA run the program:

    dplp_flashup -V hdlc dplp0

Host applications receive HDLC messages using a new set of API functions with a new type of channel handle named, CAC_HDLC_CHANNEL.

cac_hdlc_chanopen		open an HDLC channel, returning a pointer to a newly created CAC_HDLC_CHANNEL handle.
cac_hdlc_chanclose		close the HDLC channel and relinquish the associated resources.
cac_hdlc_readmessage		read a message from the HDLC channel, enabling the channel if it is the first time reading since the channel was opened.
cac_hdlc_msgstatus		retrieve status and other information about the current or most recent message of an HDLC channel.
cac_hdlc_linkstatus		get link status (whether or not HDLC flags are detected) for an HDLC channel.

A demonstration program named dplp_hdlc is provided as an example of using the various HDLC message functions.

DPL3: New API Function to Determine Expansion FPGA Type

The function, cac_get_exp_spec_type is added to the API to retrieve the expansion FPGA type specified in the EEROM settings for a DPL3 board. This is primarily used in cases when then FPGA is not yet configured and there is no loaded FPGA configuration. For example, when the cacopen function initializes the FPGA type information before the board has been initialized.

Implemented Exclusive Access to EEROM and Other Serial Bus Resources

Access to serial bus resources on DPL3, LightParser and LightParser2 boards is now protected from corruption caused concurrent access to the serial bus interface. Functions is the chanAPI that access the serial resource interface are modified to obtain and release a virtual lock when accessing serial bus resources.

The resources affected by this are:

DPL3 boards		LightParser boards		LightParser2 boards
EEROM E3/DS3 LIU E3/DS3 Jitter Attenuator		EEROM optical SFP clock frequency synthesizers voltage & temperature monitor		EEROM optical SFPs and CDRs clock/frequency synthesizer DAC voltage & temperature monitor

The lock is implemented using a bit in the "Hardware Config Register" on the board. When the bit is set it means the serial bus is in use. If the lock is busy most functions wait for up to approximately 15 seconds before declaring a timeout in acquiring the lock. In case of a timeout waiting for the lock the serial resource access fails with errno set to CAC_ERR_SERIAL_LOCK.

Added Wrap-Around Channel Buffer Mapping

Mapping the channel buffer with a wrap-around is the ability to map the channel DMA buffer into an application's memory space with a portion of the beginning of the channel DMA buffer appeneded at the end of the mapping. It allows applications that read incoming channel data directly from the channel buffer to access "chunks" of data that start near the end of the buffer and continue past the end, back at the beginning of buffer. For example, the cac_chan_io function can return a pointer to an amount of data in the buffer that physically wraps around to the beginning of the memory buffer, but the application can access it contiguously. This is mostly useful when dealing with packet-oriented data.

This capability is added to the chanAPI with a new library function called cac_chan_mapbuf_wrap and corresponding changes in the device driver. When buffer wrapping is desired, this function should be used instead of the original cac_chan_mapbuf function. The new function has an additional argument to specify the amount of wrapped memory to include in the map. The specified amount will be rounded up to the nearest multiple of the system page size. Normally it it only necessary to include an amount corresponding to the largest data packet or chunk expected.

We should point out the difference between mapping the buffer with wrap-around and the chan_io_nowrap flag used by some of the channel I/O functions. The chan_io_nowrap flag is used to limit the amount of available data reported by the device driver to the end of the buffer. In the case where the channel buffer is mapped with wrap-around, this flag limits the reported amount to the end of the what is mapped. It does not prevent the driver from utilizing the wrapped portion of the mapping.

Modified Criteria for Channel Buffer OverFlow and UnderFlow Detection

The device drivers are modified to change the criteria for detecting and declaring a buffer overflow (for receive channels) or underflow (for transmit channels). The new detection method uses a guard-band equal to one latency amount of bytes. An underflow or overflow is now declared if the amount of data or space in the buffer exceeds the size of the buffer less the latency.

Having the guard-band minimizes two potential problems. The first is that continuous data streams from board could begin to overwrite data beyond the overflow point before the device driver is notified of the overflow. The second possible problem is that host applications are not necessarily informed of an overflow or underflow event until the next time they read or write data to the channel. So it is possible that an over flow could occur, overwriting data in the channel buffer while the application is processing it.

Note that it is possible for an application to arrange for notification of overflow or underflow via software signal using the cac_chan_signal function.

Updates and Fixes for the Device Drivers

In addition to changes for channel buffer mapping and overflow/underflow detection, the following modifications are made to the device drivers for DPL3, LightParser and LightParser2 boards:

The implementation of queue flush entries (or partial-latency queue entries) is fixed so that a queue flush entry will cause the release of a process waiting for data on the channel instead of the process continuing to wait for the full amount of data requested. This may result in calls cac_chan_read or cac_chan_io returning with less data read or available than was requested when applications use the queue flush mechanism. Currently, the LightParser2 packet FPGA configuration uses queue flush entries as part of it's packet flusing mechanism.

The error code set when opening a channel fails due to not being able to allocate a channel descriptor is changed from ENOSPC ("no space left on device") to EBUSY ("device or resource busy").

The minimum timeout enforced by the Linux device driver is reduced from 10 "jiffies" or clock ticks to 1. Depending on the system's clock tick rate, the new minimum timeout is between 10 and 16 milliseconds. The limit was, and still is, silently enforced and is meant to avoid processes spinning if their channel timeouts are set below the system's clock tick resolution. The Solaris device driver does not enforce it's own minimum timeout.

Support is added in the software configuration script and Makefile for the Linux device driver to compile the driver for 2.4 kernels on x86_64 (Intel x86 EMT64) architectures. Also, the instructions for applying the bigphysarea patch on Red Hat Enterprise Linux 3 (rhel3) are updated for this architecture.

The device drivers include code to check for changes in the board's config space registers that is primarily used to detect if a board might have been replaced or power-cycled in development system with a hot-swap board adapter. This code has been fixed to only compare the lower 4 bits of the PCI config space "command register" to avoid unnecessarily resetting information about the board stored by the device driver. Most significantly, this reduces the need for re-reading settings from the serial EEPROM device.

A new ioctl command is added to the device driver for channel resource operations. For both Linux and Solaris it may be used to trace (in the system log) the current usage of channel resources which include:

- the number of open channels,
- the channel and stream IDs,
- channel descriptor location,
- buffer size and page table usage
- the process ID for each channel

The implementation for the Linux device drive also allows the command to be used to release the channel resources and buffer memory - for emergency use only. The ioctl command macro is CAC_CHANRES_OP and it takes an integer value for the operation parameters which are explained in the header file, chan_ioctl.h.

Updates to the API Library Functions

In addition to the changes and enhancements to the API previously mentioned, the following additions and modifications are made to the API functions:

A new function, cac_rmw32_atomic_chan, is added to the library to provide a way to read, write and modify board control registers given an open CAC_CHANNEL handle. The channel handles do not include memory-mapped access for control registers or memory on the board so this function uses the ioctl interface in the device driver to access them.

A new function for LightParser2, cac_freq_counter, provides an interface to the frequency counter in the LightParser2 FPGA. It is used, primarily by the lp2_cal program, to measure the frequencies of the clocks produced by the FracN devices.

The active or loaded FPGA configuration type code is now included in the CAC_HANDLE and CAC_CHANNEL handle structures for open board devices and channels. These were added to avoid having to access device registers every time it is necessary to determine the FPGA configuration type. The value stored is a code for the FPGA type and the meaning varies depending on the board type, described in the table below. The table, below, shows examples of the FPGA configuration type macros of the common type codes for each board.

Board Type		FPGA Type Code
DPL3 boards:		The L3 payload type configured in the expansion FPGA
		DPL3_L3_E3G751E1, DPL3_L3_DS3T1 or DPL3_L3_NONE
LightParser boards:		The Virtex FPGA configuration type
		DPLP_VIRT_STANDARD, DPLP_VIRT_ATM, or DPLP_VIRT_HDLC
LightParser2 boards:		The Main FPGA configuration type
		DPLP2_FPGA_STD, DPLP2_FPGA_PACKET or DPLP2_FPGA_PARSER

Theses FPGA type values are available to application programs by accessing the fpga_type member of the open resource handle. For board access handles:

     CAC_HANDLE *dev->fpga_type 

For channel handles:

     CAC_CHANNEL *chan->fpga_type 

Modified Utility Programs

Calibration of the default PWM setting for the VCXO is added to the lp2_cal program.

The lp2_init program is modified so that the -F option may be given an FPGA type name instead of an FPGA directory entry number. When a type name is specified, the first matching entry is selected. The program is also modified to load the default PWM setting stored in EEROM by the lp2_cal program.

The lp2_flashup and lp2_init programs now identify custom FPGA type names by appending "(c)" to the displayed name.

The lp2_portcfg program has some modified parameters for the -R option to support new channelization source choices (e.g. the packet processor and the parser). Also included is support for the electrical Ethernet ports to the degree that they can be selected as sources for packets. This program is also modified to refer to STM-1, STM-4 and STM-16 as signal types rather than signal rates, although the standard rates for these SDH signals are still assumed. "8B10B decoder packets" are now referred to as "8B10B mini packets".

The lp2_portcfg and lp2_portstat programs are modified to include reporting of the ports enabled when packet processing is the selected channelization source. These program no longer support the unused Rx and Tx polarity and Tx loopback MGT modes.

The lp2_sdhgroom program has a new "groomparser" command. It sets the groomer map for the parser which uses shifted output mappings to minimize the number of byte lanes required to feed the parser. The abbreviation for the new command is "gp". The fixed-pattern groomer command, "groompattern", has been renamed "groomfixed" with the abbreviation of "gf".

The dplp_streams program is modified to support HDLC and Raw DS3 configuration. It is also modified so that the payload type parameter for transmit streams uses the actual number of payload streams (64, 63 or 48) instead of arbitrary numbers.

The dplp_streamstat program is fixed to configure the Quad E3/DS3 port for DS3 mode when it configures the parser for a DS3 path from that port.

A bug is fixed in the dpl_flashup program for specifying main and expansion FPGA types.

Modified Diagnostic Programs

The new dplp_vtmon program replaces the dplp_dumpmon program. It is similar to (and build from the same source code as) the lp2_vtmon program. The dplp_dumpmon and dplp_dumptcxo programs are obsolete and no long included.

The _chantest programs are modified such that the default data path option is to not change any of the board's settings with respect to what streams are transfered to and from the host or how. This is equivalent to specifying the -p -1 option. Previously, the FPGA-loopback mode was set by default.

The _recv and _send programs have a new -u option to specify the number of bytes transfered between display updates.

Handling of SIGTERM and SIGINT (control-C) interrupts is added to the _recv programs to have them flush and remaining data captured from the channel to the output file.

The dplp_debug and lp2_debug programs have new commands, pr, pw, prmw, to read, write and read-modify-write parser configuration registers. Those registers may still be accessed for LightParser using the the rv and wv read and write virtex commands. However, those commands and the functions they use are not fully supported for LightParser2.

The _debug programs have a new -i option to disable handling of control-C interrupts. This may be required when using the programs as part of a shell script that might be interrupted.

The _alloctest programs have an added parameter to the "open channel" command to test mapping the channel buffer with a wrap-around amount.

New and Modified Demonstration Programs

The dplp_hdlc program is a new LightParser demo for reading messages from an HDLC channel.

The new lp2_packetrec program demonstrates configuring the new LightParser2 packet processing FPGA configuration and reading channels of various kinds of packets.

The lp2_gige8b10b program for LightParser2 demonstrates reading 8B10B mini-packets and processing them to extract Ethernet packets. This program was originally named lp2_gigerec as part of the developmental GIGE processing software for LightParser2.

The lp2_pktest LightParser2 program is obsolete and no longer included in the software distribution.

Component Versions for This Release

The Channelization API software release includes several components of software and hardware FPGA configurations. The "Support Version" for hardware components are the version numbers for the software support of the particular board or FPGA configuration type.

The API and device driver components have the same major and minor version numbers. Similarly, the "Hardware Support" versions have the same major and minor version numbers as the corresponding FPGA or FPGA type. However the update version number of each component may vary depending on the number of changes and iterations each component has gone through between general distribution releases.

The component versions for Software Release 0.9.0 are:

Software Components	Version
API Library and Programs	0.9.2
Solaris Device Driver	0.9.2
Linux Device Driver	0.9.2
Hardware Components	FPGA Version		Support Version
LightParser Spartan FPGA Configuration	0.8.1		0.8.1
LightParser Virtex FPGA Configurations
Standard SDH/PDH variation	0.7.0		0.7.0
ATM variation	0.7.3		0.7.3
HDLC variation	0.9.0		0.9.0
SDH/PDH Transmit variation	n/a		n/a
LightParser2 Virtex FPGA Configurations
Standard SDH variation	0.4.3		0.4.3
Packet Processing variation	0.4.4		0.4.4
LightParser2 Boot FPGA Configuration	0.0.1		0.0.1
DPL3 Main FPGA Configuration	0.7.3		0.7.3
DPL3 Expansion FPGA Configurations
E3 G751 E1 variation	0.7.3		0.7.3
DS3 T1 variation	0.7.3		0.7.3
E3 G832 Bulk variation	0.7.3		0.7.3
Generic variation	0.7.3		0.7.3

Known Problems

The following problems are known to exist in this release:
• LightParser:

  NDFs, announced and unannounced, are not properly handled in TU-12s. This problem is resolved in an experimental, pre-release FPGA configuration, version f.28, included in the distribution. To load this configuration run the dplp_flashup program as shown below:

              dplp_flashup -uv -xf dplp0
      

  Data corruption has been reported when recording raw, descrambled STM-1 signals that include AU3s.

  Using the clock extracted from the electrical (CMI) port in STM-1 mode causes some data corruption. Work-around is to use the local clock reference when receiving electrical STM-1.

  The ATM version of the Virtex is not able to frame DS3 streams from SDH streams. A work-around is to route the raw DS3 streams from SDH input through the Quad-T3 port, loop the signal back in and frame the DS3s from the Quad-T3 input.

  Detection of ATM client channel overflow (pointers to the bulk ATM buffer that are no longer valid) is not yet implemented in the ATM service of the device driver.

  E1 insertion for STM-1 transmit does not work when configured with the dplp_portcfg program. This problem was introduced in the 0.8.2 release but was not discovered until Decemeber, 2007. It is fixed as of release 0.9.2. As a work-around for software releases between 0.8.2 and 0.9.1, the dplp_debug program may used to adjust the configuration following the use of the dplp_portcfg program as shown in the example, below.

              dplp_portcfg -p0 -t67 -x5 dplp0
              echo "wv 1020 0 ; wv a9 1" | dplp_debug dplp0
      

• LightParser2:

  Handling of extended packet lengths in the packet processing FPGA is not yet complete. Continuation "chunks" for HDLC (POS and LAPS) packets larger than 3072 bytes are implemented but packets that are aborted after one or more chunks are not terminated properly. Jumbo Ethernet packets from the electrical or optical ports are not yet supported.

  Support for packet status and statistics is not yet implemented in the API.

• DPL3:

  ESF framing is not yet supported for transmitting DS1 streams multiplexed to the output DS3 signal.

  Receiving unframed E1 or DS1 de-multiplex from E3 or DS3 does not work. The received data is replaced by all-ones.

  Possible failure of framing logic for embedded E1 or DS1 streams after configuration of Expansion FPGA. See the discussion of "Initialization Problems" in the 0.7.0 release notes for details.

• Linux 2.6 Device Driver Problem / Support:

  In the Linux 2.6 kernel, unloading the device driver can sometimes cause a page fault in the the kernel's routine for unregistering the device driver object. Less likely is a similar problem when the kernel attempts to register the driver object. The problem is being investigated.

  The Linux device driver is not yet compatible with the version of the 2.6 Kernel included with Red Hat Enterprise 5.

• Solaris 10 Intel Support:

  The device driver for DPL3, LightParser and LightParser2 boards does not work on the Intel variant of Solaris 10. The problem involves allocating DMA memory for channel buffers.

  Solaris is currently supported on Intel platforms up through Solaris 9. Solaris 10 is supported on Sparc platforms.