A5E-SFP-Fabric-Example¶

Overview¶

This design enables the Intel Ethernet GTS IP in the sfp_fabric_subsys and brings a single 10G serial lane out to the SFP1 cage. The Intel Ethernet GTS example source files used by this design come from $QUARTUS_ROOTDIR/../ip/altera/ethernet/intel_eth_gts/example_project/common. In the generated system, the GTS IP is instantiated as intel_eth_gts_0 and the example packet source/sink is instantiated as intel_eth_gts_packet_client_0. The serial TX/RX pins from intel_eth_gts_0 connect directly to SFP1_TX_P/N and SFP1_RX_P/N.

This example is provided for the MitySOM and MitySOM-Mini development kits. It is not part of the MitySBC reference designs, which use a QSFP cage instead.

SFP Fabric Subsystem¶

The data path is intentionally set up for external loopback. Packets return only when the SFP path is looped externally through the module/media. The packet client is the traffic source and sink for this example: it generates packets on the TX side, hands them to the GTS IP, and then counts the packets that return on the RX side after the external loopback path.

+-------------------------------+
| intel_eth_gts_packet_client_0 |
|  packet generator             |
|  packet counters              |
+---------------+---------------+
                ^
                |
                | Avalon-ST TX/RX
                v
        +-------+--------+
        | intel_eth_gts_0|
        +-------+--------+
                ^
                |
                | serial TX/RX
                v
           +----+----+
           | SFP1    |
           | cage    |
           +----+----+
                |  ^
                |  |
                +--+
         external loopback

What This Project Offers¶

This project provides:

• A working Intel Ethernet GTS instantiation with a single 10G serial lane routed to the SFP1 cage.
• A packet generator and packet counters in fabric, so the link can be exercised with no host software beyond memtool.
• Status PIOs covering clocking, reset acknowledge, link state, system PLL lock and reference-clock buffer protection.
• SFP module sideband control, through fabric PIOs on the MitySOM-Mini and over I2C on the MitySOM.
• A starting point for attaching your own Avalon-ST traffic source in place of the packet client.

Building the Example¶

Refer to Building_fpga_261pro for building the FPGA design. Navigate into the mitysom-a5e[-mini]-ref-sfp-fabric example project before compiling the design.

• Ensure to flash the resulting a5e.hps.jic onto the hardware
• Ensure to replace the a5e.core.rbf on the SD card

Using the Example¶

The packet client control register is accessed over the lightweight bridge at address 0x30000000.

Register Map¶

The packet counters are 64-bit values split into low/high 32-bit words.

Absolute Address	Offset	Register	Access	Description
0x30000000	0x0000	Packet client control	RW	Bit 0 = start packet generation, bit 2 = continuous mode, bit 4 = select RX loopback path into TX when set, bit 6 = snapshot counters, bit 7 = clear shadow counters, bit 8 = clear TX/RX counters (self-clearing).
0x30000004	0x0004	Test loop count	RW	Packet-generator loop count.
0x30000008	0x0008	ROM start/end address	RW	Low field = ROM start address, upper field = ROM end address.
0x3000000C	0x000C	Packet client status	R	Bit 0 = loopback FIFO write-full error, bit 1 = loopback FIFO read-empty error.
0x30000010	0x0010	Latency count	RW	Read returns the measured latency count in bits [7:0]; bit 31 is used to trigger a latency measurement.
0x30000014	0x0014	Custom destination address low	RW	Lower 32 bits of the packet-generator destination-address field.
0x30000018	0x0018	Custom destination address high	RW	Upper destination-address bits in [16:0].
0x3000001C	0x001C	Packet gap	RW	Packet-gap setting for the ROM-based packet generator.
0x30000020	0x0020	TX SOP count low	R	Low 32 bits of transmitted start-of-packet count.
0x30000024	0x0024	TX SOP count high	R	High 32 bits of transmitted start-of-packet count.
0x30000028	0x0028	TX EOP count low	R	Low 32 bits of transmitted end-of-packet count. One EOP corresponds to one transmitted packet.
0x3000002C	0x002C	TX EOP count high	R	High 32 bits of transmitted end-of-packet count.
0x30000030	0x0030	TX error count low	R	Low 32 bits of transmitted error count.
0x30000034	0x0034	TX error count high	R	High 32 bits of transmitted error count.
0x30000038	0x0038	RX SOP count low	R	Low 32 bits of received start-of-packet count.
0x3000003C	0x003C	RX SOP count high	R	High 32 bits of received start-of-packet count.
0x30000040	0x0040	RX EOP count low	R	Low 32 bits of received end-of-packet count. One EOP corresponds to one received packet.
0x30000044	0x0044	RX EOP count high	R	High 32 bits of received end-of-packet count.
0x30000048	0x0048	RX error count low	R	Low 32 bits of received error count.
0x3000004C	0x004C	RX error count high	R	High 32 bits of received error count.
0x30900000	0x00900000	Clock status PIO	R	See bit definitions below.
0x30900010	0x00900010	Reset status PIO	R	See bit definitions below.
0x30900020	0x00900020	Link status PIO	R/W	See bit definitions below.
0x30900030	0x00900030	System PLL lock PIO	R	See bit definitions below.
0x30900040	0x00900040	SFP input PIO	R	Only for the MitySOM-Mini. See bit definitions below.
0x30900050	0x00900050	SFP output PIO	RW	Only for the MitySOM-Mini. See bit definitions below.
0x30800000	0x00800000	Intel Ethernet GTS reconfiguration slave	RW	IP reconfiguration interface of intel_eth_gts_0.
0x20000500	n/a	GTS refclk fail status PIO	R	On the lightweight bridge outside this subsystem. See bit definitions below.
0x20000520	n/a	GTS refclk on PIO	W	10-bit register method for re-enabling a refclk buffer. See below.

Status PIO Bit Definitions¶

Clock status PIO (0x30900000)¶

Bit	Name	Description
0	o_cdr_lock	RX CDR lock status.
1	o_tx_pll_locked	TX PLL lock status.
2	o_tx_lanes_stable	TX lane-stability indication.
3	o_rx_pcs_ready	RX PCS ready indication.

Reset status PIO (0x30900010)¶

Bit	Name	Description
0	o_rst_ack_n	Active-low overall reset acknowledge from the GTS reset logic.
1	o_tx_rst_ack_n	Active-low TX reset acknowledge from the GTS reset logic.
2	o_rx_rst_ack_n	Active-low RX reset acknowledge from the GTS reset logic.

Link status PIO (0x30900020)¶

Bit	Name	Description
0	o_rx_block_lock	RX block lock indication.
1	o_local_fault_status	Local fault indication.
2	o_remote_fault_status	Remote fault indication.
3	i_stats_snapshot	Software-controlled snapshot bit driven from the PIO toward the GTS status interface.
4	o_rx_hi_ber	RX high-BER indication.
5	o_rx_pcs_fully_aligned	RX PCS fully aligned indication.

System PLL lock PIO (0x30900030)¶

Bit	Name	Description
0	o_sys_pll_locked	System PLL lock indication.

GTS refclk fail status PIO (0x20000500)¶

This PIO exposes gts_rst_seq_left.o_refclk_fail_status[7:0], the status of the GTS input reference clock buffer protection. It sits at 0x20000500, outside the 0x30000000 block used by the rest of this subsystem, because the GTS reset sequencer lives at the top level of the design rather than inside sfp_fabric_subsys. Only the left-hand sequencer is instantiated for this single-lane SFP configuration. The GTS hardware monitors each enabled refclk input buffer and automatically turns a buffer off when it no longer detects a toggling clock at the input (the detector covers 25 MHz to 380 MHz; a clock outside that range is also treated as absent). A bit value of 1 means the corresponding refclk buffer was enabled at configuration but has since been turned off due to the absence of a clock at its input. A bit value of 0 means the buffer has not been turned off.

Bit	Name	Description
0	o_refclk_fail_status[0]	1A/4A local refclk buffer status. 0 = buffer on, 1 = buffer turned off (no clock detected at the input).
1	o_refclk_fail_status[1]	1A/4A global refclk buffer status. 0 = buffer on, 1 = buffer turned off (no clock detected at the input).
7:2	o_refclk_fail_status[7:2]	Reserved / unused in this single-lane SFP configuration.

NOTE - A refclk buffer that has been turned off by the protection mechanism does not turn back on by itself. On ES silicon, the register method must be used to turn a refclk buffer back on. On production devices, either the register method or the status and control ports method may be used. However, the design is currently only configured to support the register method, which is driven from gts_left_refclk_on_pio, a 10-bit write-only PIO at 0x20000520. See section 3.7.5 of the GTS Transceiver PHY User Guide for more details

SFP input PIO (0x30900040, only for the MitySOM-Mini)¶

Bit	Name	Description
0	SFP1_TX_FLT_N	SFP transmitter fault input.
1	SFP1_MOD_PRSNT_N	SFP module-present input.
2	SFP1_LOS	SFP loss-of-signal input.

For the MitySOM, these values can be read from the PCAL6416A on I2C bus 3 at address 0x20.

SFP output PIO (0x30900050, only for the MitySOM-Mini)¶

Bit	Name	Description
0	SFP1_TX_DIS	SFP transmitter-disable output.
1	SFP1_RS0	SFP rate-select output.

For the MitySOM, these values can be read from the PCAL6416A on I2C bus 3 at address 0x20.

Hardware Setup¶

This example can be run on either the MitySOM Development Kit or the MitySOM-Mini Development Kit.

It can be exercised using either two development kits connected together or a single development kit with the SFP interface looped back.

• Program a design that includes this SFP fabric example.
• Install the SFP module(s) on SFP1 and create the external TX-to-RX path. For a two-kit setup, connect the two SFP1 interfaces together. For a single-kit setup, use an external loopback on SFP1.
• Boot Linux on the HPS so memtool can access the lightweight bridge.

Note: For the MitySOM, the TX Disable line must be configured low for the loopback to work. This can be done by running the following:

i2cset -y 3 0x20 0x06 0xFD
i2cset -y 3 0x20 0x02 0xFD

Note: For the MitySOM, the SFP cage must be installed into J5.

Optional Pre-Check¶

Before starting traffic, it is useful to confirm the clocks and link-side status bits are alive:

memtool 0x30900000 1
# Expected healthy value: lock/ready bits asserted. Typically 0x0000000F.

memtool 0x30900010 1
# Expected healthy value: active-low reset acknowledge bits deasserted/high. Typically 0x00000007.

memtool 0x30900020 1
# Expected healthy value after link comes up: block lock/alignment asserted and no faults. Typically 0x00000021.

memtool 0x30900030 1
# Expected healthy value: system PLL locked. Typically 0x00000001.

memtool 0x20000500 1
# Expected healthy value: all refclk buffers on. Typically 0x00000000.

The reset-status PIO is mainly a debug aid because its bits are active-low acknowledge signals.

Typical signs of a healthy path are:

• Clock status shows the lock/ready bits asserted.
• System PLL lock is asserted.
• GTS refclk fail status reads as 0x00000000 (no refclk buffer has been turned off).
• Link status shows RX block lock / alignment once the external loopback path is in place.

Start Packet Generation¶

Start the packet generator with:

memtool 0x30000000=5

In this example, 5 means:

• bit 0 = start transmitting packets
• bit 2 = keep running in continuous mode

Verify TX/RX Packet Activity¶

Read the TX/RX packet counters and the TX/RX error counters:

memtool 0x30000028 1
memtool 0x30000030 1
memtool 0x30000040 1
memtool 0x30000048 1

For a full 64-bit snapshot, read the 12 words starting at 0x30000020:

memtool 0x30000020 C

To watch the key TX/RX count and error registers in one read, use watch to read the 12 words starting at 0x30000020. The packet counters should keep increasing if the example is working and the SFP path is looped back correctly:

watch memtool 0x30000020 C

Expected behavior:

• 0x30000028 increments as packets are transmitted by the packet client.
• 0x30000040 increments as those packets return through the external SFP loopback path.
• 0x30000030 and 0x30000048 should remain at zero during a clean run.
• RX should track TX closely in steady state.

If TX counts increase but RX counts stay at zero, the packet generator is running but the serial link is not looping back. In that case, re-check the external loopback path and the status PIO registers at 0x30900000, 0x30900020, 0x30900030 and 0x20000500.

Stop Packet Generation¶

To stop the example:

memtool 0x30000000=0