BH PCIe Root Port bring-up on MitySOM-A5E (A5ED-B64-144-SRC-X) — hard system hang, likely missing GTS reset sequencer for right-side bank
Added by Brady Heater 1 day ago
Subject: PCIe Root Port bring-up on MitySOM-A5E (A5ED-B64-144-SRC-X) — hard system hang, likely missing GTS reset sequencer for right-side bank
Setup:
Board: MitySOM-A5E, part A5ED-B64-144-SRC-X, mitysom-a5e-ref-base project, pro_25.3.1_stable branch
Quartus Prime Pro 25.3.1
Goal: enable the PCIe Gen3 x4 Root Port (pcie_rp_subsys, already instantiated in a5e.qsys on the GTSR4C transceiver bank) to talk to a Broadcom 9660-16i RAID HBA via: FMC connector → HiTech Global HTG-FMC-PCIE-RC adapter → a PCIe Gen5 x8 Lane Reassignment Adapter → the HBA
Issues found and already fixed on our end:
Missing custom IP for the PCIe subsystem. pcie_rp_subsys referenced 5 custom IP components (irq_stretcher, config_timeout, msi_ordering, pll_locked2reset, addr_filter) that don't exist in mitysom-a5e-ref-base/custom_ip/, though they do exist in the dedicated mitysom-a5e-ref-pcie-gen3x4-rp variant (matching lane count/gen speed) at git commit fccadac. We copied them across and Platform Designer now generates cleanly.
PCIe reference clock disabled at the board level. i_pcie_refclk is pinned to PIN_AP16/PIN_AP21 (REFCLK_GTSR4C_CH1), sourced from the carrier's SI5332 clock synthesizer (U68, I2C addr 0x6A). Decoding the register file scripts/Si5332-GM3-RevD-r0-MitySOM-A5E-Std-Dev-Kit-Registers.csv against the Si5332 reference manual, register 0xB6 bit 1 (OUT1_OE) was 0 — that output channel was disabled, so the PCIe transceiver's reference clock was never actually present regardless of any FPGA-side pin assignment. We changed 0xB6 from 0xE5 to 0xE7 (enabling bit 1) and confirmed via the regenerated config_pll.scr (i2c mw 0x6a 0xB6 0xE7) that this is now applied at boot.
Current blocking issue:
With the clock fix and PCIe device-tree overlay (socfpga_agilex5_mity_a5e_devkit_pcie0_rp.dtbo) applied, the board now boots much further than before (previously hung instantly on FPGA host-bridge init). However:
With no PCIe device attached, the kernel's altera-pcie driver enumerates the bus, retries, and eventually the RCU stall detector reports CPU 3 specifically hung (0 ticks this GP) — a single-core lockup, likely the driver looping on a config-space read to a nonexistent downstream device with no proper timeout.
With the real device chain attached (FMC adapter + lane-reassignment adapter + Broadcom HBA), the failure is worse: a full system lockup, all cores unresponsive, no further kernel output, no response to input — occurring slightly earlier in boot than the no-device case.
Investigating a5e.qsys, we noticed it instantiates two GTS reset-sequencer elements: gts_rst_seq_left (fully wired — dozens of parameter/port-mapping references, backed by ip/a5e/a5e_gts_rst_seq_left.ip on disk) and gts_rst_seq_right (only 3 bare references: a UI placeholder element, one IP-XACT library tag, and a single exposed o_pma_cu_clk conduit port — no real connections). No a5e_gts_rst_seq_right.ip file exists anywhere in the repo, and it has no history in git across any board variant — it appears to have never been completed. Since pcie_rp_subsys sits on GTSR4C (a right-side bank), and gts_rst_seq_left only sequences the left-side banks, this looks like a plausible gap: the right-side transceiver bank hosting the PCIe hard IP may have no working reset sequencer, which could explain a hard hardware-level hang once real link training is attempted.
What we'd like from you:
Does the PCIe Root Port configuration on GTSR4C require a properly parameterized gts_rst_seq_right (analogous to gts_rst_seq_left), and if so, could you provide the correct Platform Designer component configuration/connections for it?
Alternatively, if gts_rst_seq_right isn't actually required for this bank/IP combination, is there a different known reset-sequencing requirement we're missing for bringing up PCIe hard IP on a right-side GTS bank?
Is mitysom-a5e-ref-pcie-gen3x4-rp (the variant we pulled the custom IP from) known to have been fully bring-up-tested on real hardware, or is PCIe support in this reference design family still work-in-progress?
Current state of our tree: we've reverted scripts/boot.cmd to not apply the PCIe overlay by default, so the board boots reliably in the meantime. The Si5332 fix and restored custom IP remain in place, ready to re-enable once the reset-sequencer question is resolved.
Replies (1)
MF RE: PCIe Root Port bring-up on MitySOM-A5E (A5ED-B64-144-SRC-X) — hard system hang, likely missing GTS reset sequencer for right-side bank - Added by Mike Fiorenza about 7 hours ago
Hi Brady,
Thanks for the details. Both of your fixes look correct and I think you're one step from the real problem (which you pointed out).
You moved the PCIe IP from the left shoreline bank to the right shoreline bank (the FMC), but it sounds like you still have it attached to the left reset sequencer. The reset sequencers are a per-shoreline resource, so a right side bank needs its own instance.
You can go into Platform Designer and declare a new GTS reset sequencer IP and check the box for PCIe and then connect the PCIe IP to it similarly to how it was connected to the previous reset sequencer.
One other thing that has to move with the bank: PERST has to land on one of the device's dedicated PERST pins. The M.2 design uses PIN_BU109, which is the L1B PERST pin. For a bank 4C port you want PIN_PERST_N_R4C_0 (BW28) or PIN_PERST_N_R4C_1 (BF32) instead, there's a _0 and a _1 for each bank and which one applies comes from the PERST erratum. If PERST isn't on the right dedicated pin the IP never comes out of reset, which looks exactly like a hang.
To answer your questions:
1. Yes you need a separate reset sequencer for the right and left side banks of the device.
2. Yes we have tested this design on our Agilex5 platforms, however only with the IP on the left side bank. We are currently working on releasing a new updated PCIe example design for our platforms, however, that may still be several weeks away. Our examples are based off of Altera's example, so you can look at their design for reference here in the meantime: https://github.com/altera-fpga/agilex5-ed-pcie-rp
- Mike