• Table of contents
• Building PCIe Hard IP
  • Example
    • Fabric Resource Utilization
  • Output Files
  • Prerequisites
  • Getting Started
    • Timing with PCIe

Building PCIe Hard IP¶

This page exists as a starting point for people interested in using the PCIe interface on the Critical Link Development Board. This page contains links to all the necessary portions (FPGA, kernel, dtb, etc.) for getting the system up and running quickly. This page also documents specifics on the PCIe Hard IP and pitfalls that may occur.

Example¶

Currently we are using a ported version of the RocketBoards.org PCIe Root Port With MSI example. The original unmodified source can be found at http://www.rocketboards.org/foswiki/view/Projects/PCIeRootPortWithMSI.

Fabric Resource Utilization¶

For the example provided in this Wiki we have recorded the following Cyclone V fabric utilization based upon the 110KLE sized Cyclone V SoC:

Resource	Available	Used	% Used
ALM	41,910	10,230	24%
Total registers	N/A	14,246	N/A
Memory (M10K & MLAB)	5.6Mbit	2.7Mbit	48%
DSP Blocks	112	0	0%
Transceivers TX & RX	6	4	67%
PLLs	12	1	8%
DLLS	4	1	25%

Altera PCIe Hard IP Documentation: [[http://www.altera.com/literature/ug/ug_c5_pcie.pdf]]

Output Files¶

• FPGA RBF for the 5CX-H6-42A model with PCIe support - dev_5cs.rbf
• Bootloader for the 5CSX-H6-42A model - u-boot-with-spl.sfp
• Linux Kernel (with PCIe and SATA/AHCI support compiled in) - zImage
• Device Tree blob with PCIe support - socfpga_mitysom5csx_devkit.dtb

Prerequisites¶

Using a Linux machine (Ubuntu 22.04, is what this was tested on) or WSL2 in Windows (guide for install WSL can be found here: https://learn.microsoft.com/en-us/windows/wsl/install)

1. Download the pre-built Yocto SDK installer: poky-glibc-x86_64-mitysom-image-base-cortexa9hf-neon-mitysom-c5-toolchain-4.0.19.sh
2. Install the script to /opt/poky/4.0.19/.
   

   ./Downloads/poky-glibc-x86_64-mitysom-image-base-cortexa9hf-neon-mitysom-c5-toolchain-4.0.19.sh
   

Getting Started¶

1. Build the FPGA for the dev_5csx_h6_42a_pcie reference project following the steps in Building the FPGA (Note: the timing may fail when compiling; see Timing with PCIe for information on how to continue the FPGA build).
2. Build the bootloader following the steps in Building the Bootloader.
3. Build the filesystem following the steps in Building the Filesystem OR use the provided filesystem mitysom-image-base-mitysom-c5.tar.gz.
4. Build the sd card following the steps in Building the SD Card Image.
5. Source the Yocto SDK.
   

   source /opt/poky/4.0.16/environment-setup-cortexa9hf-neon-poky-linux-gnueabi
   

6. Grab the kernel from our Git repo.
   

   git clone https://support.criticallink.com/git/linux-socfpga.git -b socfpga-6.1.55-lts
   

7. Change into that directory.
   

   cd linux-socfpga
   

8. Add the following node within the arch/arm/boot/dts/socfpga_mitysom5csx_devkit.dts file
   

   soc {
                   pcie_0_msi_to_gic_gen_0: msi@0xff20c000 {
                           compatible = "altr,msi-1.0", "altr,msi-1.0";
                           reg = <0xff20c080 0x00000010>, <0xff20c000 0x00000080>;
                           reg-names = "csr", "vector_slave";
                           interrupt-parent = <&intc>;
                           interrupts = <0 45 4>;
                           msi-controller = <1>;
                           num-vectors = <32>;
                   };
                   pcie_0_pcie_hip_avmm: pcie@0xc0000000 {
                           compatible = "altr,pcie-root-port-23.1", "altr,pcie-root-port-1.0";
                           reg = <0xc0000000 0x10000000>, <0xff208000 0x00004000>;
                           reg-names = "Txs", "Cra";
                           interrupt-parent = <&intc>;
                           interrupts = <0 46 4>;
                           interrupt-controller;
                           #interrupt-cells = <1>;
                           device_type = "pci";
                           bus-range = <0x00000000 0x000000ff>;
                           ranges = <0x82000000 0 0x00400000 0xc0400000 0 0x0fc00000>;
                           msi-parent = <&pcie_0_msi_to_gic_gen_0>;
                           #address-cells = <3>;
                           #size-cells = <2>;
                           interrupt-map-mask = <0 0 0 7>;
                           interrupt-map = <0 0 0 1 &pcie_0_pcie_hip_avmm 1>,
                                   <0 0 0 2 &pcie_0_pcie_hip_avmm 2>,
                                   <0 0 0 3 &pcie_0_pcie_hip_avmm 3>,
                                   <0 0 0 4 &pcie_0_pcie_hip_avmm 4>;
                   };
           };
   

9. Add the following kernel configs in the arch/arm/configs/mitysom5csx_devkit_defconfig file
   

   + CONFIG_PCIE_ALTERA=y
   + CONFIG_PCIE_ALTERA_MSI=y
   + CONFIG_PCI_MSI=y
   + CONFIG_BLK_DEV_NVME=y
   + CONFIG_NVME_CORE=y
   

10. Set the configuration for the kernel.
    

    make ARCH=arm CROSS_COMPILE=arm-poky-linux-gnueabi- -j8 mitysom5csx_devkit_defconfig
    

11. Build the kernel. The kernel will be built at arch/arm/boot/zImage.
    

    make ARCH=arm CROSS_COMPILE=arm-poky-linux-gnueabi- -j8 zImage
    

12. Build the device tree blobs. The device tree blobs will be built in arch/arm/boot/dts.
    

    make ARCH=arm CROSS_COMPILE=arm-poky-linux-gnueabi- -j8 dtbs
    

13. Follow the steps in Building the SD Card Image to replace the kernel and device tree with the new zImage and socfpga_mitysom5csx_devkit.dtb.

Timing with PCIe¶

When porting the RocketBoards.org project to the Critical Link 5CSX development board there were issues with the system meeting timing. The RocketBoards.org project is targeted at a 5CSXFC6D6, while the Critical Link MitySOM has a 5CSXFC6C6. We were only able to successfully port the example project and not have timing errors if we started with the cv_soc_rp_simple_design.tar.gz.

To continue building the FPGA after the timing failure, run the commands below which will finish the 'make rbf' command.

mkdir -p stamp
touch stamp/quartus.stamp
quartus_cpf -c output_files/dev_5cs.sof output_files/dev_5cs.rbf