Mity CPU Platforms » MitySOM-5CSX Altera Cyclone V

Hi Lucas,

Do you have the pull up resistors on the SD Card interface?

Have you looked at this support page on Carrier Board Gotchas?

-Mike

Here's the relevant area on my schematic:

Hi Lucas,

It looks like you do have the pull ups on CMD and D0. Can you post your console log of the SOM hanging during boot?

Thanks,
Dan

No output from console on the module that has the NCONFIG light come on.
New module doesn't boot up, so no output from console.

Correction!

I held down the POR low a little longer and it booted...

U-Boot SPL 2013.01.01 (Sep 14 2021 - 18:34:24)
BOARD : Critical Link MitySOM-5CSx Module
CLOCK: EOSC1 clock 25000 KHz
CLOCK: EOSC2 clock 25000 KHz
CLOCK: F2S_SDR_REF clock 0 KHz
CLOCK: F2S_PER_REF clock 0 KHz
CLOCK: MPU clock 600 MHz
CLOCK: DDR clock 400 MHz
CLOCK: UART clock 100000 KHz
CLOCK: MMC clock 50000 KHz
CLOCK: QSPI clock 400000 KHz
RESET: COLD
INFO : Watchdog enabled
SDRAM: Initializing MMR registers
SDRAM: Calibrating PHY
SEQ.C: Preparing to start memory calibration
SEQ.C: CALIBRATION PASSED
SDRAM: 512 MiB
ALTERA DWMMC: 0

U-Boot 2013.01.01 (Sep 14 2021 - 18:34:36) Critical Link MitySOM-5CSx

1. Starting application at 0x1FF75598 ...
2. Application terminated, rc = 0x0
3. Starting application at 0x1FF75598 ...
4. Application terminated, rc = 0x0
5. Flattened Device Tree blob at 00000100
   Booting using the fdt blob at 0x00000100
   Loading Device Tree to 03ff7000, end 03fff402 ... OK

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 4.9.78-ltsi-g818b616 (user@MitySOM-Dev) (gcc version 7.3.0 (GCC) ) #1 SMP Tue May 11 13:18:50 EDT 2021
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt:Machine model: MitySOM-5CSX Altera SOCFPGA Cyclone V
Memory policy: Data cache writealloc
percpu: Embedded 14 pages/cpu @8fdca000 s26764 r8192 d22388 u57344
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 65024
Kernel command line: console=ttyS0,115200 root=/dev/mmcblk0p3 rw rootwait mem=256M
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 249820K/262144K available (6144K kernel code, 446K rwdata, 1384K rodata, 1024K init, 153K bss, 12324K reserved, 0K cma-reserved, 0K highmem)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0x90800000 - 0xff800000 (1776 MB)
lowmem : 0x80000000 - 0x90000000 ( 256 MB)
pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
.text : 0x80008000 - 0x80700000 (7136 kB)
.init : 0x80900000 - 0x80a00000 (1024 kB)
.data : 0x80a00000 - 0x80a6f9ec ( 447 kB)
.bss : 0x80a6f9ec - 0x80a960a4 ( 154 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
Hierarchical RCU implementation.
Build-time adjustment of leaf fanout to 32.
NR_IRQS:16 nr_irqs:16 16
L2C: DT/platform modifies aux control register: 0x02060000 -> 0x02460000
L2C-310 erratum 769419 enabled
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410030c9, AUX_CTRL 0x76460001
GIC: PPI13 is secure or misconfigured
GIC: PPI13 is secure or misconfigured
clocksource: timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
sched_clock: 32 bits at 100MHz, resolution 10ns, wraps every 21474836475ns
Switching to timer-based delay loop, resolution 10ns
Console: colour dummy device 80x30
Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=1000000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
ftrace: allocating 21519 entries in 64 pages
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x100000 - 0x100058
CPU1: failed to come online
Brought up 1 CPUs
SMP: Total of 1 processors activated (200.00 BogoMIPS).
CPU: All CPU started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 512 (order: 3, 32768 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
EDAC MC: Ver: 3.0.0
FPGA manager framework
fpga-region soc:base-fpga-region: FPGA Region probed
clocksource: Switched to clocksource timer1
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
workingset: timestamp_bits=30 max_order=16 bucket_order=0
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
ntfs: driver 2.1.32 [Flags: R/W].
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
io scheduler noop registered (default)
dma-pl330 ffe01000.pdma: Loaded driver for PL330 DMAC-341330
dma-pl330 ffe01000.pdma: DBUFF-512x8bytes Num_Chans-8 Num_Peri-32 Num_Events-8
Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
console [ttyS0] disabled
ffc02000.serial0: ttyS0 at MMIO 0xffc02000 (irq = 45, base_baud = 6250000) is a 16550A
console [ttyS0] enabled
brd: module loaded
at24 0-0050: 2048 byte 24c16 EEPROM, writable, 1 bytes/write
cadence-qspi ff705000.spi: n25q128a11 (16384 Kbytes)
6 ofpart partitions found on MTD device ff705000.spi.0
Creating 6 MTD partitions on "ff705000.spi.0":
0x000000000000-0x000000040000 : "Preloader Image 0-3"
0x000000040000-0x000000050000 : "U-Boot Env"
0x000000050000-0x000000060000 : "DTB"
0x000000060000-0x0000000e0000 : "U-Boot"
0x0000000e0000-0x0000008e0000 : "Kernel"
0x0000008e0000-0x000001000000 : "FPGA"
libphy: Fixed MDIO Bus: probed
CAN device driver interface
c_can_platform ffc00000.can: c_can_platform device registered (regs=908eb000, irq=26)
c_can_platform ffc01000.can: c_can_platform device registered (regs=908ed000, irq=30)
stmmac - user ID: 0x10, Synopsys ID: 0x37
Ring mode enabled
DMA HW capability register supported Enhanced/Alternate descriptors
Enabled extended descriptors
RX Checksum Offload Engine supported
COE Type 2
TX Checksum insertion supported
Enable RX Mitigation via HW Watchdog Timer
libphy: stmmac: probed
eth%d: PHY ID 00221611 at 3 IRQ POLL (stmmac-0:03) active
dwc2 ffb40000.usb: EPs: 16, dedicated fifos, 8064 entries in SPRAM
dwc2 ffb40000.usb: DWC OTG Controller
dwc2 ffb40000.usb: new USB bus registered, assigned bus number 1
dwc2 ffb40000.usb: irq 47, io mem 0x00000000
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: DWC OTG Controller
usb usb1: Manufacturer: Linux 4.9.78-ltsi-g818b616 dwc2_hsotg
usb usb1: SerialNumber: ffb40000.usb
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
mousedev: PS/2 mouse device common for all mice
rtc-abx80x 0-0069: model 1805, revision 2.3, lot 3f9, wafer 3, uid 31cb
rtc-abx80x 0-0069: Oscillator failure, data is invalid.
rtc-abx80x 0-0069: rtc core: registered abx8xx as rtc0
i2c /dev entries driver
lm73 0-004c: sensor 'lm73'
EDAC MC: No ECC/ECC disabled [0x000A8042]
Synopsys Designware Multimedia Card Interface Driver
dw_mmc ff704000.dwmmc0: IDMAC supports 32-bit address mode.
dw_mmc ff704000.dwmmc0: Using internal DMA controller.
dw_mmc ff704000.dwmmc0: Version ID is 240a
dw_mmc ff704000.dwmmc0: DW MMC controller at irq 39,32 bit host data width,1024 deep fifo
mmc_host mmc0: Bus speed (slot 0) = 50000000Hz (slot req 400000Hz, actual 396825HZ div = 63)
dw_mmc ff704000.dwmmc0: 1 slots initialized
lp5562 0-0030: internal clock used
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
fpga_manager fpga0: Altera SOCFPGA FPGA Manager registered
altera_hps2fpga_bridge ff400000.fpga_bridge: fpga bridge [lwhps2fpga] registered
altera_hps2fpga_bridge ff500000.fpga_bridge: fpga bridge [hps2fpga] registered
oprofile: no performance counters
oprofile: using timer interrupt.
NET: Registered protocol family 17
NET: Registered protocol family 15
can: controller area network core (rev 20120528 abi 9)
NET: Registered protocol family 29
can: raw protocol (rev 20120528)
can: broadcast manager protocol (rev 20161123 t)
can: netlink gateway (rev 20130117) max_hops=1
8021q: 802.1Q VLAN Support v1.8
Key type dns_resolver registered
ThumbEE CPU extension supported.
Registering SWP/SWPB emulation handler
input: gpio_keys as /devices/platform/gpio_keys/input/input0
ttyS0 - failed to request DMA
Waiting for root device /dev/mmcblk0p3...
mmc_host mmc0: Bus speed (slot 0) = 50000000Hz (slot req 50000000Hz, actual 50000000HZ div = 0)
mmc0: new high speed SDHC card at address e624
mmcblk0: mmc0:e624 SS08G 7.40 GiB
mmcblk0: p1 p2 p3
EXT4-fs (mmcblk0p3): mounting ext3 file system using the ext4 subsystem
EXT4-fs (mmcblk0p3): warning: mounting fs with errors, running e2fsck is recommended
EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) on device 179:3.
devtmpfs: mounted
Freeing unused kernel memory: 1024K
usb 1-1: new full-speed USB device number 2 using dwc2
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
random: fast init done
INIT: version 2.88 booting
EXT4-fs error (device mmcblk0p3): ext4_mb_generate_buddy:758: group 3, block bitmap and bg descriptor inconsistent: 16506 vs 16253 free clusters
EXT4-fs error (device mmcblk0p3): ext4_mb_generate_buddy:758: group 4, block bitmap and bg descriptor inconsistent: 17612 vs 17635 free clusters
usb 1-1: device descriptor read/64, error -71
EXT4-fs error (device mmcblk0p3): ext4_mb_generate_buddy:758: group 5, block bitmap and bg descriptor inconsistent: 18448 vs 18688 free clusters
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
Starting udev
usb 1-1: device descriptor read/64, error -71
udevd⁷⁹: starting version 3.2.2
udevd⁸⁰: starting eudev-3.2.2
usb 1-1: new full-speed USB device number 3 using dwc2
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
usb 1-1: device descriptor read/64, error -71
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
dwc2 ffb40000.usb: XactErr
usb 1-1: device descriptor read/64, error -71
usb usb1-port1: attempt power cycle
EXT4-fs (mmcblk0p3): re-mounted. Opts: (null)
rtc-abx80x 0-0069: Oscillator failure, data is invalid.
hwclock: RTC_RD_TIME: Invalid argument
Mon Nov 16 22:59:50 UTC 2020
JBD2: Spotted dirty metadata buffer (dev = mmcblk0p3, blocknr = 0). There's a risk of filesystem corruption in case of system crash.
INIT: Entering runlevel: 5
Configuring network interfaces... socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
RTNETLINK answers: Device or resource busy
Starting system message bus: dbus.
Starting OpenBSD Secure Shell server: sshd
NET: Registered protocol family 10
done.
Starting rpcbind daemon...done.
starting statd: done
Starting atd: OK
exportfs: can't open /etc/exports for reading
NFS daemon support not enabled in kernel
loading RTC calibration values from /etc/abrtc.cal ...
Starting system log daemon...0
Starting kernel log daemon...0
Starting crond: OK
Starting Network Interface Plugging Daemon:socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
eth0.
Starting tcf-agent: OK

Poky (Yocto Project Reference Distro) 2.4.4 mitysom-c5 /dev/ttyS0

mitysom-c5 login: socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed
socfpga-dwmac ff702000.ethernet: Failed to reset the dma
stmmac_hw_setup: DMA engine initialization failed
stmmac_open: Hw setup failed

(continued)

Hi Lucas,

In your boot log, I am seeing several error messages around the SD-MMC card (ext4-fs /dev/mmcblk0p* devices) that I would not expect to see with a freshly flashed SD card.

What model SD card are you using? Some of the cheaper ones you can get in bulk are really not any good. I've seen bad SD cards exhibit similar behavior.

Another issue could be some sort of signal integrity issue on the SD-card bus and data lanes from the SD-card slot to the module interface. Are your bus lines and clock matched (within reason) in length and clear of any cross-talk aggressors (particularly the clock line)? Can you take a snapshot of this portion of the layout or share design files? We can arrange to do that via email / privately if you are agreeable to that and put an NDA in place if there are concerns.

Do you have a Critical Link MitySOM-5CSX devkit? Can we try some experiments to isolate the issue to the SOM, carrier card, or SDCARD? You could try the using the SD-card and/or swapping questionable modules to see if the problem tracks or not.

Let me know what you think.

-Mike

We've used fancypants Sandisk "Industrial" ones, but also regular Sandisk branded ones. I'll put on the design review list of testing SD cards and/or plan to move to eMMC.

It might be worth pointing out that we're using microSD cards. The holder has been on my suspect list as they're somewhat mechanically flimsy until they're soldered down.

I think the lines from the card holder to the module holder are somewhat close, but they weren't specifically designed to be matched. Definitely a point to compare units that booted versus units that didn't.

We do have the dev kit. We need to do some differential diagnosis on our ADC connections anyway, so I'll bug Travis into running some tests there as well.

Thanks!

Here are the parts of the board in question and this version does look like a confused mess now that I'm looking at it.

The SOIC on top is a TPS2044D to handle the power switching for a USB hub that I added.

Got together the dev kit:

-booted fine (both LEDs on the module came on, then one went off) with the microSD in an adapter

-tried the module that was turning on the NCONFIG light on my board. On the dev board D405 lit up and D100 sort of lit up (tri-state-ish)

This is probing the HPS_POR_N:

Hi Lucas,

Did the SOM that was turning on the NCONFIG light on your baseboard boot fine in our Dev Kit?

What do you have connected to the HPS_NOR_N pin on your baseboard? Here's a snippet from our design guide about connecting up that pin:

These should be connected to an open collector output if driving the reset from
the carrier board. The MitySOM module includes a 10KΩ pull-up resistor on each reset line

Dan

The Module that turned on our NCONFIG light did not boot on the dev board. I copied our NCONFIG LED from the dev board reference schematic with 3.3v -> 267 ohm resistor -> NCONFIG pin w/ TP10

HPS_POR_N is connected to a 1000pF cap and a switch to pull it to ground.

Hi Lucas,

It is sounding like the SOM might of gotten damaged. Does the green LED on the SOM turn on? This is the power good LED that will turn on if all the power supplies on the SOM come up correctly.

Dan

No lights come up on the module that lights up NCONFIG light.

On the second test module, both lights stay on until a few seconds after HPS_POR_N is pulled low. Then the yellow light goes off. This is the condition that got as far as that boot log above.

I've had the condition where the board will boot after HPS_POR_N is pulled low, and a condition where the boot doesn't occur but the illumination level of the LEDs on the module changes.

If I disconnect supply power to the ethernet (L501 and L503, same as the dev board) the board goes from the second condition to the first.

Is Vin to the module supposed to come up before VIO supply voltages? I have 5, 3.3, and 2.5 all coming up together.

On the dev board, R109 is a pullup for BOOTSEL2 from +1.8VIO, but there is also C523 and R503 connected to the bus and R503 is a pullup on the DVDDH_1V8 rail.

When I had to switch to the KSZ9021 because the KSZ9031 was unavailable, as a result had to change DVDDH to 2.5volts.

Problem would seem to be that Bank7B is fixed for 1.8 volts...

Once 2.5v supply to the Ethernet is removed and the trace connecting the BOOTSEL2 pullup to the DVDDH_2V5 pullup is cut then there is boot.

Well. Gather egg off face, make omelette I suppose.

Hi Lucas,

Sorry for the delay, the cyclone V can handle all the supplies powering up together.

Ah that could do it because as you said Bank7B is fixed at 1.8V and the PHY would be overvoltaging those IOs on the SOM. Sorry I didn't notice this when you were talking about swapping to the KSZ9021 a few months ago, I was too focused on the software side and missed the IO voltage mismatch.

Dan