Project

General

Profile

I2C Slave Communication Hanging

Added by Jared Kirschner over 6 years ago

We're experiencing a problem with I2C communications with a slave on address 0x55 on a MitySOM-5CSX, shown in the attached image (scope_3.png).

In the image, a read is attempted, with the first byte NACK'd by the slave, and an invalid stop condition. It almost looks like the master attempts to pull the clock (yellow) high before data is pulled high (which would be a valid stop condition), but the clock is held low by something else for some time. In Linux kernel / user space, we see this as a "lost arbitration error", which makes sense given the observed behavior of the signals.

Some additional context:
  • We have other I2C busses on the system, and have had no issues communicating with multiple slaves on said other busses.
  • The slave is a Sitronix ST1663 (address 0x55).
  • Said slave is the only slave on its I2C bus.
  • I2C writes seem to complete correctly. Only reads have the aforementioned problem.
  • We're fairly confident the problem isn't caused by the slave. If we use an Aardvark (https://www.totalphase.com/products/aardvark-i2cspi/) as a master to said slave, the slave behaves correctly / I2C reads and writes are fine. We have also used the Aardvark as a slave to the MitySOM-5CSX (without the actual slave connected), and get the same behavior (the attached screenshot was actually taken with the Aardvark as a slave).
  • We've combed through our FPGA code / schematic / device tree and can't find anything that would be interfering with the bus.
  • We're using Linux 4.1.22 LTSI RT.

Have you ever encountered a similar problem / do you have any thoughts as to why this might be happening?

Is there additional information we could provide that might make it easier to figure out what's going on?

Lastly, and this is because we're grasping at straws at this point, there's something mentioned briefly in the Cyclone V HPS Technical Reference Manual (page 2880, in Section 20. I2C Controller, about the ic_tar and ic_sar addresses)... ic_tar mentions " If the ic_tar and ic_sar are the same, loopback exists but the FIFOs are shared between master and slave, so full loopback is not feasible. Only one direction loopback mode is supported (simplex), not duplex). A master cannot transmit to itself; it can transmit to only a slave." The default values for ic_tar and ic_sar are 0x55, the same as our slave address. I'm not sure if this is causing any problems. I did an initial test in which I tried to force ic_sar to something other than 0x55 within the i2c kernel driver, but I'm not 100% sure I did this correctly.

If we can't get this working through the i2c system soon, we may resort to bit-banging the communication, so we have full control (as something on the SOM/Linux side of things seems to be the cause of the problem).

We appreciate any help you can provide -- thanks.

scope_3.png (32.5 KB) scope_3.png Attempted read with invalid stop condition

Replies (12)

RE: I2C Slave Communication Hanging - Added by Michael Williamson over 6 years ago

Which I2C bus are you using? An HPS peripheral connected directly to HPS pins or routed via FPGA fabric?

This almost looks like there is no pull-up on the bus...

-Mike

RE: I2C Slave Communication Hanging - Added by Michael Williamson over 6 years ago

Also, Can you share a snippet of the user space code you are using to issue the read request?

Thanks.

-Mike

RE: I2C Slave Communication Hanging - Added by Jared Kirschner over 6 years ago

We are using I2C2 routed through the FPGA fabric (see image "I2C2_Connector.PNG"). It does have pullups to the best of my knowledge (see image "I2C2_Pullups.PNG").

Though we initially tried to communicate via user space, the vendor indicated that repeated start is not supported between write and read, and all of the I2C user space functions we could find issued a repeated start. Thus, we've been using a kernel driver provided by the vendor to put traffic on the bus (automatically performed at system start-up). I'll see if I can send along the driver files / I2C protocol description.

RE: I2C Slave Communication Hanging - Added by Jared Kirschner over 6 years ago

I've attached the driver source/header. I'm waiting to hear from the touch controller vendor whether I can share the protocol description.

RE: I2C Slave Communication Hanging - Added by Michael Williamson over 6 years ago

Are you using the open drain primitive in the top level of the FPGA project?

-Mike

RE: I2C Slave Communication Hanging - Added by Clyde Shappee over 6 years ago

All,

This is the open drain primitive we are using for the I2C interfaces:

wire w_i2c3_scl_oe, w_i2c3_scl, w_i2c3_sda_oe, w_i2c3_sda;
assign DVI_SCL = (w_i2c3_scl_oe) ? 1'b0 : 1'bz;
assign DVI_SDA = (w_i2c3_sda_oe) ? 1'b0 : 1'bz;
assign w_i2c3_scl = DVI_SCL;
assign w_i2c3_sda = DVI_SDA;

Clyde

Clyde R. Shappee
SENIOR ELECTRICAL ENGINEER

CONTINUUM
21 Drydock Avenue, Suite 410 W
Boston, MA 02210 USA
+ 1 617 969 5400
ContinuumInnovation.com

NOTE: This email and any attachments may contain confidential information. If you're not the intended addressee, and it has been sent to you by mistake, please let us know by return email and delete it, immediately, without using any information included or any attachments therein. Thank you.

From: Jared Kirschner
Sent: Tuesday, July 18, 2017 2:25 PM
To: Clyde Shappee <>; Jay Petrie <>
Subject: FW: [MitySOM-5CSX Altera Cyclone V - Software Development - msg5373] RE: I2C Slave Communication Hanging

Can either of you answer this question?

This is regarding the I2C (touch controller bus). The link to the whole thread is included below.

This is a priority for the software, so please answer at your earliest convenience.

From: <mailto:> [mailto:]
Sent: Tuesday, July 18, 2017 1:09 PM
Subject: [MitySOM-5CSX Altera Cyclone V - Software Development - msg5373] RE: I2C Slave Communication Hanging

RE: I2C Slave Communication Hanging - Added by Jared Kirschner over 6 years ago

Some additional context / correction:

I2C2 and I2C3 are both configured the same way. If you replace "3" with "2" in the above example, and replace "DVI" with "TFT" (related to our ST1633 touch controller), the result is the I2C2 setup, so...:

wire w_i2c2_scl_oe, w_i2c2_scl, w_i2c2_sda_oe, w_i2c2_sda;
assign TFT_SCL = (w_i2c2_scl_oe) ? 1'b0 : 1'bz;
assign TFT_SDA = (w_i2c2_sda_oe) ? 1'b0 : 1'bz;
assign w_i2c2_scl = TFT_SCL;
assign w_i2c2_sda = TFT_SDA;

[ We're successfully communicating with something on I2C3. ]

Does this indicate to you that we are correctly using the open drain primitive for said pins?

Thanks!

RE: I2C Slave Communication Hanging - Added by Daniel Vincelette over 6 years ago

Jared,

To me it looks like the scope capture you've shown should be correct and as you've said there shouldn't be a stop condition between the first and second byte, which is why the clock stays low. Though the second byte isn't ack'ed by the slave, which is confusing.

Do you have a capture of a successful read request between the Sitronix and Aardvark you could share? I would like to compare what happens between the first 2 bytes and then see if the 2 byte is ack'ed.

Dan

RE: I2C Slave Communication Hanging - Added by Jared Kirschner over 6 years ago

Dan,

We'll be conducting investigations this afternoon with both the Linux I2C kernel driver and a bit-bang I2C user-space module we just finished writing. As a part of those investigations, we'll send along a read request capture between the Aardvark and the Sitronix IC.

Thanks,
Jared

RE: I2C Slave Communication Hanging - Added by Jared Kirschner over 6 years ago

Dan,

I've attached 2 scope shots between the Aardvark as master and the ST1633 as slave:
1. A single byte write with a value of 0x01
2. A subsequent single byte read, which we expect to return 0x04 (and does, in this trace)

This transaction looks correct to us (i.e., the Aardvark does not indicate an error, and the stop condition looks valid).

Note: in these traces, the second byte on the read is also not ACK'ed. I think this is correct behavior. When a master-receiver is reading data, I think it indicates to the slave that it is done reading by NACKing on the last byte it wants to read. The difference here is that, when the Aardvark is a master, the stop condition is valid. When the MitySOM is the master (in scope_3.png earlier in the thread), the stop condition is invalid. In both cases, I believe the last write was 0x04 (so the same register is being read on the 1-byte read).

Any thoughts as to what might be going on?

We'll be testing with bit-bang I2C tomorrow to see if the behavior changes when we bypass the I2C kernel driver.

Thanks,
Jared

aardvark_to_st1633_write_addr_0x04.png (35.2 KB) aardvark_to_st1633_write_addr_0x04.png First transaction: write 0x04 to slave 0x55
aarvark_to_st1633_read_single_byte.png (34.9 KB) aarvark_to_st1633_read_single_byte.png Second transaction: read one byte from slave 0x55

RE: I2C Slave Communication Hanging - Added by Jay Petrie over 6 years ago

We appear to have finally discovered the root cause of our issue. We actually had a self-inflicted bug where the SCL line on this one port was defined as "output" rather than "inout". Thus, we didn't really have the "open drain" we thought we had.

I wish there really was a simple primitive to define an open drain. If there had been, the mistake would have very much stood out.

RE: I2C Slave Communication Hanging - Added by Michael Williamson over 6 years ago

So I didn't respond right away as you are using verilog and I am more of a VHDL guy, but there is an open drain primitive called OPNDRN.

You can see an example of it inferred in VHDL here (bottom of file) for I2C routing like you are doing.

https://support.criticallink.com/gitweb/?p=mitysom-5cs/mitysom_5csx_dev_board.git;a=blob;f=dev_5csx_h6_42a_displayport_bcon/vhdl/dev_5csx_h6_42a_top.vhd;h=9f12cde9f15d98066ee72658c7580fdb38df69ae;hb=refs/heads/bcon_adapter

I suspect there is an equivalent instantiation in verilog, I just don't know what it is off hand.

-Mike

    (1-12/12)
    Go to top
    Add picture from clipboard (Maximum size: 1 GB)