12 Replies Latest reply on Jun 12, 2018 7:43 PM by zedhed

    eMMC not found on PicoZed Rev E board (Rev C board works)

    mattriggs
      I am checking out the PicoZed Rev E boards which will be replacing the Rev C boards we've been successfully using on our product for several years. We boot out of the QSPI flash, and it appears Rev E is working except that the eMMC device isn't detected or mounted at boot by the SDHCI driver.
       
      Here is the relevant part of dmesg for the Rev C board:
       
      sdhci: Secure Digital Host Controller Interface driver
      sdhci: Copyright(c) Pierre Ossman
      sdhci-pltfm: SDHCI platform and OF driver helper
      sdhci-arasan e0101000.sdhci: No vmmc regulator found
      sdhci-arasan e0101000.sdhci: No vqmmc regulator found
      mmc0: SDHCI controller on e0101000.sdhci [e0101000.sdhci] using ADMA
      ledtrig-cpu: registered to indicate activity on CPUs
      hidraw: raw HID events driver (C) Jiri Kosina
      usbcore: registered new interface driver usbhid
      usbhid: USB HID core driver
      TCP: cubic registered
      NET: Registered protocol family 17
      can: controller area network core (rev 20120528 abi 9)
      NET: Registered protocol family 29
      can: raw protocol (rev 20120528)
      can: broadcast manager protocol (rev 20120528 t)
      can: netlink gateway (rev 20130117) max_hops=1
      Registering SWP/SWPB emulation handler
      /opt/petalinux-v2014.4-final/components/linux-kernel/xlnx-3.17/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
      ALSA device list:
        No soundcards found.
      Freeing unused kernel memory: 4452K (4061c000 - 40a75000)
      INIT: version 2.88 booting
      usb 1-1: new high-speed USB device number 2 using zynq-ehci
      Creating /dev/flash/* device nodes
      mmc0: BKOPS_EN bit is not set
      mmc0: new high speed MMC card at address 0001
      mmcblk0: mmc0:0001 MMC04G 3.52 GiB
      usb 1-1: new high-speed USB device number 3 using zynq-ehci
      mmcblk0boot0: mmc0:0001 MMC04G partition 1 16.0 MiB
      mmcblk0boot1: mmc0:0001 MMC04G partition 2 16.0 MiB
      mmcblk0rpmb: mmc0:0001 MMC04G partition 3 128 KiB
       mmcblk0: p1 p2
       mmcblk0boot1: unknown partition table
       mmcblk0boot0: unknown partition table
      random: dd urandom read with 26 bits of entropy available
      usb 1-1: new high-speed USB device number 4 using zynq-ehci
       
      And here is the relevant bit from the Rev E board:
       
      sdhci: Secure Digital Host Controller Interface driver
      sdhci: Copyright(c) Pierre Ossman
      sdhci-pltfm: SDHCI platform and OF driver helper
      sdhci-arasan e0101000.sdhci: No vmmc regulator found
      sdhci-arasan e0101000.sdhci: No vqmmc regulator found
      mmc0: SDHCI controller on e0101000.sdhci [e0101000.sdhci] using ADMA
      ledtrig-cpu: registered to indicate activity on CPUs
      hidraw: raw HID events driver (C) Jiri Kosina
      usbcore: registered new interface driver usbhid
      usbhid: USB HID core driver
      TCP: cubic registered
      NET: Registered protocol family 17
      can: controller area network core (rev 20120528 abi 9)
      NET: Registered protocol family 29
      can: raw protocol (rev 20120528)
      can: broadcast manager protocol (rev 20120528 t)
      can: netlink gateway (rev 20130117) max_hops=1
      Registering SWP/SWPB emulation handler
      /opt/petalinux-v2014.4-final/components/linux-kernel/xlnx-3.17/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
      ALSA device list:
        No soundcards found.
      Freeing unused kernel memory: 4452K (4061c000 - 40a75000)
      INIT: version 2.88 booting
      usb 1-1: new high-speed USB device number 2 using zynq-ehci
       
      The SDHCI driver comes up in both cases, but only the Rev C board discovers the eMMC device. On the Rev C board, /dev/mmcblk0 is present after boot, but I can't find anything similar on the Rev E board. I haven't changed our software at all yet, so here is the relevant device tree info:
       
      From zynq-7000.dtsi (not modified by me):
       
      sdhci1: sdhci@e0101000 {
        compatible = "arasan,sdhci-8.9a";
        status = "disabled";
        clock-names = "clk_xin", "clk_ahb";
        clocks = <&clkc 22>, <&clkc 33>;
        interrupt-parent = <&intc>;
        interrupts = <0 47 4>;  
        reg = <0xe0101000 0x1000>;
      };
       
      and from pcw.dtsi:
       
      &sdhci1 {
        clock-frequency = <50000000>;
        status = "okay";
      };
       
      I'm not sure how to proceed with debugging this issue -- is there something I need to do in software to support Rev E's new eMMC device?
        • Hi Matt, 
          jafoste4

          Hi Matt, 

          Are you aware of the Rev E changes? In the Rev E SOM we removed the MUX that connected between either the eMMC or the Pmod. It is now by default set to eMMC. Did you take this into consideration?


          Thanks,

          Josh

          • Hi Josh-
            mattriggs

            Hi Josh-

            I appreciate the response!

            Yes, I am aware of the Rev E changes, but I was under the impression that there would be no change to the software, since we were using the mux in its default state (enabling the eMMC) and want to continue to use the eMMC in the same way.

            I know the eMMC part changed from a v4.41 to v5 interface and from 4 GB to 8 GB of capacity, but I'm not aware of any software changes I need to support that.

            Am I missing something?

            • Still looking for assistance
              mattriggs

              I am still struggling with this issue, and hoping someone has other suggestions. We've done some investigation since Monday, and here's a summary of what we know:

              •   The same firmware image running on a Rev C board can access the eMMC
              •   The QSPI (which we boot from) is working, so the power rails and reset line should be working for the eMMC too
              •   During boot, we see several clock pulses on the clock line to the eMMC, but all other IO lines are high with no transisitions
              •   We have tried two Rev E boards now and they behave the same
              •   On Rev C boards the mux defaulted to select the eMMC, and on the Rev E board the jumpers are hardwired to select the eMMC, so there should be no additional actions needed to use the eMMC

              Our board house is nearly out of Rev C boards, so we are running out of time to resolve this issue -- any help or suggestions is greatly appreciated! Thanks!

              • Hello Matt,
                jafoste4

                Hello Matt,

                I will speak with out Software Engineering group first thing tomorrow morning and get back to you.

                Thank,

                Josh

                • Hi Matt,
                  narrucmot

                  Hi Matt,

                  I am working to track down why your Linux image built for the Rev C PicoZed SOM won't boot on the Rev E SOM.  In the meantime would you be willing to test either of our posted BSPs for the PZ7020 and FMC v2 carrier that were built with the PetaLinux and Vivado 2017.4 tools and tested on my Rev E SOM?:
                  http://zedboard.org/sites/default/files/design/pz7020_fmc2_2017_4_0.zip

                  http://zedboard.org/sites/default/files/design/pz7020_fmc2_sd_oob_2017_4_0.zip

                   

                  Thanks!

                  Tom

                  • RE: eMMC not found on PicoZed Rev E board (Rev C board works)
                    zedhed

                    Hi Matt,

                    Any update on this issue?  Were you able to try one of the BSPs that Tom mentioned?

                    http://zedboard.org/sites/default/files/design/pz7020_fmc2_2017_4_0.zip

                    When I boot a PicoZed Rev. E using the 2017.4 BSP that Tom mentioned, I can see the system comes up and that the eMMC device is found:

                    sdhci: Secure Digital Host Controller Interface driver
                    sdhci: Copyright(c) Pierre Ossman
                    sdhci-pltfm: SDHCI platform and OF driver helper
                    mmc0: SDHCI controller on e0100000.sdhci [e0100000.sdhci] using ADMA
                    mmc1: SDHCI controller on e0101000.sdhci [e0101000.sdhci] using ADMA
                    ledtrig-cpu: registered to indicate activity on CPUs
                    usbcore: registered new interface driver usbhid
                    usbhid: USB HID core driver
                    NET: Registered protocol family 10
                    sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
                    NET: Registered protocol family 17
                    can: controller area network core (rev 20120528 abi 9)
                    NET: Registered protocol family 29
                    can: raw protocol (rev 20120528)
                    can: broadcast manager protocol (rev 20161123 t)
                    can: netlink gateway (rev 20130117) max_hops=1
                    Registering SWP/SWPB emulation handler
                    hctosys: unable to open rtc device (rtc0)
                    of_cfs_init
                    of_cfs_init: OK
                    ALSA device list:
                      No soundcards found.
                    Freeing unused kernel memory: 1024K (c0900000 - c0a00000)
                    mmc1: new high speed MMC card at address 0001
                    mmcblk1: mmc1:0001 Q2J55L 7.09 GiB
                    mmcblk1boot0: mmc1:0001 Q2J55L partition 1 16.0 MiB

                    INIT: mmcblk1boot1: mmc1:0001 Q2J55L partition 2 16.0 MiB
                    mmcblk1rpmb: mmc1:0001 Q2J55L partition 3 4.00 MiB
                    mmcblk1: p1 p2
                    version 2.88 booting

                    If that same BSP works on your Rev. E then that would give us a data point to begin dividing and conquering this issue.

                    Can you elaborate on which tools you are using for Vivado and PetaLinux?

                    Best Regards,

                    -Kevin

                    • Kevin and Tom-
                      mattriggs

                      Kevin and Tom-

                      Thanks for getting back to me -- our board house was able to locate enough Rev C SOMs to avert the immediate crisis, so I was pulled away dealing with suddenly-more-immediate issues, but I'm back and looking at this again.

                      The product we use picoZed on was developed in 2015, so we were using PetaLinux 2014.4 running in a Ubuntu 2014.4 VM. I tried installing the BSP Tom linked in the 2014.4 environment but it failed almost immedately -- I've now got Petalinux 2017.4 installed in a Ubuntu 16.04.1 VM, and when I got the project created I saw the images directory containing pre-built images. I was able to load the BOOT_QSPI.bin file using Xilinx SDK 2014.4, and it came up without issue (this is on the Rev E SOM attached to our custom carrier board).

                      Interestingly mmc0 isn't present, but an mmc1 device is, here's some relevant snips from dmesg:

                      <snip>
                       
                      Xilinx Zynq CpuIdle Driver started
                      sdhci: Secure Digital Host Controller Interface driver
                      sdhci: Copyright(c) Pierre Ossman
                      sdhci-pltfm: SDHCI platform and OF driver helper
                      mmc0: SDHCI controller on e0100000.sdhci [e0100000.sdhci] using ADMA
                      mmc1: SDHCI controller on e0101000.sdhci [e0101000.sdhci] using ADMA
                      ledtrig-cpu: registered to indicate activity on CPUs
                      usbcore: registered new interface driver usbhid
                       
                      <snip>
                       
                      ALSA device list:
                        No soundcards found.
                      Freeing unused kernel memory: 1024K (c0900000 - c0a00000)
                      INIT: mmc1: new high speed MMC card at address 0001
                      mmcblk1: mmc1:0001 Q2J55L 7.09 GiB
                      mmcblk1boot0: mmc1:0001 Q2J55L partition 1 16.0 MiB
                      mmcblk1boot1: mmc1:0001 Q2J55L partition 2 16.0 MiB
                      mmcblk1rpmb: mmc1:0001 Q2J55L partition 3 4.00 MiB
                       mmcblk1: p1 p2
                      version 2.88 booting
                      Starting udev
                       
                      <snip>
                       
                      And here is my /dev directory:
                       
                        root@pz7020-fmc2-2017-4:~# ls /dev
                        block               mtd2                tty0                tty43
                        char                mtd2ro              tty1                tty44
                        console             mtd3                tty10               tty45
                        cpu_dma_latency     mtd3ro              tty11               tty46
                        disk                mtdblock0           tty12               tty47
                        fd                  mtdblock1           tty13               tty48
                        full                mtdblock2           tty14               tty49
                        gpiochip0           mtdblock3           tty15               tty5
                        gpiochip1           network_latency     tty16               tty50
                        gpiochip2           network_throughput  tty17               tty51
                        i2c-0               null                tty18               tty52
                        iio:device0         port                tty19               tty53
                        initctl             psaux               tty2                tty54
                        input               ptmx                tty20               tty55
                        kmsg                pts                 tty21               tty56
                        log                 ram0                tty22               tty57
                        loop-control        ram1                tty23               tty58
                        loop0               ram10               tty24               tty59
                        loop1               ram11               tty25               tty6
                        loop2               ram12               tty26               tty60
                        loop3               ram13               tty27               tty61
                        loop4               ram14               tty28               tty62
                        loop5               ram15               tty29               tty63
                        loop6               ram2                tty3                tty7
                        loop7               ram3                tty30               tty8
                        mem                 ram4                tty31               tty9
                        memory_bandwidth    ram5                tty32               ttyPS0
                        mmcblk1             ram6                tty33               urandom
                        mmcblk1boot0        ram7                tty34               vcs
                        mmcblk1boot1        ram8                tty35               vcs1
                        mmcblk1p1           ram9                tty36               vcsa
                        mmcblk1p2           random              tty37               vcsa1
                        mmcblk1rpmb         shm                 tty38               vga_arbiter
                        mtab                snd                 tty39               watchdog
                        mtd0                stderr              tty4                watchdog0
                        mtd0ro              stdin               tty40               xdevcfg
                        mtd1                stdout              tty41               zero
                        mtd1ro              tty                 tty42
                         
                        It looks like I've reproduces what you were seeing Kevin, with the eMMC device now at mmc1 instead of mmc0.
                         
                        Do you think it's possible to get this functionality working with the 2014.4 version of petalinux? We've got an installed base of this product already using Rev C boards with firmware built in 2014.4; it would be a real effort to test and verify a build using the new tools, especially since we'd want whatever firmware I build to be compatible with the Rev C SOMs as well.
                         
                        I appreciate the help!
                         
                        -Matt
                      • I still don't have an answer
                        mattriggs

                        I still don't have an answer to this, but I noticed that the kernel config option in 2017.4, CONFIG_MMC_SDHCI_OF_XILINX_PS, isn't available in 2014.4, which uses CONFIG_MMC_SDHCI_OF_ARASAN. Everything else looks similar, and I modified my device tree to enable both SDHCI devices in the same way Tom's example does -- still nothing.

                        At this point, it looks like I'll need to migrate my code to 2017.4 in order to use the Rev E boards. Please let me know if I'm mistaken.

                        -Matt

                        • RE: I still don't have an answer
                          zedhed

                          Hi Matt,

                          Looking through my old notes from hockeyman on this, based upon development work done with older PetaLinux builds, the following were recommended when using eMMC devices:

                          1) Set device tree property "broken-mmc-highspeed" in the sdhci node.

                          2) Set the "max-frequency" property to 25000000.

                          I don't have 2014.4 tools setup to experiment with these settings, but perhaps you can check to see if that enables you to continue using 2014.4 builds on Rev. E?

                          Best Regards,

                          -Kevin

                          • &sdhci1 {
                            zedhed

                            &sdhci1 {
                            status = "okay";
                            clock-frequency = <25000000>;
                            xlnx,has-cd = <0x1>;
                            xlnx,has-power = <0x0>;
                            xlnx,has-wp = <0x1>;
                            non-removable;
                            broken-mmc-highspeed;
                            };
                             

                            • Hmmm, fighting the zedboard
                              zedhed

                              Hmmm, fighting the zedboard.org SPAM detection system...

                              The devicetree might look something like the one that I posted above.

                              Best Regards,

                              -Kevin

                              • I found another thread which
                                zedhed

                                I found another thread which could possibly be related if you are depending upon MIO0 for WP signaling:

                                http://zedboard.org/content/sdcard-not-writeable-picozed-rev-e-board-rev-c-board-works

                                Best Regards,

                                -Kevin