25 Replies Latest reply on Sep 16, 2019 3:21 PM by pjestico

    Possible faulty Azure Sphere device

    pjestico

      I have recently received an Avnet Azure Sphere device and it appears to be faulty.

      It has not been claimed or used to create a tenant by me previously however I am unable to create my first tenant

      I know the commands are good and have posted the issue with Microsoft but support is slow.

       

      Is there a way to determine the cause of the issue and either resolve or replace the device?

        • Re: Possible faulty Azure Sphere device
          danzima

          peterfenn

          bwilless

          Hi folks, any insights regarding this problem?


          1 of 1 people found this helpful
          • Re: Possible faulty Azure Sphere device
            Fred27

            Creating a tenant is something done in Azure rather than on the device. Are you sure it's the hardware that's the problem?

             

            In my recent roadtest I tried to describe some of the issues that can crop up when getting started with the device and how to work around them.

            AVNET Azure Sphere MT3620 Starter Kit - Review

            • Re: Possible faulty Azure Sphere device
              bwilless

              Hi Peter,

              Can you give us more details on the problem you're seeing?  It sounds like maybe your PC is not able to communicate with your device.  Is that what you're encountering?

               

              Have you reviewed and worked through the troubleshooting guide here:http://avnet.me/Azure_Sphere_Install_Troubleshooting.  If not, that's a good place to start. 

               

              Let us know what you find,

               

              Brian

              1 of 1 people found this helpful
              • Re: Possible faulty Azure Sphere device
                pjestico

                Thanks for all the suggestions.

                I have been through the documentation and the troubleshooting guide suggested by Brian and the guide suggested by Fred but all is setup as suggested.

                 

                The issue I have is that the device is now out of the box.

                I have setup my AAD properly and also created an account to use with the Global Admin role applied.

                When I issue the azsphere login command with that account I get the following which is expected as I haven't created the tenant yet but proves the account is working:

                 

                warn: No Azure Sphere tenants found under the selected AAD user.

                Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.

                Command completed successfully in 00:00:10.6725886.

                 

                Next I do the logical thing and try to create my tenant but this is what is returned:

                 

                error: Failed to create an Azure Sphere tenant with name 'AzureSphere'.

                Trace ID: acc4be19-6a9f-4133-9fde-073e92f6100b

                error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.

                error: Command failed in 00:00:19.5623966.

                 

                I have tried using names with spaces, without spaces, with quotes and without and tried random names in case I have hit some reserved term.

                I have also tried using the force option and added verbose to see if the logging tells me anything.

                Nothing results in a different outcome and the logging just gives me a very long winded "400 Bad Request" result.

                 

                I can run the command "azsphere dev show-attached" and get a result describing the device ID and its IP so the comms to the device seems good.

                 

                If I log into a different Azure account that I know does already have a tenant created and run the following command I have seen Microsoft suggesting tells us if the device has been pre claimed all seems good.

                azsphere dev show-ota-status

                Your device is running Azure Sphere OS version 19.07.

                error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.

                error: Command failed in 00:00:03.1368024.

                 

                Based on Microsofts suggestions it seems this confirms its not claimed.

                 

                So what next?

                You can only claim a device once and I haven't claimed it anywhere and it seems the above confirms that.

                You can only use the device to create one tenant, I haven't created a tenant yet. The account where I had access to a tenant was seeing a tenant created by someone else.

                The account with access to a tenant is in a completely different AAD than the one where I'm unable to create a tenant, so different AADs and different accounts.

                 

                The reason I suspected hardware is due to MS saying that you can only create a tenant with a device attached and you can only use a device to create a single tenant.

                That to me means that something in the devices identity is used to create that tenant and if the device isn't responding properly then that's going to cause a Bad Request response, all supposition but the best I have right now.

                 

                I have already run an "azsphere dev recover" to ensure the OS version on the device is up to date and that seems to succeed.

                I have checked my AAD config and can see that the 2 Azure Sphere Applications (API and Utility) are listed.

                 

                In short I have now run out of places to look and wonder if anyone else can suggest anything or if we think it is a hardware issue does anyone know how Azure Sphere returns work?

                2 of 2 people found this helpful
                  • Re: Possible faulty Azure Sphere device
                    clem57

                    Thanks for the detailed response.

                     

                    Firstly, while logged into each id you used or created, issue:

                    • azsphere tenant list
                    • azsphere tenant selected

                     

                    Then drop in the results here. What we need is the Guid's and names associated with them

                    Thanks,

                    Clem

                     

                      • Re: Possible faulty Azure Sphere device
                        clem57

                        BTW, I worked with multiple devices and learned the hard way how to deal with issues.

                        • Re: Possible faulty Azure Sphere device
                          pjestico

                          For the account with a tenant I get the following:

                           

                          azsphere tenant list

                          ID                                   Name

                          --                                   ----

                          821b1fe3-aaf7-41bb-9816-ef5b7430124f Emily Sphere 1

                           

                          azsphere tenant show-selected
                          Default Azure Sphere tenant ID is '821b1fe3-aaf7-41bb-9816-ef5b7430124f'.

                          For the account without a tenant I get the following:

                           

                          azsphere tenant list
                          warn: No Azure Sphere tenants found under the selected AAD user.
                          warn: Use 'azsphere login' to change AAD user, or create a new Azure Sphere tenant using 'azsphere tenant create'.
                          error: You do not have access to any Azure Sphere tenants.
                          error: Command failed in 00:00:01.3978586.

                           

                          azsphere tenant show-selected
                          No default Azure Sphere tenant is selected. If only one Azure Sphere tenant is available
                          using your current AAD login, it will automatically be used. Otherwise, use this command
                          to select a default Azure Sphere tenant: 'azsphere tenant select --tenantid <tenant ID>'
                          Command completed successfully in 00:00:00.3617572.

                           

                          I would love to get my hands on a few more devices and work through how this all hangs together but when you get a failure to launch on your first device it doesn't inspire you to invest further.

                          Hopefully we can get over this hurdle and I will procure more devices.

                            • Re: Possible faulty Azure Sphere device
                              clem57

                              OK, login to the one with the tenant and device attached.

                              Run this:

                              azsphere get-support-data --output logs.zip

                               

                              The log will have debugging information which can show what is true. Post it here and I can interpret it.

                                • Re: Possible faulty Azure Sphere device
                                  clem57

                                  The azuresphere_securityservice_213 txt file in zip is the one to look for...

                                    • Re: Possible faulty Azure Sphere device
                                      pjestico

                                      Clem,  really appreciate your help with this.

                                      Command executed and zip file generated.

                                      There was file called "AzureSphere_SecurityService_133.log" and the contents are below:

                                       


                                      AZURE SPHERE SECURITY SERVICE CONFIGURATION:
                                      -> Using tenant ID: '821b1fe3-aaf7-41bb-9816-ef5b7430124f'.
                                      -> Using device ID: '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.

                                      AVAILABLE TENANTS:
                                      -> [821b1fe3-aaf7-41bb-9816-ef5b7430124f] 'Emily Sphere 1'

                                      OVER-THE-AIR CONFIGURATION:
                                      -> Could not get over-the-air configuration: Device with ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3' could not be found.

                                      OVER-THE-AIR UPDATE STATUS:
                                      -> Device OS version(s): 19.07

                                      Could not gather details from the Azure Sphere Security Service: Getting images for device failed.
                                         at Microsoft.Azure.Sphere.PublicApi.Clients.PublicApiClient.<ListImagesToBeDeployedToDeviceAsync>d__21.MoveNext()
                                      --- End of stack trace from previous location where exception was thrown ---
                                         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                                         at Microsoft.Azure.Sphere.Tools.SecurityServiceOperations.DeviceManager.<GetSoftwareAsync>d__4.MoveNext()
                                      --- End of stack trace from previous location where exception was thrown ---
                                         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                                         at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetCloudImagesAsync>d__18.MoveNext()
                                      --- End of stack trace from previous location where exception was thrown ---
                                         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                                         at Microsoft.Azure.Sphere.Tools.Common.OsVersionChecker.<GetTargetedOsVersionAsync>d__15.MoveNext()
                                      --- End of stack trace from previous location where exception was thrown ---
                                         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
                                         at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<LogOtaStatusAsync>d__30.MoveNext()
                                      --- End of stack trace from previous location where exception was thrown ---
                                         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                                         at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
                                         at Microsoft.Azure.Sphere.Tools.AzSphere.GetSupportDataCommand.<GatherCloudDataAsync>d__25.MoveNext()

                                        • Re: Possible faulty Azure Sphere device
                                          clem57

                                          Here is what I see from above:

                                          1. Your device is not ready for side loading applications. Therefore try this:
                                            azsphere device prep-debug
                                          2. Your device appears to be claimed by tenant 'Emily Sphere 1'. This is good as far as I can tell.
                                          3. If #1 works, try to load the blinky or hello world scripts to verify functioning.

                                          Let me know how things turn out.

                                          Clem

                                            • Re: Possible faulty Azure Sphere device
                                              pjestico

                                              Thanks for the update but I haven't run a claim as I don't want to use the device there as that's work and the device is for personal testing.

                                              I have specifically avoided claiming until I have this AAD setup with a tenant and will then claim the device into there.

                                               

                                              Is this suggesting the device has been associated irrevocably with that tenant just by logging into that account?

                                               

                                              To test this further I have run the suggested command to move the device into dev mode and that fails as below suggesting I need to claim the device first.

                                              This would appear to support the fact the device hasn't been claimed and certainly not into the 'Emily Sphere 1' tenant.

                                               

                                              I am at risk of sounding like I'm arguing with you which I'm not as I appreciate your knowledge and help but what I know I have done and the evidence I seem to revealing suggests this device isn't claimed but yet wont create the tenant in my AAD into which I will claim it.

                                              More useful logging from these devices when you use the verbose option is going to be essential before I think about proposing this device for real world solutions.

                                              I don't fancy trying to unpick this type pf problem with an SLA clock ticking and the current level of information to work with.

                                               

                                              azsphere login
                                              The selected Azure Sphere tenant 'Emily Sphere 1' (821b1fe3-aaf7-41bb-9816-ef5b7430124f) will be retained.
                                              Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.
                                              Command completed successfully in 00:00:05.5350969.

                                               

                                              azsphere device prep-debug
                                              Getting device capability configuration for application development.
                                              Downloading device capability configuration for device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.
                                              error: Failed to create device capability configuration for device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'. Ensure that you are logged in to the correct Azure Sphere tenant ('azsphere login'), that the device is claimed ('azsphere device claim') to that tenant, and try again.
                                              Trace ID: c3542e70-676e-42e6-ac8d-a1fcb9264937
                                              error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.
                                              error: Command failed in 00:00:02.8190509.

                                                • Re: Possible faulty Azure Sphere device
                                                  clem57

                                                  OK, Seems to confirm the claim is not done. So try to create a tenant with  the board plugged in on the correct AAD. If that works, you can claim the device.

                                                  As an aside, here is some suggested debugging https://docs.microsoft.com/en-us/azure-sphere/install/troubleshoot-installation .

                                                   

                                                  Please read carefully first: If the claim fails again, carefully take the device id from the zip file we did, and use it without the device attached using the additional parameter. If you make a mistake, it will record that as a device and I see no way of correcting it. So you end up with phantom device!

                                                   

                                                  -i, --deviceid GUIDSpecifies the ID of the device to claim. If omitted, azsphere claims the attached device.

                                                   

                                                  Clem

                                                    • Re: Possible faulty Azure Sphere device
                                                      pjestico

                                                      I know that doesn't work as that's what kicked this thread off, when I found that action was failing as below:

                                                       

                                                      azsphere tenant create --name "AzureSphere Testing" --force -v
                                                      Azure Sphere Utility version 19.5.4.53731
                                                      Copyright (C) Microsoft Corporation. All rights reserved.

                                                      Start time (UTC): Wednesday, 11 September 2019 14:18:49
                                                      verbose: Creating new Azure Sphere tenant with name 'AzureSphere Testing'.
                                                      verbose: No Azure Sphere tenants found under the selected AAD user.
                                                      verbose: Successfully logged in with the selected AAD user. This authentication will be used for subsequent commands.
                                                      verbose: No SerialSlipToTunService port is set in the registry; defaulting to 56665.
                                                      verbose: Using attached device ID '389B8BE2FF2F649E3438BD1F32DE21F4DBCEFA4CF9ABF2E2D80B92484ADE5EF36A9880FBA7DC3D562F48C875F5ABB54D367FAE623F4BDBAEB54FF2675F1C70F3'.
                                                      error: Failed to create an Azure Sphere tenant with name 'AzureSphere Testing'.
                                                      Trace ID: 4837d2d7-2e04-4de2-a0bf-3fa46f5f0ec2
                                                      Received HTTP response: 400 (Bad Request)
                                                      -> Endpoint: POST https://prod.core.sphere.azure.net/tenants
                                                      -> Content Type: NULL
                                                      error: An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support. Trace ID: 4837d2d7-2e04-4de2-a0bf-3fa46f5f0ec2

                                                        • Re: Possible faulty Azure Sphere device
                                                          clem57

                                                          This is exactly as this post https://social.msdn.microsoft.com/Forums/en-US/caea57b7-9f9d-45c6-841b-8a2e718393aa/changing-directory-and-tenant-before…

                                                          I suggest you follow this thread for the answer since Microsoft has to fix it.

                                                          I am sorry we could not get the ball rolling,

                                                          Clem

                                                            • Re: Possible faulty Azure Sphere device
                                                              pjestico

                                                              Unfortunately that post doesn't reflect what's going on here as it seems that device had been used for a one way action previously which mine hasnt.

                                                              I had reviewed that one before posting here and was hopeful when I started reading it.

                                                               

                                                              However I have contacted the Microsoft Azure Sphere email address listed there for assistance.

                                                              I will update this thread if I ever get a clear answer in case its helpful to others in a similar situation.

                                                                • Re: Possible faulty Azure Sphere device
                                                                  bwilless

                                                                  Peter,

                                                                   

                                                                  Can you confirm that you're able to communicate with the Azure Sphere device and read the device ID?  I think I remember seeing a post from you that showed you could.

                                                                   

                                                                  azsphere device show-attached

                                                                   

                                                                  I don't think that the device needs to be claimed to execute this command.  If you are able to read the device ID from the device, then the tenant creation issue you're having is likely on Microsoft's side. 

                                                                   

                                                                  I think you suspect that the device has been previously claimed.  This is highly unlikely.  We manufactured the 20,000 free kits all together and as far as I know, there were not any devices removed from the production run.  All these devices were manufactured, tested and shipped directly to distribution centers.

                                                                   

                                                                  Brian

                                                                  1 of 1 people found this helpful
                                                                    • Re: Possible faulty Azure Sphere device
                                                                      pjestico

                                                                      Brian,

                                                                       

                                                                      Yes I can confirm that I get a response from the device that contains a device ID, IP address and connection path.

                                                                       

                                                                      Peter

                                                                        • Re: Possible faulty Azure Sphere device
                                                                          bwilless

                                                                          Thanks Peter,

                                                                           

                                                                          That's what I thought.  I've never been through the create tenant process myself, so I gave it a try this morning using a personal email address.  I created a new Azure account using this link in the documentation.  That took me to the interface that signed me up for a new Azure account, created the Azure Active Directory in my account and created my tenant user. 

                                                                           

                                                                          I can login to the new account using the >"azsphere login" command and the new credentials.  But when I go to create the tenant I get the same error you're getting.  I don't have a unclaimed device with me, so I was using a previously claimed device.  I have a fresh unclaimed device at home and I'll try again on Monday.  I'll post the results here on Monday.  I suspect I'll run into the same issue as you, but I need to try it with a fresh device.

                                                                           

                                                                          Brian

                                                                          1 of 1 people found this helpful
                                                                            • Re: Possible faulty Azure Sphere device
                                                                              pjestico

                                                                              Thanks Brian, look forward to hearing how things go on Monday.

                                                                                • Re: Possible faulty Azure Sphere device
                                                                                  pjestico

                                                                                  Hi Brian, did you get a chance to try a tenant creation today?

                                                                                    • Re: Possible faulty Azure Sphere device
                                                                                      bwilless

                                                                                      Hi Peter,

                                                                                       

                                                                                      I just used my new device (never claimed before) to create a new tenant.

                                                                                       

                                                                                      Just to document what I did . . .

                                                                                       

                                                                                      I wanted to use a personal email address to setup a new Azure account and new Azure Sphere Tenant

                                                                                       

                                                                                      • I followed this link to get started:  https://account.azure.com/organization
                                                                                        • I used my personal email address
                                                                                        • This page helped me create a <myDomain>.onmicrosoft.com domain
                                                                                      • I followed the link to create a new Azure Pay-as-you-Go account
                                                                                      • The Azure Account was created that included a new Azure Active Directory and a new user that matched my new domain
                                                                                        • The user was granted "Global Administrator" rights
                                                                                      • From the Azsphere command line interface
                                                                                        • I plugged in my fresh device
                                                                                        • Upgraded the os: azsphere device recover
                                                                                        • I created the Tenant: azsphere tenant create --name <myNewTenantName>
                                                                                          • I was prompted to login to my tenant, I used my new user's login
                                                                                            • <MyUser>@<MyDomain>.onmcirosoft.com

                                                                                       

                                                                                      So it looks like your device may have been claimed before, although I have no idea how that could have happened.  My recommendation is that you contact the site where you got your device to get your device replaced. 

                                                                                       

                                                                                      Brian

                                                                                      1 of 1 people found this helpful
                                                                                        • Re: Possible faulty Azure Sphere device
                                                                                          pjestico

                                                                                          Thanks Brian, really appreciate your help.

                                                                                          The process I followed to set my domain up and account up and create the tenant matches the one you followed.

                                                                                           

                                                                                          I bought the device through Farnell and sent a note to their support to start the process last week.

                                                                                          I will chase them up now I have your results.

                                                                                           

                                                                                          Assuming I get the device replaced and the device works as expected I will update this thread again.

                                                                                           

                                                                                           

                                                                                          Peter

                                                                                          1 of 1 people found this helpful