21 Replies Latest reply on Nov 9, 2019 8:38 PM by colporteur

    Feedback regarding a companies comments on user expectations

    kas.lewis

      Hello All,

       

      I was looking to get some connect regarding feedback I received from NXP regarding my review of their motor control eval platform. As I do not wish to sway sentiment one way or the other I will leave my comments for after receiving some feedback from the community. I would very much appreciate you open honest comments on both my review as well as the response to this review.

       

      Thanks

      Kas

       

      The review can be found here, the company response/commnet can be found below.

       

       

      Hello,

      I'm really glad that there is some feedback on the product. I'm not going to comment on the documentation, but I'd like to give you some comments on the technical matter.

      Just a general introduction: personally, I have studied electric drives at the university, I have done my PhD in related field and I have intensively worked with sensorless motor control for 4 years now. Even after that I cannot say that I fully understand AC electric drives. And the same you can hear even from established professors - the more you know, the less you understand (or the more you realize how much you don't understand).

      Just to clarify, the S12ZVML-MINIKIT is a tool, which helps educated people to start with a very basic application, which is based on widely known field-oriented control (PMSM) or six-step/block commutation (BLDC). If one doesn't understand how FOC works, I agree it would be very difficult to understand the application. On the other hand, 10 minutes of reading some quick start guide cannot replace 2 semesters and more of electric drives course. I agree that there are gaps in linking the documentation, which is already available, even dedicated to another product, but using the same approach. Normally (if a link is provided correctly) you would follow this document: https://www.nxp.com/docs/en/application-note/AN4912.pdf  or this one https://www.nxp.com/docs/en/application-note/AN4642.pdf. The AN5327 https://www.nxp.com/docs/en/application-note/AN5327.pdf  is providing information on the FOC control implementation, however, it's worth to look for more info, even a basic one at 3-Phase PMSM Control Workshop with NXP's Model-Based Design Toolbox or some external sources.

      My feeling is, that customers expect to have a mobile flight simulator game experience and with that background, to fly a real aircraft with the same results. Maybe an exaggerated statement, but many times not so far from the real expectations.

       

      Why this product is not so easy to use? The application itself has to be open enough to be used by professionals and that has been successful so far. Our experience is, that even if there are some generic use cases, many electric drives have to be threaten case by case. Making a super-generic application can do a good job with a specific demo motor, but especially in automotive applications, more fine-tuning and maybe some additional algorithms need to be added. EDIT: Therefore, any "smart feature" simplifying the user experience would make it hard or even impossible to add these advanced features.

      For a new-bee, a motor is just spinning, slow or fast, left or right and that is good enough. For a professional, low acoustic noise, high efficiency or high dynamics make the difference.

       

      Ad SOFTWARE

      From your testing, I can see that you are not familiar with the open-loop start-up sequence of sensorless drives. Why? Your sensorless settings are using 200 RPM as the Mergin speed 1 and the same setting for Merging speed 2. In the default settings, these are set to 150 RPM and 300 RPM. There are also some other settings changed especially in the speed loop control, which may cause an unexpected behavior. If this was the test case, then it was an example of misused settings. It would be fair to mention that random tuning of random values and expecting the application to run perfectly fine is just not the approach an engineer would do with such a development platform.

      Testing the application at 100 RPM indicates that you are forcing the application to run in the region it was not designed for. In model-based sensorless operation, the field-oriented control works with BEMF observer roughly from 10% of the base (nominal) speed. In the MCAT, sensorless operation starts from the Merging speed 2, which is 200 RPM by your setting. Below this speed, the motor is designed just to speed up, not to operate continuously. And I would be very careful about saying that 200 RPM is perfect - based on my personal experience with the Linix motor, it would be at least 250 RPM, but it depends on the current limit, load and speed ramp. Errors thrown during the tests within the open-loop to sensorless transition are most likely connected to wrong settings of the open-loop start-up and the BEMF observer itself. There are many articles at IEEE Xplore covering this topic, in summary: even after years of research, there is no generic solution that works for all the motors. If certain setting works for one, it most likely will not work for other motors. And we are working on a solution which covers as many cases as possible.

       

      Ad HARDWARE

      It's worth to say that S12ZVML-MINIKIT is dedicated to automotive low-cost applications. It demonstrates how an application would look like in terms of almost minimal configuration for up-to 10 Amps of current (for sure, the on-board debugger is not intended to be part of the final application). There is also S12ZVM EVB or the devkit (MTRCKTSPNZVM128|Development Kit S12 MagniV | NXP ), which is probably the one you would be looking for, if more I/O pins and features are needed.

       

      Thank you for the review, I believe it will rise some action items on our side.

        • Re: Feedback regarding a companies comments on user expectations
          shabaz

          Hi Kas,

           

          I don't think the comments you've received from the company are negative, they're perhaps a little defensive, but also informative. However, there's definitely in part a technical communication barrier. For a start, it's wrong to assume that engineers will know all the acronyms - even in the response, they refer to BEMF. Well, I've always called it back-emf, not BEMF. As for other acronyms and motor control algorithms, after a 4-year general university course, I'm still likely to not know industry-specific terms, and only know at a high-level how some things operate. I can google some stuff, read books or papers, but there's a limit, it depends how deeply I need to work in any area. Looking at the screenshots of the software in your review, I would have no clue what SVM or VSI means. If there is 'tooltips' or a glossary for that, that would helps. I think your review reads well, the only slight thing I'd say (and everyone is different, this is just my opinion) is that personally I'm uninterested in the first 25-30% of the review, because although you encountered difficulty determining which documents were useless and which were useful, I'd just prefer you got to the point and told me about the useful information. The information that you found that was wrong or useless could be summed up in a single paragraph, mentioning your experience. It's a story that is less interesting to someone more interested in learning about the technology basically.

          So, personally I'd have liked to have read a brief explanation of what the board and software is doing with FOV, Back EMF, and SVM etc (not to the extent that it is teaching people like in a tutorial, but just at a high level). I can appreciate if you don't have this information, then that's difficult to do.

          The things you mention in the hardware section sounds like very fair criticism, it is weird if they're using those single pins for connections, they would slide off too easily when moving the motor.

          In summary, I don't think the response from the manufacturer is criticising the review, and from what I can your review seems fair too (just that I personally would have cut out some of the first 25% of it, were it me, but everyone is different). You're the perfect target audience for writing the review in my opinion, because you're technically aware, but not so deeply involved in motor control algorithms that you would assume others would be too when writing your review.

          In general I think writers should not overly assume that all target readers (those that have an interest and use for their technology) will be aware of the acronyms and algorithms in detail. Some will, but some won't. Personally I like expanding acronyms, but others think I go over the top with expanding so many acronyms before using them, but it's good not to over-assume what areas the reader will be specialised in. You can't please everyone all of the time : (

          Anyway, long story short, it was a good review, and also it was good to see the comments you pasted too.

          5 of 5 people found this helpful
          • Re: Feedback regarding a companies comments on user expectations
            DAB

            Hi Kas,

             

            I agree with Shabaz, you did a very good review and the response from NXP did not dispute your findings.

             

            The response did correctly point out some of the science and engineering issues with the supplied material. The device was not a simple DC motor control and there are a lot of subtle issues involved with the newer motor control device.

             

            I agree that the tone of the response was a little deprecating, but I do not think it was openly hostile. There was clearly some bias about "non engineers" trying to use the sophisticated and advanced motor control features, but he did have a point about the user needing some decent background in the new motor control techniques.

             

            In the end, he agreed that there are some serious short comings in the current documentation and in the software provided with the kit. I would take that comment as a vindication of your detailed review. I thought most of your observations to be pertinent and valid.

             

            Overall, I would take his response as good feedback on a job well done. There is too little dialog between users, both professional and novice, for this technology. Using motors today is much different from what they used to be.

             

            There is clearly a valid need for better documentation and a simplification of the description of the technology before it will transition into wide spread use.

            If you have helped move that process along, I think you should declare victory for now and hopefully you will get a chance to review some of the updates to see if they are moving in a direction you feel is beneficial.

             

            DAB

            6 of 6 people found this helpful
            • Re: Feedback regarding a companies comments on user expectations
              colporteur

              The companies response doesn't make a good first impression. It reminds me of an instructor chastising students for doing poorly on an exam. It not the instructors fault they did poorly. The students had the material.

               

              I have followed the RoadTest coordinators difficulties in awarding and receiving RoadTest results. The assumption is, there are no problems on the vendor side submitting products for a review.

              The response from the vendor to me, is a brush off. It causes me to question the vendors goal when the product was submitted for a RoadTest exercise. If you think the RoadTest review was just an exercise to say the product was reviewed, then the goal was achieved. Maybe thinking the review would bring about change was your goal and not the vendors.

               

              I feel your review was commendable in light of the difficulties you pointed out. I don't think your suggestions are beyond the realm of being fixed. My confidence level is low that any feedback from RoadTests will bring about positive change. Support cost money. It draws away from profits. So the customer had to struggle. In the end they got it working on their dime and not ours. We care about our customers but not if it costs us money.

               

              Sean

              4 of 4 people found this helpful
              • Re: Feedback regarding a companies comments on user expectations
                Jan Cumps

                That is a reply from an engineer (probably the designer) without filtering by the communication team .

                If I make something that I care for and am proud of, and I get the slightest remarks as feedback, these same things typically spin in my head. Then my communication coach comes along and says: "this is the type of message where you don't reply what you think ".

                She is right. It never leads to something fruitful.

                5 of 5 people found this helpful
                • Re: Feedback regarding a companies comments on user expectations
                  three-phase

                  I thought your review was excellent and found it very interesting to read. At the end of the day a roadtest is a personal appraisal of a product and how well it worked in your particular review scope, that the vendors know before you are selected for the review.

                   

                  Communication in the engineering world is always an issue, especially from engineers working in niche areas needing to explain the workings of their products. All to often engineers are quick to believe that people should understand their explanations and if they don't then it is not the engineer that is wrong, but in reality it probably is. I agree with shabaz that acronyms should always be written in their expanded format first, or a list of acronyms provided with the documentation.

                   

                  Whilst it is great, if the person you are conversing with can understand your field and product, this is often not the case, and sometimes engineers do need to look at how they explain things and reduce their jargon. But, I guess, criticism is sometimes hard to swallow and I wonder if the company saw your review as more of a criticism than you probably intended.

                   

                  As @DAB has said, if your review has opened their eyes a little and made them realise some parts that they can improve on, then your roadtest has achieved more than any of mine ever have.

                   

                  Kind regards.

                  2 of 2 people found this helpful
                    • Re: Feedback regarding a companies comments on user expectations
                      shabaz

                      three-phase  wrote:

                       

                      ..sometimes engineers do need to look at how they explain things and reduce their jargon.

                      Agree : ) it is a major benefit of RoadTests, that it is a good way for people to become world-experts at that : ) Products are complicated enough, it's a nice skill to have. Here people are not shy to even use colouring pencils to draw cute explanations : )

                      I often think "what would Apple do".. even though I don't own an Apple device, I admire how they can really simplify tech so that everyone can understand and use their products.

                      1 of 1 people found this helpful
                    • Re: Feedback regarding a companies comments on user expectations
                      jadew

                      First of all, I found the review to be very good and with a strong bottom line.

                       

                      Likewise, I found the NXP engineer's response to be good and the explanation to make sense. I also think they got the message, that they have to dumb it down a little (where possible) in order to make it more accessible to engineers that don't have the required background, and for which making proper use of the current "advanced mode" would be more difficult.

                       

                      I've been on his side and it's a difficult reply to give to a customer. What they meant to say was "Ok, seems like you don't know what you're doing, but here's some resources to get you on the right path.". People don't take this kind of straight answer easily, so you have to tip-toe around it, which sometimes results in a confusing situation, because the customer is smart and sees through that, so he ends up not knowing how to feel about your reply. The reality is that they had the best intentions at heart, it's just that it's difficult to tell people they don't know stuff.

                       

                      All in all, I think it was a good and fruitful exchange for both parties. They received a fair review that shows them how to improve their solution, and you received some high quality feedback on the areas you can improve upon, if you ever want to drive your motors like a pro.

                       

                      Going forward with the conversation, I don't think it's necessary to pile on, on the things you've already said, because there are only two ways in which that can go:

                      1) They already got it, they agree, and it's pointless to make any further arguments.

                      2) They already got it, but they have decided that you are wrong, in which case you'll put them in the awkward situation in which they have to further explain why you are wrong (could be useful for you).

                       

                      I'm very curious to read about your interpretation of the feedback you received and the answer you had in mind.

                      3 of 3 people found this helpful
                      • Re: Feedback regarding a companies comments on user expectations
                        neuromodulator

                        Communicating with the product company is always a good idea. I've so far roadtested 2 products. One company didn't respond my email but the other did and was quite helpful. They even gave me access to unpublished info of the product (which of course I didn't publish in the roadtest).

                        2 of 2 people found this helpful
                        • Re: Feedback regarding a companies comments on user expectations
                          lui_gough

                          I have read your review and the response comprehensively. It seems that your review did raise a number of issues, all of which I would think are valid observations especially for those who have never used this particular kit or combination before.

                           

                          From my point of view, the response you received while perhaps a bit unpolished, is the exact kind of candid response that I often appreciate. It pays to look beyond the way the thoughts are expressed and see what is trying to be conveyed. From my reading, I feel that the response:

                          • acknowledges that it can be difficult to understand motor control theory (and that's one reason I haven't dived into such RoadTests myself).
                          • acknowledges that their documentation is perhaps not as complete as it could be, while also providing a counterpoint that it is not intended to teach the user the fine art of motor control as it is something of a "dark art".
                          • provides some hints as to other documentation which might be relevant (but I didn't assess whether it is helpful or not).
                          • explains that the item is perhaps more targeted towards professionals that might have a better idea as to what all the parameters may mean.
                          • provides a counterpoint that having the additional parameters is a feature especially in the case of professionals with demanding motor drive requirements.
                          • attempts to explain some of the unexpected behaviour and possible remedies.
                          • provides a justification as to product positioning, with an alternative product in case of greater needs.
                          • acknowledges that some changes may be necessary.

                           

                          I do think the expression may be slightly poor or be misinterpreted as defensive/blaming the user, but this is something I've come to worry about less and less as many may not have English as their first language and really intend to be helpful rather than confrontational. But it's also quite likely that this particular contact within the company may also be willing to have a more rapid back-and-forth communication into the future as things change or new questions arise.

                           

                          This is contrasted to cases where I may have a RoadTest with technical issues and end up contacting them via their marketing team playing "gatekeeper", such that every single problem report takes two weeks to turnaround as an official investigation is launched, a report is written, solutions are discussed and then someone has to draft an e-mail back to me with the absolute minimum of details in the hopes they won't accidentally admit fault. Incidentally, those companies that have this kind of workflow often can "lose" the clarity of the reports and replies as they pass through many hands and are "corrupted" by careful ass-covering. Just my 2c.

                           

                          - Gough

                          1 of 1 people found this helpful
                            • Re: Feedback regarding a companies comments on user expectations
                              colporteur

                              Your benevolent perspective of the vendors response is one way to placate the reader. There are two points in your response that raise my ire.

                               

                              P1: "It seems that your review did raise a number of issues, all of which I would think are valid observations especially for those who have never used this particular kit or combination before."

                              P2: "It pays to look beyond the way the thoughts are expressed and see what is trying to be conveyed."

                               

                              Why does P1 entitle the vendor a pass? My investigation didn't uncover any knowledge limits or expectations that could be used by the reader. I didn't find the disclaimer that indicated "If you find this information is incomplete or inconsistent you must be a novice and don't have sufficient knowledge to be using it". Other than understanding acronyms, how would higher learning have changed the inconsistencies of QUICK Start Guide references?

                               

                              I am reluctant to condone the approach you suggested in P2 because it requires a subjective approach.  What effective protocol standard permits the transmitter to send what ever they feel like and leave it up to the receiver to interpret what is received?

                               

                              Justin Halpern wrote a book called Shi#! My Dad Says using quotations made by Halpern's father, Sam, regarding various subjects. After I read the book, I thought, this is stuff I would like to say but something in my understanding of social order tells me it is not appropriate. Something in the message motivated kas.lewis  to ask for feedback from the group. I speculate he perceived a tone in the message that didn't align with what he felt was appropriate. I read the review before reading the vendor comments. I then read the vendors comments and felt something I labelled as dismissive.

                               

                              Now I could use your P2 suggestion and try and find the good in the message. Unfortunately my teeth are long in technology careers and find the suggestion demeaning and requires me to feel contrite. It is formula for acceptance or acquiesce. The alternate interpretation offered by the vendor, I find is not appropriate and I am not willing to give them a pass.

                               

                              I believe the RoadTest review program offers great value to vendors for minimal investment on there part. I believe my perspective is standing up for the program. I respect your commentary but remain with a dissenting opinion. 

                               

                              Sean

                                • Re: Feedback regarding a companies comments on user expectations
                                  Jan Cumps

                                  For me, as non-native English speaker it sounds dismissive too. Talking down.

                                  • Re: Feedback regarding a companies comments on user expectations
                                    lui_gough

                                    colporteur  wrote:

                                    Why does P1 entitle the vendor a pass?

                                    To be fair, I never said it did. I always provide feedback to manufacturers in my RoadTests which even include itemised lists of errors in documentation. Poor documentation can cause safety issues, can destroy products and can waste time - I would agree that quality documentation and support is very important for development-type products.

                                     

                                    I had only expressed it in the way I did to make clear that some of the perceived issues (e.g. with the number of bewildering parameters and options) would not in itself necessarily be a problem for someone who may be a professional in motor drives. It was not meant to be understood as giving a vendor a free pass to leave things in its status quo.

                                     

                                    If you go and buy yourself a Porsche, you won't expect the manual to include a complete guide to high-performance driving ... likewise, there are reasonable limits to be expected for some of these things, and from my perspective, this will vary depending on the purchaser's existing level of familiarity with the class of products and the field. This kit is not targeted at people like myself who know absolutely nothing about motor drives - but as to whom it actually targets, this is more "implied", hence I acknowledged the validity of the observations up-front.

                                     

                                    colporteur  wrote:

                                    I am reluctant to condone the approach you suggested in P2 because it requires a subjective approach.

                                    In my younger days, I would have said the same. In an ideal world, communication would be precise and concise with no room for ambiguity, but unfortunately, that's often not the case. I was blind to this before, but I guess, the more I take to reading other people's work and writing my own, the more I begin to appreciate the greys and try to ensure that I don't write something that would be too easily misinterpreted. After all, to be human is to embrace the subjectivity of life, even though it is something that (as an engineer) I find difficult at times.

                                     

                                    But sometimes, it's all about making sure you have some thick skin and sleep well at night. There are more important things than the polish on a few words. I've met some Chinese vendors in the past who had a very terse, confrontational response, but this was because they were not able to communicate effectively in English, relying on phrase-books and online translators. Without any non-verbal cues, intent can be quite hard to judge, but I'm defaulting to presume the best rather than the worst.

                                     

                                    Instead, I find that every communication is a chance to open a dialogue (for better or for worse) - in fact, the regime we live in is basically what you have described just there. I can write whatever I feel like I mean, but depending on your command of the language (and even the ever-changing definition of the language), your interpretation may well differ. This is often why there are so many lawsuits and legal loopholes, as the spirit of the law and the letter of the law often differ. I think even in that case, the legal precedent is often to respect the spirit of the law as intended. That's why dialogue is important as it offers an opportunity to elaborate and clarify the communication. It's part of the reason why comments can be so valuable to any posting.

                                     

                                    It pays to remember that sometimes the conflicts that happen are not intended - merely by a misunderstanding in communication. But to wage war would be an unproductive effort for both sides.

                                     

                                    colporteur  wrote:

                                    Unfortunately my teeth are long in technology careers and find the suggestion demeaning and requires me to feel contrite.

                                    [...]

                                    I respect your commentary but remain with a dissenting opinion. 

                                    Very much in the same vein, I completely understand your feelings and respect your opinion, as this was one that I myself have had in the past. Sometimes, to be contradicted so directly could be considered a wound to your pride. I know if it was directed to myself rather than someone else, I might have had that immediate reaction on the first read. Often, I will sit on things for a while, as the first knee-jerk reaction may not always be the right one. But I found that it was ultimately unproductive in terms of my own sanity and health, and put up a barrier between myself and the other party involved, so I often acquiesce and compromise. I have been accused of being a push-over ... so I guess that's just part of my character.

                                     

                                    As an academic researcher, I've come to realise how insignificant I am in the grand scheme of things. I can only do my best effort - but from there, what is taken and what is "left behind" is not up to my choosing. But then again, it's not my prerogative to influence - merely provide my opinion, whatever it is worth. At the end of the day, perhaps even this response might not be useful to anyone ... but I felt that I would be remiss if I didn't at least take the opportunity to explain the philosophy behind my benevolent character is for my own benefit as well. Life has enough worries already ... no need to add more.

                                     

                                    - Gough

                                    1 of 1 people found this helpful
                                      • Re: Feedback regarding a companies comments on user expectations
                                        colporteur

                                        Your response is much appreciated. Your maturity is reflected in your prose.

                                         

                                        I would like to publicly acknowledge your response demonstrates what I had hoped to achieve, that is an open examination of a perspective. That is what I enjoy about the Element14 community.

                                         

                                        I struggled with my decision to post a response to your commentary. I'm a curmudgeon by age and bad tempered by choice. I am not interested in starting flame wars or belittling the host of a commentary. I get no satisfaction either reading or participating in such activity. If I am a contributor to such disdain I feel guilty. I have used standing on my tongue in the past as an effective sleeping pill.

                                         

                                        I've read and re-read your response and failed to obtain your enlightenment. I believe our opinions align on some points and others remain opposed. Struggling to argue the minutiae of details is exhausting and fruitless. I will ensheathe my pen with a contrary perspective and say thank you for the opportunity.

                                         

                                        Sean

                                        1 of 1 people found this helpful