End user feedback is extremely valuable.
It’s highly likely that you could and should create open lines of communication between yourself and end-users. This is true in the majority of environments you’ll find yourself in as a new product manager. I say ‘create’ because you’ll quickly find that it takes a lot of effort and planning actually to make this happen, but it’s worth it.
Most of my experience so far has been working on B2B2C products (here’s a great primer on B2B2C by a16z GP Alex Rampell @arampell). I’ve generally focused on new product development of applications used by end-users, whether they be consumers, merchants/businesses, or internal employees. Regardless of who the end-user is, these systems generally have a front-end(s) for them to interact with, notifications (emails, SMS, phone calls, etc.), aka non-app touch-points, back-end components, alerting/monitoring, reporting needs, regulatory elements, etc. There’s a lot to think about, build, and manage! And none of the effort is worth it unless your end users are getting a lot of value out of the product. That’s why feedback matters.
I’ll probably write a post specifically addressing various product & company stages but let’s quickly glance at this slide from Dan Olsen:
Feedback is useful at all 3 stages (prior to first releasing your product, after it’s live but not yet reached Product-Market Fit, and after PMF).
Feedback is an essential component of most Product Prioritization Techniques. A popular classification framework is ‘Three Feature Buckets‘ from Adam Nash (@adamnash). The three buckets include Metrics Movers, Customer Requests, and Customer Delight. The focus on ‘the customer’ should not be overlooked. He writes, “Conversely if you find yourself without one of these buckets represented, it likely represents a serious hole in either your channels for customer feedback, your product execution, or your innovation capabilities. These holes will significantly impact both your short term and long term success in this area.” The image below helps visualize a roadmap leveraging this framework (h/t Brent Johnson from this product prioritization post).
I’m going to focus this post on two simple yet effective ways to get feedback from end customers (particularly Consumers) during the middle product phase between going live and achieving PMF.
Sounds sophisticated, right? That’s the point. It’s super simple and very effective for gathering up to date anecdotal feedback.
- Work with the necessary person in your company to create feedback@<yourCo>.com
- Provide her with the employees who should be included as recipients. This will vary significantly based on whether you’re a small startup (in case you’ll likely add a bunch of different people including the CEO) or huge org (in which case you may just include your immediate product team, your delivery team or perhaps just yourself and your UX partner).
- Provide her with the auto-reply details (Alias, address, body). The considerations section below explains why this may be important for you. The Body should likely contain a note thanking them for their feedback and setting the expectation that they will not receive a reply, and to contact customer service at x phone number or via x link if it’s a customer service issue or emergency.
- Create a filter to apply a ‘Feedback’ label and forward it to a ‘Feedback’ folder instead of going directly to your Inbox.
- Make the availability of the feedback email known to customers at the right touchpoints. Maybe add it to your FAQs, Welcome Email, or anywhere else that makes sense for your product.
- It’s cheap, easy, and reliable.
- It’s a direct line between the customer and the person creating the product that customer is interacting with.
- Many customers are going to mistakenly or purposefully try to leverage this as a Customer Service channel. They could be frustrated with a different channel (IVR, live agent, etc.) and will do anything to get through to someone else with less resistance. Others will try to use this just because they don’t feel like bothering with the appropriate support channel. The point is that not every ‘feedback email’ will contain actual feedback. You may even end up with a letter from a customer’s lawyer or emails from someone trying to sell you something. This is why the auto-response is essential. You’ll need to work with other teams (ops, legal, etc.) to determine a set of practices for managing this channel. Will you set SLA’s related to time to read, will you forward every email that mentions legal action to your legal team, will you forward customer service responses to customer service, or will you truly only use it for monitoring feedback? This is something you and your org need to decide for yourselves.
- If you decide to respond and engage with a customer, then remember that they received an auto-reply stating they won’t receive a response. You’re setting a precedent with them. And you should assume that could ‘leak’ to Twitter or a forum for others to see. Just be ready for that possibility.
NPS Surveys (with Promoter.io)
Net Promoter Score (NPS) measures the loyalty that exists between a provider and a customer. It’s calculated based on responses to the question of, ‘How likely is it that you would recommend our <company/service> to a friend?’ The answer is a scale of 0-10 and where they land labels them as either a Promotor, Passive, or Detractor. A positive NPS (>0) is often considered ‘good’ while an NPS >50 is ‘excellent.’ However, the industry for which the company operates in usually dictates what score is deemed to be good vs. excellent.
It’s easy to conduct ongoing NPS Surveys thanks to services like Promotor.io. The high level dashboard ends up looking something like this:
If you spend the time to apply tags to the responses, then you can start to dig in even deeper to see trend analysis that looks like below.
While your marketing counterparts are hoping to rely on leveraging the score for marketing reasons, the trend analysis is much more valuable to you as a product manager who is trying to figure out what’s going on inside the minds of customers. There are a lot of criticisms against NPS that you should be aware of. Inevitably, someone in your org will let you know about them if you suggest conducting a survey. But remember, the score isn’t necessarily what you’re after as a product manager.
More valuable than the score is the trend analysis shown above. But more valuable than the trend analysis is the freeform Feedback that you can get during the survey and the ability to analyze it based on various customer profile characteristics.
Above is what it looks like as the administrator.
- You can see the freeform feedback provided by the customer during the survey.
- You can have a dialogue with the customer from within the tool itself (leveraging email).
- Here are the attributes/characteristics I loaded into the tool along with each recipient. I anonymized each label and corresponding value, so the 3rd party tool wouldn’t have any more information than needed to conduct the survey. Like the scores, the feedback can be tagged, organized, and sliced and diced when analyzing. It’s like a feedback email method on steroids.
(assuming you’ve already cleared the initiative with legal and all necessary parties have agreed to conduct the survey)
- Create a Promoter.io account, play with their demo, poke around the website, and spend time reading their blog.
- Determine who is going to receive your survey. All existing customers? All active customers?
- Discuss plans for future surveys. Sending to same customers + new ones every six months, just sending to new customers 30 days after account opening, etc.?
- Work with the necessary team to pull the list of recipients (maybe you can do it, perhaps someone from marketing, or maybe someone from your analytics group can help like Brian Kane, and Steven Wang have for me) and the corresponding attributes you want to include (account age, activity range, acquisition channel, etc.) and determine if you’re going to upload them to Promotor, how you’re going to upload them, or if you’re going to export the results later and marry the results up with your data set for analysis in your tool.
- Set up the email itself. It’s a pretty simple template where you provide a logo and some content, the email address, and re-send settings (re-end after ten days of opening but not responding).
- Test it internally.
- Get to sending (recommended in batches). Whenever doing something like this, I just find that it’s safer to send out in batches (10%, 25%, 25%, 40%) and to take your time to avoid any mistakes like double sending to the same customer or other unexpected issues. I think Promotor.io offers some features to help protect against things like that (# times a recipient can receive the same survey).
- NPS criticisms are legit. And the score can be misused for good and bad, just like all surveys. But remember, you’re looking at leveraging this for product feedback purposes and less-so marketing. Keep that in mind when discussing with colleagues whether doing these is right for your team.
- Similar considerations from the Feedback Email apply here regarding setting the precedence for replies and some customers trying to use this as a customer service channel.
- It can take considerable time to properly work with the appropriate team members to identify the information you want to include as part of the customer’s profile, to pull it, anonymize it, etc. It might not even be necessary depending on your team’s capabilities (they may suggest just exporting the results and using your tools for analysis). Work with them to determine the best method.
- Always discuss initiatives like this with your legal department ahead of time to understand what they’re comfortable and uncomfortable with regarding customer communications and data sharing with 3rd parties.
‘A Product Manager’s Best Friend‘ is a series aimed at demystifying the role of young product managers. I’m Jared, a PM who has worked in a wide range of environments (fintech & adtech, seed-stage startup to large publicly traded tech & financial services companies, remote and colocated, etc.). I’ve found that there’s a need for practical, straightforward content for PM’s not fortunate enough to find themselves in powerhouses with top-notch product programs like Google & Facebook. Click here for more posts, and please reach out with suggestions or feedback.