How To Test Your Mvp App With Potential Customers Reddit

Jiaona Zhang is currently the VP of Product at Webflow.

If love is in the little things, then Jiaona "JZ" Zhang has a keen eye for the details that make users fall head over heels.

From her first PM role at Pocket Gems, where she crafted mobile gaming features that kept players logging in for more, to her time at Dropbox, where her instinct for zooming in on user problems led to the creation of some of the company's standout UI interactions, Zhang's built an impressive career centered on spotting just the right touch that captivates and inspires users.

She's also brought her expertise to bear on pixels and physical spaces, at both Airbnb — where she served as the product lead for the Plus launch and Head of Product for the Homes Platform team — and now at WeWork, as Director of Product Management. (Zhang also somehow manages to carve out time to deploy her product savvy in other arenas, as an angel investor and as a lecturer at Stanford, where she teaches students embarking on the journey of early product-building.)

Looking back at the through all these experiences, it's clear that love is at the (ahem) heart of Zhang's ethos on product design, the secret to her track record of launching products that users cherish. And if companies want to stay competitive, she argues, product folks need to be even more diligently attuned to a fledgling product's lovability — not just viability. "The minimum viable product was appealing because it was cheap, and you could get it to market faster. But we've advanced past a world where products are 'the first of X,'" she says. "Stiffer competition means that MVPs aren't going to cut it anymore. If startups truly want to stand out, they need to strive toward creating a minimum lovable product instead."

Consider this analogy that Zhang uses with her students at Stanford: "Say you're trying to test whether people like pizza. If you serve them burnt pizza, you're not getting feedback on whether they like pizza. You only know that they don't like burnt pizza. Similarly, when you're only relying on the MVP, the fastest and cheapest functional prototype, you risk not actually testing your product, but rather a poor or flawed version of it."

The concept of the minimum lovable product (MLP) sounds easy enough to get behind: You can yield far more valuable insights if your learnings are based on an early product that's closer to something users could love. But while the MLP is a familiar philosophy often bandied about in manifestos on product design, in order to build truly lovable products, founders and product designers need to take the concept from the theoretical to the concrete.

"At a high level, it's hard to disagree with the concept that we should build products that people love. But actually putting in the intention to build those products and understand those people is another matter entirely. It's not about slapping a few shiny features into your beta and calling it a day," says Zhang. "As product builders, we need to drill a layer deeper. What does love even mean? How do you identify what would make your product lovable? How do you do it in a way that's fast and efficient?"

In this exclusive interview, Zhang digs into why a deep understanding of MLPs benefits founders and product leaders. She breaks down the four principles for how founders and product managers can zero in on exactly the right problem to solve. Then, she walks through how to put MLP into action — all it takes is careful attention to user needs and a little bit of pixie dust.

LOOKING FOR LOVABLE IN ALL THE WRONG PLACES: A CAUTIONARY TALE

Shipping MLPs can bring powerful advantages for companies of all sizes. "If you're at an early-stage startup, you need to be able to make the case for your users to stay while you build-measure-learn on the way to strong product/market fit," she says. "For larger companies, creating MLPs shows that you care about users enough to continue to invest in offerings and new product lines that remind them why they fell for your products in the first place."

Building a minimum lovable product should start with a strong foundation of understanding. "If you don't have a firm grasp on how to put an MLP into practice, you can waste a lot of time building a product that you think is lovable, but ultimately isn't," says Zhang.

She shares a lesson learned the hard way from Dropbox, to illustrate what happens when teams fail to understand what makes early products lovable. At the time, Zhang was leading the charge on Project Harmony, which involved building a feature that signals users when someone else has opened the same document.

The ability to see other collaborators solved a distinct pain point: Previously, if two users unknowingly made edits on the same document at the same time, that would create two conflicting copies of the document. The thinking behind the new feature was that users might collaborate more effectively if they could see one another on the same document.

"When we first started building, we were satisfied with creating the ability to see if another collaborator was in the document. We thought that was good enough to ship," she says. And while Project Harmony launched to positive reception, in retrospect, Zhang believes that the product fell short of lovable.

"In hindsight, we missed the most valuable insight," says Zhang. "Being able to see someone in the doc was pragmatic — presumably, you'd hop out of the doc and avoid conflicted copies. But it only solved a very basic pain point. It didn't actually deliver something users would love."

Lovable products aren't simply functional or useful — they demonstrate an acute understanding of what users find valuable.

"If you dig into what makes Google Docs such a delightful product to use, it's not only that you can see that there's another collaborator in the doc — it's that you can see in absolute real-time each user's edits, cursor by cursor," she says. "With Project Harmony, being able to see another person on the doc, while useful, was ultimately unsatisfying. In fact, it highlighted a fundamental limitation of Dropbox's product: the fact that it didn't yet support cloud-based, real-time collaborative editing. Had we spent the time upfront to try to understand what the user would value most, we wouldn't have made the same investment," says Zhang.

HOW DEEP IS YOUR LOVE? FOUR PRINCIPLES FOR ZOOMING IN ON THE PROBLEM

Whether you're a scrappy startup or at an established org, the first step for putting the MLP approach into practice is the same: "Start with a high-value problem for your users, then get deeply curious about your users' needs and motivations," says Zhang.

Zhang highlights the four principles that she uses to guide founders and product managers toward the problem:

1. Start with the user's why, not the business why.

2. Separate the problem space from the solution space.

3. Listen to your users — but don't take their word as gospel.

4. Enter the solution space and choose your game.

Below, she breaks down each principle, illustrating them with ample examples from her own teaching and product management experience.

1. Start with the user's why, not the business why.

"If there's one pitfall that companies fall into, it's that they focus on the 'why for the business' instead of 'the why for the users,'" says Zhang. "Companies will naturally start with a why for the business, just because of how they operate. But as a product leader, if you don't pair it with a 'why' for the user, you won't be able to set up an effective problem statement for your team."

If product managers aim their efforts toward creating a product that solves a "why" for the business, rather than a "why" for the customer, they'll miss lovability by a long shot.

To return to the example from Project Harmony, Zhang's team had missed the "why" for the user. "The user 'why' wasn't simply that they didn't like conflicted copies. They wanted multiple collaborators to be able to work on the same document and make real-time changes. So when we tried to keep iterating on Project Harmony, we were only optimizing a local maximum," says Zhang. "On the other hand, if we had more quickly invested those resources into building a more lovable product, like the technology that would become Dropbox Paper or Spaces, we would have worked toward a global maximum."

Aim for a global, not local, maximum when building early versions of your product.

To further illustrate the disparity between the two "why's," Zhang shares an example from her time at Airbnb, where she led several supply side teams. "One key business 'why' for Airbnb is growing the supply of homes. This is obviously top of mind for us, since supply growth is a leading indicator of overall nights booked," she says.

But growing the supply of homes leaves the user-centric "why" unaddressed. "Sure, you could say that increasing the supply of homes gives guests more optionality. But is having more homes what our customers truly want?" says Zhang.

A stronger understanding of the user leads to a more precise "why" statement — and to more lovable solutions. "Not all supply is created equal. Customers don't want to see more lodgings in places they don't want to visit. They want more listings in extraordinary locations, by ski resorts or beaches, or where major events are happening," she says. "Airbnb needs to grow its supply in top markets to give users more optionality at desirable destinations. That's the user-centric why."

"Shifting our priority from simply 'growing supply' to 'optimizing high quality supply in certain markets' was the first step toward lovability," she says. It helped her more clearly define the minimum lovable product — and the right metrics — for her supply teams at Airbnb.

Zhang has a handy litmus test for knowing if you're aiming your intentions in the right direction.

"Start with the statement, Wouldn't it be amazing if this product could...? " she says. "Your statement should end with value for the user. It should frame the problem to prioritize the user's needs and wants." Zhang shares how she focused the "why" when she was working on launching Airbnb Plus, Airbnb's more high-end tier of accommodations:

Don't ask: Wouldn't it be amazing if this product could help Airbnb compete with luxury hotels?

Instead, ask: Wouldn't it be amazing if users had a variety of high-quality options that made their precious vacation time even more special?

2. Separate the problem space from the solution space

When Stanford students file into Zhang's product management class at the start of the semester, they're already brimming with ideas for the final project, where they create a new product and work on it through the entire product lifecycle. The students often start chattering about building the next marketplace for X, a new platform for Y, Uber but for...

Zhang stops them right there. On day one, she spells out the first rule of class: Don't think about your idea for the first third of the semester.

"Being able to detach yourself from what you think your users need and what they're actually experiencing is often the hardest lesson for PMs to learn — but it's the cornerstone for building lovable products," says Zhang. "This is most challenging at early-stage startups, where the founder is very passionate about an idea. But if you don't loosen your grip on a proposed solution, you won't be able to take in valuable feedback about the problem."

Never start with solutions. Just because you prefer the hammer or the screwdriver doesn't mean it's the tool you need to fix what's broken.

3. Listen to your users — but don't take their word as gospel.

"In order to develop a nuanced understanding of their high-value problems and pain points, you need to strike the balance between listening to your users carefully , but not literally ," says Zhang.

Here, she flags common mistakes that teams tend to make when it comes to conducting user interviews:

Asking customers what they want. "When you ask your users what they want, there's a tendency for them to talk about solutions they think you could build for them. Taking your customer too literally limits the scope of ideation."

Positioning yourself as the voice of authority. "Users often see you, the product manager, as the expert on the problem. You're not. They are. Make that dynamic clear at the outset, or you won't be able to unearth what your user truly cares about."

Startups need to be particularly methodical about user interviews because they start out with a small set of users and with fewer resources to devote to research.

"Especially if you're a startup, be clear about what user persona you're targeting first. It's hard to build a product that several different user types all love , especially at the get-go. Focus on talking to enough people to learn about that one user type, and make sure you're asking open-ended questions to ensure you're getting the right data," she says.

To yield the most fruitful insights from your users, Zhang suggests asking the following question: What do you find tedious, stressful, or painful?

"Start with the problem. If users say something like, I'm so frustrated by this thing. I can't believe I'm spending so many hours in my day doing this, then you've likely struck a great opportunity to intervene," she says.

If you can't listen to your users, you might not even build something that's viable. And you certainly won't build something that's lovable.

4. Enter the solution space and choose your game

Finally, once you've gathered data, get your team from the problem space to the solution space. Here's an exercise that Zhang uses with her students:

Draw a vertical line. On the left side, write down only the possible problems to solve. "If you hear anything during the brainstorm that feels like a solution, note it on the right side. Acknowledge the thought, but don't discuss anything on the right side yet," Zhang says. Imposing a separation between problems and solutions keeps the discussion diligently trained toward the user whys, not business whys.

"Once all the problems are laid out, narrow in on the highest-value problem that you believe you can solve," she says. "When I do this with my students, I ask them to decide if that's the game they truly want to play for the quarter."

Zhang challenges founders to "choose their game" as well. "Ask yourself: Do you have the right talent to tackle this problem? Are you willing to spend 5-10 years tackling it? Being really intentional about choosing your game equips you to define the problem space and also to put constraints on your solution space."

Finally, once students have focused on the problem space, identified the highest-value problem and chosen their game, they can open up what Zhang calls solution space ideation. "The entire approach, from start to finish, is similar to design thinking," she says. "Diverge first to understand the problem space. Converge on the most important problem. Diverge again to come up with solutions. Converge on the solution that's both usable and lovable."

"Once you understand the problem, then you can think about possible solutions. Bring those solutions back to the user," says Zhang. "Then you can ask them: Out of these things, which do you wish existed?'"

To distinguish between a viable solution and a lovable solution, listen for your users' reactions. "In most cases, people will look at a prototype and say, 'This is cool. I can see myself using this.' That's fine. But that's not what you're aiming for," says Zhang. "You want your users to say, 'Oh my gosh, when can I start using it?' That's when you know you've struck a chord. If they don't, either you haven't made the product lovable enough or — and this is more likely — you're not solving a high-value problem."

Your user's face should light up when they try out your prototype. If the user isn't lovestruck, you probably aren't solving a high-value problem — and people won't be willing to pay for your solution.

PUT A LITTLE LOVE IN YOUR HEART: THE NUTS AND BOLTS OF DELIVERING LOVABLE VALUE

Once you've honed your understanding the problem, it's time to put your MLP into action. In this section, Zhang illustrates how to take MLP from proposal to product — from striking the balance between speed and lovability, all the way to the final trial of user testing.

Strike the balance of minimum and lovable

The hardest part of creating the minimum lovable product is in the name itself: How do you create the most lovable product while expending the least effort?

"This is why cultivating deep understanding beforehand is so important: It lends you insight into your users' core problem. When you start building, zero in on just one or two features that truly bring value and delight to the user. That's what makes the MLP 'minimum,' yet still lovable."

If you try to solve every problem with your product, you'll do it all poorly. I call this the Peanut Butter Principle: spread too thin, it's no longer tasty.

When a team neglects minimum and over-engineers on lovable, time and resources suffer. When Zhang was launching Airbnb Plus, her team needed to figure out what characteristics would define Plus accommodations as a cut above Airbnb's budget options. "We asked ourselves, 'What specific touches would make a space more comparable to a hotel?'" she says.

At first, her team swung hard in the direction of lovable. "We put together a 200-point list of all the hotel-grade amenities that we thought Plus homes needed to have: room-darkening shades, four pillows on every bed, etc. But when we actually looked at this gargantuan list, we thought, If we require all of this, we will have zero qualified homes," says Zhang.

To course-correct, she focused on the qualities that actually set Airbnb apart. "People don't love Airbnb because it's like a hotel. They love that they get to cook with their family, or they get a peek into a host's unique design style," she says. Instead of defining the Plus category by a hotel-grade checklist, her team doubled down on what made Airbnb feel like home.

Jiaona Zhang ("JZ"), Director of Product Management at WeWork.

"To do this, we turned our attention to elevating the online booking experience," says Zhang. "For the Plus homes, we created a Home Tour feature that allows people to photographically 'move' through each room. The flow feels like a digital open house — like users are walking in through the door and being welcomed into someone's home. That's a lovable, distinctively Airbnb experience."

Don't get so caught up in the minutiae that you lose sight of what people actually love about your product.

Furthermore, investing in a digital feature ended up being a more efficient use of resources. "It was so much cheaper to invest in a thoughtfully-curated digital feature than trying to cram physical items into these homes," says Zhang. Her team had struck the perfect balance of minimum and lovable.

That balancing act is useful for thinking through not only how to solve a problem, but which problems to prioritize. Zhang and her team confronted this while they were revamping the mobile app for Airbnb's hosts. At the beginning of the project, she and her team followed the four principles for drilling deep into the problem: They segmented their personas to figure out the customer "why," solicited feedback through surveys and user groups, and identified the strongest problems before diving into solutions.

First, they made the inbox easier to access. "After talking to people and examining the data for the previous iteration of the app, we realized that the inbox was the most-used feature — but it was buried under a side menu. So we moved the inbox to the landing page," she says.

Second, they updated the calendar feature so that hosts could see multiple listings at once. "We could have done so many things to improve on the calendar feature, like advanced price toggles or data blocking. But at the end of the day, the most important thing for the user was that they could see everything at once and map out their time more effectively."

Finally, they introduced a new feature that streamlined a tedious, repetitive process for hosts: allowing users to save templates of messages, like check-in instructions, that they sent repeatedly. "To be honest, this feature was an unexpected by-product of our user research; before, my team was excited to spend time on problems like improving the speed or the granularity of inbox search," says Zhang. "But we learned that a big reason why hosts searched their inbox was actually to copy-paste messages. It a tedious, repetitive task. So being able to send message templates ended up being one of the features the hosts loved most."

Because of the team's laser focus on the most lovable solutions to the most pernicious problems, the entire redesign took less than six months. "We launched the redesigned app at Airbnb Open, and hosts literally gave us a standing ovation. They were so happy with the changes," she says.

"You already know you could do a million things to improve your product. But focus on what's most important to the user and what you can deliver in a timely way. That prioritization made all the difference in creating a lovable app," she says.

The same principles of prioritization apply for early-stage companies starting at square one. "If you've done your homework in the problem space, then you'll know what's most useful and valuable to the user. Start by designing your product around that most essential feature," she says. "Think of Dropbox: it started off as a solution to replace flash disks and thumb drives. Now, it offers an entire suite of collaboration products across consumer and enterprise markets."

Start with the one feature that's going to make your product indispensable. Build from there.

Finally, understand your user in context. When you're deciding how you want to build out an MLP, the medium matters. "You need to understand what your users expect out of a medium," Zhang says. "What do people need on a desktop? If you're building a phone app, what do people need on their phones, in the moment?"

Zhang notes that this is an area where the original Airbnb app for hosts fell short. "Originally, every feature on the desktop site was carried over to the mobile app. The app was basically a mobile clone of the desktop version," she says. "But when you actually look at how users employ each platform, that just doesn't make sense. Hosts use their phones to snap photos and shoot off messages. They won't use it for more intensive work, like setting up a listing and researching local regulations," she says. "Because of that oversight, the original app tried to do too much without doing anything well."

Embrace your user misbehavior

"Sometimes, observing what users do that's contrary to your product's intended functionality can lead to insights on what's lovable," says Zhang.

She shares a story from when she was at mobile gaming company Pocket Gems. Her team was working on a game called Tap Paradise Cove in 2011, at the height of popularity of games like Farmville. "The primary directive of Paradise Cove was to have players tap on a plot of land in order to 'till' it for crops," she says.

But Zhang noticed that users attention was drawn elsewhere. The game's opening image was an aerial view of an island shrouded with mist, dense enough that a player could see the tops of houses and templates emerging from the fog but couldn't quite tell what they were. Initially, this was a purely aesthetic choice. "But we noticed that players kept tapping on the objects poking out of the mist, even though they didn't necessarily serve a purpose in the game," she says.

To build on the intrigue that the fog sparked in players, Zhang and her team designed several narrative quests around the shrouded objects — and effectively made the user misbehavior a key part of the product.

Be observant about how a user interacts with your product. When a user behaves against your expectations, they might just be leading you toward what they find most lovable.

Conjure up pixie dust

Moreover, creating an MLP is not only about knowing what details matter, but being able to have fun with them. "I like to call it pixie dust. It's that magical moment in the user journey where the user realizes that this product is different from anything else they've ever experienced," says Zhang.

She emphasizes that adding pixie dust to your product features doesn't need to take too much effort. Here, Zhang provides some questions to ask yourself to make sure your pixie dust products take flight:

When a user signs up for your product, what should be the first emotion they feel?

What would bring a smile to a user's face?

What would give a user a reason to rave about your product?

"That last one is key. Especially in the beginning, your product should inspire people to want to tell their friends. Word-of-mouth creates a powerful flywheel — and surprise and delight provides an excellent talking point," says Zhang.

One feature Zhang built at Airbnb shows that it doesn't take a complex feature to give users a memorable experience. "When a traveler gets close to their destination, and they have their GPS turned on, a check-in guide will pop up on their screen. In one tap, a user can access information about where the key is hidden, the lockbox combo, the WiFi password — all of the crucial check-in information," she says.

The gesture, while small, was acutely impactful because it anticipated a key pain point: the stress of scrambling for check-in information when you're in an unfamiliar locale. "Creating effective pixie dust doesn't just mean knowing what to do, but when it would be most important for the user," she says.

Zhang's taken notice of one company in particular that's used pixie dust to help its products take flight. As an angel investor, she backed mobile food ordering company Snackpass. Launched on Yale's campus, Snackpass currently encompasses several major university campuses. "From the moment I started playing around with their product, I was struck by the pixie dust sprinkled all throughout the experience," she says. "From the carefully-picked emoji in the copy, to a gifting feature with puns like 'I love you a latte,' the product showed incredible attention to their audience of students."

Snackpass underscores a key insight about pixie dust: It's powerful because it successfully speaks to a particular user's problem. "No one wants generic pixie dust. Get specific about what would resonate most with your users," says Zhang. (In fact, before she invested, Zhang verified the pixie dust with her friends' younger siblings who used Snackpass on their campuses. "The young'uns confirmed it was on-point," she laughs.)

This is a strength of Snackpass's. "That kind of casual, playful communication speaks to their audience. But if you copy-and-pasted that same approach to another demographic, it wouldn't land as well," she says.

LOVABLE IS A BATTLEFIELD: PUTTING THE MLP TO THE TEST

Finally, take a careful approach to testing to see if your product is truly lovable. "If you release it to everyone all at once, you'll get mixed feedback that's hard to do anything with," says Zhang.

She recommends a gradual, structured framework for testing whether pixie dust lands: by mining for insights from your beta before rushing into the GA release. "People don't leverage their beta enough," she says. "The redesigned Gmail was in beta forever, right? You can gather your most insightful feedback, not to mention achieve a lot of growth, just with your beta."

When it comes to testing your MLP, don't put too much weight on the feedback from your alpha. "Your alpha group is the one that already loves you, that would even love your MVP. It's your mom who will love anything you make," Zhang says.

On the other end of the spectrum, your GA is far more skeptical. "In Airbnb's case, the GA includes people who wouldn't dare to use Airbnb because staying in another person's home seems like such a preposterous concept," says Zhang. "Your GA is much harder to please. Their trust is hard to earn and even harder to regain once lost."

Launching unlovable features is like being the boy who cried wolf. You can launch another feature, but you can't guarantee you'll still have users' trust the second time around.

Your beta is your sweet spot for testing. "Like your alpha, your beta group is rooting for you. But they're also willing to give you constructive criticism," she says. "Airbnb's beta users are people who have used Airbnb at least 10 times and feel satisfied. However, they also remember a few moments when check-in didn't go as smoothly, or they got locked out — and they're going to let someone know."

Testing your pixie dust on your beta is key because you've already built a level of trust with those users who will give you valuable, candid feedback. "It allows room for taking risks without compromising your credibility," she says.

ALL YOU NEED IS LOVE: MLP AND THE FUTURE OF STARTUPS

If startups want to build products that users cherish and that stand out from the competition, it's all the more important for them to raise the bar from viable to lovable. Building an MVP starts with honing in on the user's most frustrating problem. Choose one or two features that would most acutely address the problem, then work on balancing the elements of intrigue and familiarity to create the most lovable experience with limited resources. Finally, find ways to add pixie dust, testing it with the group that's ready to show you some tough love.

Looking back at her experiences adding intrigue in the virtual world of gaming, solving productivity challenges at Dropbox, teaching students at Stanford, and advising companies like Snackpass, Zhang is hopeful for the future of the MLP. And with her time at Airbnb and now at WeWork, she's been experimenting with new ways to build MLPs in the hybrid world where software meets physical space.

"There's so much room in the next 10 years to make pixie dust beyond pixels. So many products in the next decade are going to be some combination of the digital and physical realm," she says. "We're only going to see more room for making magic."

Illustrations courtesy of Jiaona Zhang. Photography by Bonnie Rae Mills .