Product Metrics for Product Designers - The Ultimate Guide
Vložit
- čas přidán 5. 07. 2024
- 🚀 Get 2 free months of Skillshare Premium & watch my design course for free: skl.sh/2TVhf8z
In this video, I will help you understand how to define metrics when solving a problem as a product designer. I will share plenty of examples from companies like Airbnb, Swiggy, Cred, Uber and Trade Republic to help you understand. At the end there will also be a small test.
🚀 The Ultimate Guide to Become a Product Designer: learnproduct.design
🚀 Create an account on Mobbin for free:
mobbin.com/?via=chethan
🔶 Chapters
00:00 Introduction
02:05 Agenda
04:07 How are metrics defined?
07:35 Types of Company Goals
08:56 Examples of company goals of different companies
12:07 Airbnb Example
17:02 Uber Example
19:05 Trade Republic Example
29:41 Mini Case Study
34:20 Tackling difficult situations
40:07 The final test
-----------------------------------------
Mega Product Design Course for Beginners:
• The Mega Product Desig...
Webflow Course for Beginners:
• Webflow Course for Beg...
Photoshop Tutorials:
• Photoshop CC Tutorials
Illustrator Tutorials:
• Illustrator CC Tutorials
After Effects Tutorials:
• After Effects CC Tutor...
UI Design Tutorials:
• UI Animation/Interacti...
Design Resources, Tools and Softwares:
• Design Resources, Tool...
Adobe XD CC Tutorials:
• Adobe XD CC Tutorials
eSports Design Tutorials:
• eSports Design Tutorials
--------------------------------------------------------------------------------------------
Contact me :
Portfolio: chethankvs.design
Twitter: / kvschethan
Instagram: / design_pilot
Mail: designpilot21@gmail.com
Behance: behance.net/chethankvs
Dribbble: dribbble.com/chethankvs
--------------------------------------------------------------------------------------------
LIKE, SHARE, COMMENT & SUBSCRIBE :) - Krátké a kreslené filmy
![[Part 1] How to write an Effective UI/UX Case Study - Tips and Tricks!](http://i.ytimg.com/vi/1LM-4dF5_qM/mqdefault.jpg)
Was waiting for this long time
1. I guess cred can classify there users into two types:
(A). Based on the engagement within the range of time: For example how many users are actively opening the app to check garage, Store, or travel services or checking for rewards, next due payment etc..
(B). based on the payments done through the app: How many users are actively making credit cards or UPI payments through the app within the specific time range.
2. Airbnb's main moto behind that feature was to help users to fight with the paradox of choices (because airbnb have lots of listings it can be confusing for the users to make a decision). So if the completion rate is improved after rolling out that feature it means it was worth it.
1. So what do you think acts as a valid activity? Not all users will use garage or store. But might only make credit card payments. Do users have to do all the activities? Or some of them or one of them to be considered an active user and how often do they have to do it?
How do you think checking payments done by users in a specific time period helps?
@@DesignPilot 1. Yaa I agree with that not all users will use all the services. But I guess Cred must divided there team into smaller divisions for example team A handles services like travel, garage and store, Team B handles Gamifications and rewards, team C cards, utility and payments part. And most probably They are measuring Active users based on those divisions individually.
2. Payments done by users in a specific time period is the crucial metric because of the business model of Cred. Cred makes money from Listing fees from other companies in their store, from rent payments, loans etc. and other than that currently Cred focus on Growth & expansion so this the metric that Investors and stock holders mainly focus on.
Really amazing video a lot to learn from this. We need more videos from your end, They add so much value
What do you want me to make videos on?
@@DesignPilot can you make video on design process in a practical manner instead of using lean design process you can use lenghty one where you have a problem statement and your are solving it by setting up the context about probelm then thinking about the solution hypothetically then you are doing primary and secondary research inclunding usability testing comptetive analysis doing ideation by user journey mapping creating user flow and wireframe coming up with final solution we do not need the final ui we just want to deep dive in ux process we know that you allready have some videos but they are very hypothetical please consider this
@DragonWolfGamer I already have a few videos on that. Using ChatGPT. You can watch those
Very nice video can you also make a followup video on how to make data driven decisions. Thanks
Will think about it
@@DesignPilotYes that would be great. Having an example of you using publicly available or fake data to make product or design decisions will be good
I believe by for airbnb, by adding this new feature, they are increasing the benchmark for hosts to keep the hotels or homes high quality. It almost becomes like a competation for room owners to be best ranked on airbnb.This i believe will do 2 things 1.increase competition among room owners and bring more room hosts into the game 2. This will increase customer trust and increase more customers per host who fall within the top 25 percentile. Mainly I feel airbnb will use number of customers as a metric to judge this feature. I say this beacuse now with the top 25 percent badge feature, more people will trust the place. This will lead to more booking for hosts within top 25. So its like double advantage. More people going to top 25 percent hotels , so more revenue for these hosts. So in turn more hosts will try to get this badge to increase their revenue. Overall its a win for airbnb, as they get more customers in top 25 percent homes and they have more hosts competing for the glory badge. So overall almost all hosts listed in airbnb increase their overall quality and in turn increase the quality of airbnb. Please feel free to comment on my guess. If there is a different appraoch to this please comment. This could be a good start to a discussion!!
Sorry for this long essay😂
1. Great thinking. But why do you feel the ranking is related to home quality? Could it not be something else?
2. When you say 'Mainly I feel airbnb will use number of customers', what do you mean by number of users?
3. But what are the metrics you would measure to ensure that the quality of homes is improving?
Basically what data will you see to know if the feature worked or not?
@@DesignPilot The feature was giving some kind of a trophy or rank to the home right. So considering that I would use customer reviews itself to awards these home with the trophy. So at the end of checkout, a customer should fill out a precise form giving their rating about the house. It can be limited questions but precise questions like how was the house quality, host behaviour, accessibility to the main areas of a city/tourist spot, price etc. I would try and keep the question limited to max of 5 to 7.Now how easy the form is to fill is a different problem. But my point is based on genuine customer feedbacks, the host will rise up the ranks and be popular. And since it's best rated home, more people will prefer it and inturn more income to the host.I assume Airbnb takes some kind of commission from top rated homes, so more revenue to Airbnb. Again this is an assumption that they take commission as their revenue, feel free to correct me.
I think customer feedback can be one of the factors. Other factors could be, how often it's booked, number of customer complaints (not just good feedback), how long they have been on the platform etc.
@@DesignPilot right!! That's actually a valid point!!I actually didn't think about how long they could be on the platform. Retention of the hosts also it a key factor for Airbnb survival.
Great video, thanks for it, on to the tests
1st test CRED
I think cred decides anyone out there who is using credit cards to shop is their target audience and they want to attract them to redeem those rewards through them etc.
2nd Text Air BNB
Goal = To increase revenue
Primary KPI = more bookings for expensive Air bnb's
Secondary KPI = less users dropping of due to not finding a quality listing
1. So there is a difference between target user and an active user. Active users are those people who use the product often and helps Cred grow and make money. Not all users of Cred are active users. So what do users who use Cred have to do, to be considered as an active user by Cred?
2. Why expensive? Also isn’t the Primary and Secondary KPI the same in your explanation? More bookings = Less drop off.
@@DesignPilot
1. You're right, hmm I would say they have to often log into the cred app or redeem through the app, and I think that would qualified by CRED as an active user
2. Expensive as the quality structure is put in place demand increases and since the supply is limited, prices will go up, however I can be wrong in thinking that
About primary and secondary KPI being the same, not necessarily to link this to the expensive part, I meant more bookings would be there for the top listings that's the primary KPI
Now that this system is in the place all the hosts will try to make it to the top 1% which will lead to the less drop off, I stated this as a secondary KPI as to me it felt like an consequence than same.
NOTE: thankyou for replying, just wanted to say I've haven't used CRED nor AirBnB all that much so I can be very wrong my guesses as they lack solid understanding of apps and businesses, I would love to hear if you have any idea on how to detect these things better or they're something that comes with practice ?
Cred active user can be:
Type1: someone who uses the app to track/pay their credit card and utility bills and look for rewards.
Type 2: the recent UI changes and offers suggest that they are focusing more on UPI so someone who makes atleast 1 UPI transaction can be considered as an active user
1. Do they have to do all 3? Pay credit card bills, utility bills and look for rewards?
2. Can you define 'look for rewards'?
3. This 1 UPI transaction per user, what's the time period? 1 per day or week or month or something else?
@@DesignPilot 1. I think anyone making apayment of any one of the bills on the platform can be considered to be an active user on Cred.
2. By looking for rewards I mean, someone who engages with the app to gain rewards. They have these fun engaging game that lead to collecting cred coins, vouchers etc. Someone who comes to the app to grab these daily rewards can also be considered as an active user.
3. You’re right! I missed the mention of a time period. I believe there are metrics in place for various timeframes like DAU, MAU right? Each will have to be tracked and will play a different role
In the case of airbnb, I assume
1. Company goal- to increase revenue
2. Primary kpi- it could be increase booking or reducing drop off rates (in this case I am assuming it is to reduce drop off rate)
3. Secondary kpi- increasing number of decision making parameters so that users can make informed decision without getting confused which can be measured using task completion rate.
For feature kpi, we can compare the number of bookings before and after earning the trophy for the hosts may be..
1. For the secondary KPI do you mean conversion rate?
2. Are you saying number of bookings is feature or primary KPI?
1. Yes I guess. I mean the number of users completing the booking instead of dropping off in the middle. Like if a user is dropping off after going through the listings and making comparison, I'm assuming they are finding it difficult to make decision as they are not able to distinguish good airbnbs (Ofcourse there could be many reasons for this)
2. While analysing the number of bookings for individual hosts before and after earning the trophy could indicate success of the feature, having overall increase in bookings can be a primary kpi.
Please do correct me, if I have misunderstood the concept.
Valid!
Appreciate the effort, I am working on a non-profit project and I'm sure it will help me a lot. And I am interested In the your Learn Product Design course. Is there a way to pay in at least 2 Installments 50/50 or one has to make full, one time payment?
You will have to make 1 full payment
For Airbnb, session length in relation to actual conversions could be Metric to track? Since users are going to be on the app with an intention to book a stay, how quickly they are able to make a decision with the help of this feature can be a good track?
Also, love this new format of a quiz at at the end. Forces us learners to pause and think and not just move on to the next thing :)
1. Why does it matter to Airbnb if users take 5 mins or 25 mins to book a stay? Why does Airbnb want users to make a decision faster?
Glad to hear you like the quiz format.
@@DesignPilot
1. I think anyone making apayment of any one of the bills on the platform can be considered to be an active user on Cred.
2. By looking for rewards I mean, someone who engages with the app to gain rewards. They have these fun engaging game that lead to collecting cred coins, vouchers etc. Someone who comes to the app to grab these daily rewards can also be considered as an active user.
3. You’re right! I missed the mention of a time period. I believe there are metrics in place for various timeframes like DAU, MAU right? Each will have to be tracked and will play a different role
@@DesignPilot The more options the user sees the more confused he may get. Chances are he may also go to another app to make a booking. With this feature the user gets an additional layer of assurance that the stay is good and make a booking faster.
You mentioned CTR right, what are some of the mobile analytics tools that companies integrate to check these metrics?
You can Google. There are plenty of tools
This is my educated guess - 1. I feel cred's main active users are young people with an access to credit card. These people can be young kids who have access to their parents credit cards with cred score more than 750.( I believe thats a mandatory criteria for cred). I strongly believe the main force driving these kids towards the app are 1.the cool rewards 2.Marketing by cool influencers like tanmay bhat etc and fancy ad collabs 3. The feeling of using a premium app as you know it acts as a status symbol to have access to app which allows only credit card users of 750+ credit score.
I think you misunderstood the question. An active user is someone who actively uses the product. Not the target customer or userbase.
@@DesignPilot correct me if I'm wrong but isn't target customer meant to be the active user. Your building the product for them right.I understand that all of target audience might not use the product but the primary TG should be the active users right?
*TA
So basically the definition of active is someone who is already using the product. But they are actively using the product. If you use CRED just because you have a credit card, will CRED consider you as an active user? Don't you have to use the app regularly? If yes, what are the things you have to do for CRED to treat you as an active user.
@@DesignPilot okayy now I get it, it might include a lot of activities related to credit card payments and not just the rewards right! Got it!
👏Nice
Answer to question 1: I think there could be multiple types of active users. a. People who opened the app at least once in a day. b. People who make one transaction in a day. c. FANS: Who have referred more than 5 users AND open app once a day at least AND make one transaction in a day.
Answer to Question 2: I think there could be two KPIs here. a) Reduction in time taken to make a decision to book when one of these top Hosts are shown against the ones who don't have this badge. b) Obviously, increase in the bookings of the Hosts who have this top 25% or 1% badge.
Chetan, I just edited the comment. Have a look.
1. How does opening the app help CRED’s business?
2. This is something good to measure
3. Referrals are usually very low for products like this when marketing spend is very high. But maybe Cred does have a lot of referrals. However it’s very rare to have 5 people referring a product as per common user behaviour.
@@DesignPilot 1. Yes I do agree that it won't help their business right away, but isn't it good to track a user who at least opens your app at least once a day and is not making any transactions YET?. Maybe they could nudge them or give a special bonus ONLY for those users to make their next transactions? Or maybe from the backend they can pick a few of those users to do a user research and try to to understand what's stopping them from making a transaction. What do you think?
3. Agreed.
Good questions.
1. It's rare for users to open the app without intent. If users are opening the app, they are doing something. Browsing things to buy, Checking credit card balances, or viewing some information. Not everyone will want to use Cred UPI to make transactions as they might have already been using Gpay or PhonePe.
2. Cred is already tracking feature adoption rate and retention of users for all the products it has. It doesn't have to track the number of times an app was opened.
3. With regard to Airbnb, it doesn't make sense to track time spent to book something as there are multiple factors to consider. For example, I might be just shortlisting some properties. I might be researching where to stay and comparing it with other platforms like booking. com for example.
4. Booking the right stay is an important decision and it should be fine if you spend a lot of time booking something. But yes, as you said, an increase in bookings is the right metric. Not time taken to book a stay
@@DesignPilot 1. Ya sure Chetan. Actually when I said open the app, I meant people who open the app and maybe do all these things AND don't do any transactions. Anyway, you pretty much cleared the point.
2. This is interesting. Had not heard of feature adoption rate term. Learnt something new. Thanks.
3. Ok, I agree with you to some extent. We could have this 'time spent' metric as a secondary metric with Number of bookings as the primary metric. There was a reason why I mentioned the time spent: I was trying to capture the QUALITY of EXPERIENCE that the users having. Here my reading was, if a user is spending too much time in searching for a Host, that means he is getting confused or struggling to make a decision. But with this feature, if the AirBnB team successfully manages to reduce the AVERAGE time taken for each booking(which according to our assumption, means users were less confused and their decision making process was made easy) that means the feature succeeded. And of course, the point that you mentioned about camparing is pretty valid. That's why we are looking at the average time taken.
4. I kind of covered the response for this point too in the above point.
What are your thoughts Chetan?
Let's first understand what an active user means. An active user is someone who makes payments through your application every day, at least once a day, or has bills to pay every month. These users become active users of your app because they use it frequently. One way to measure this is by whether the user is making payments or paying bills every month. Also, targeting such users and giving them more cashback compared to other users can lead to growth and expansion for Cred.
👍
@@DesignPilot Make sense?😅
Yes. So Cred can have a set of rules for example, making a payment, buying something once a month. Or making a UPI transaction per week.
Cred's active users(s) is:
A person who regularly uses credit card to make purchase and pay bill by using cred pay
A person who wants to pay rent via credit card
A person who uses UPI feature on daily basis and uses credit upi pay for that
Airbnb
metric/KPI : Time taken to find a airbnb
Company goal : Increase revenue by helping user to find an airbnb and book it asap
Also, Is Trade republic not available on mobbin anymore?
1. Why can't users pay the credit card bill in which ever way they want. Why only cred pay?
2. Why rent? What's your logic?
3. How many times a day should they use cred UPI?
Yes, we asked them to take it down due to legal reasons.
@@DesignPilot
1. Cred felt easy to use, they got it via referral, users gets rewards(cred coin), they want a premium service or well designed app to pay the bill which cred provides.
2. Cred provide a feature to pay rent via credit card and user gets benefits mentioned above so they uses cred over other platforms.
3. Chatgpt says average upi transaction/day is 2-3 so 2-3 times or more than that
Also, any comment on airbnb test?
@@DesignPilot
1. Cred provide few feature that other platform are not providing that user likes so they people active user of cred. Feature like: premium feel and well designed app, rewards like cred coin and referral.
2. They are using cred to pay rent as cred gives above feature
3. Chatgpt says 2-3 times UPI transation/day is average, so 2-3 times/day or higher
Also, any comment on Airbnb
1. Cred provide few feature that other platform are not providing that user likes so they people active user of cred. Feature like: premium feel and well designed app, rewards like cred coin and referral.
2. They are using cred to pay rent as cred gives above feature
3. Chatgpt says 2-3 times UPI transation/day is average, so 2-3 times/day or higher
Also, any comment on Airbnb
@@DesignPilot 1. Cred provide few feature that other platform are not providing that user likes so they people active user of cred. Feature like: premium feel and well designed app, rewards like cred coin and referral.
2. They are using cred to pay rent as cred gives above feature
3. Chatgpt says 2-3 times UPI transation/day is average, so 2-3 times/day or higher
Also, any comment on Airbnb
💙💙💙
With CRED
I guess the active user for CRED will be a user who has done n number of repayments using CRED maybe in a month
What’s a repayment?
And how many in a month?
Repayment as in the credit card or some sort of bills the user pays through cred
please tell me how to conduct secondary research please
You can Google this
@@DesignPilot i already google it but find nothing
@@DesignPilot please help me
@@AICreations432 To find ideas on "how to conduct secondary research as a UX designer", try searching on Google and looking through various websites, videos, and blogs. This can help you gain a better understanding of the process. If you still need help, consider taking a basic UX design course that is affordable or free. Avoid Figma courses, as they may not cover all aspects of UX design.
You can go for a AI name- perplexity
Company goal :- Increase revenue (and may be Growth & Expansion because of referrals by retained users)
Primary KPI :- Increase bookings
Secondary KPI :- User retention rate :- as it becomes easy for people to find airbnb accommodations, they'll have less of a burden comparing many different options and come out with the best decision. And they will prefer to come again and again for using the app.
But users aren’t really going to come to the app often right? They are going to come only when they have a need to travel. Even if the recommendations are excellent it won’t matter as users won’t really want to travel.
@@DesignPilot That's absolutely right. So I guess these may be the secondary KPI's :
Increase in Superhost status :- Introducing the ranking feature on Airbnb means hosts will want to get the best ranking for their listings because people list their places on Airbnb to earn a good amount. So, to get better ratings and reviews, hosts will work hard to improve the experience guests have when they stay at their place.
User satisfaction rate :- this can be tracked by looking at reviews and ratings after the new feature is introduced.
1. But who is this feature for? The hosts or the guests?
2. Can you explain the user satisfaction rate?
@@DesignPilot 1. The feature is mainly for the guests.
2. Tracking whether users are satisfied or not according to their feedback in reviews and ratings. The increase in the reviews and ratings as well as bookings compared to before the feature's implementation can depict the feature's success. Like a rise in the bookings for highly ranked accommodations and the decrease for lower ranked ones suggest that guests are using the rankings to make booking decisions.
So which metrics will you exactly be measuring and will you be able to quantify each metric you’re measuring?
In one of your previous video you told us that when we will have a list of problems that we got from user flow then we can define user stories and then we can say if we solve one of the particular problem then how it will impact the product. And to measure that we need metrics. So i think metrics is basically the name of the solution. And if we say we want to measure that metrics that's mean we want to see is this solution is working or not and how much working, how much not working. Right? But in this video you r talking about kpi and i think that is basically the sub-goals. And to achieve those sub-goals we want to identify the problems and come up with solutions then measure the solutions and that is metrics. That is my understanding. Please correct me if i am wrong. And i have some more question about metrics but i can't put it here. Can i DM you in Instagram?
Not sure I fully understand, but in the end you’re solving a problem for a user and for the business. You measure the metric/KPI/goal to see if you solved the problem or not. All the explanations you’ve given is the same thing.
Sure you can ask me on IG
@@DesignPilot thanks a lot for your time