Dec 14, 2022 • 15M

What to measure pre-product and pre-revenue?

Track desirability of your startup with this framework

 
0:00
-14:41
Open in playerListen on);
Actionable insights for founders poured in stories, straight from my experience mentoring 150+ startups and my PhD research on early-stage startups.
Episode details
Comments

🚀 Booyakasha: I surpassed the 800 subs milestone. Whoopwhoop.


Listen: 15 min
Fast reader: ±5 min
Slow reader: ±10 min

🎙️ Listen on Spotify or Apple (or anywhere else)


What to measure if there’s nothing to measure?

Being data-driven can be great. It helps you to get an objective sense of where you are. Which metrics are important for startups? You might have seen this list of 24 startup metrics pop up on LinkedIn or Twitter:

These metrics focus on relatively mature startups. For very early-stage, this doesn’t work well (Others agree)

It’s based on a 2015 blog post by A16z on metrics in a (SaaS) startup. Makes sense from their 2015 portfolio point of view: they look at software startups and assess their health.

However, if you are (very) early-stage, still figuring out your value proposition or solution, staring at your ARR, Churn or MoM growth rarely gives you a lot of insight. If you are pre-revenue, or pre-product, most of those metrics will be 0 or close to 0 for weeks.

These 24 metrics were never intended for that stage. But, if you find yourself in the early stage, you still might want to have something to give you a sense of progress.

What to measure in the early stage?

Of the 24 variables, 80% focus on the viability of the business. This is appropriate for businesses that are in operation with actual customers.

However, at the start of your journey, you will have 0. Therefore, at the very early stage, desirability is often riskier than viability.

Measurements of the early stage should show you desirability. To de-risk desirability, I like to see evidence on desirability. That is twofold:

  • Evidence that people want your solution

  • Evidence that people want your solution so badly they are willing to pay for it

The Desirability Scale

‘Measuring’

Measuring desirability isn’t straightforward. You can’t count the value of a single potential customer interview in the same way you can count the LTV of a paying user.

Problems interviews are good. Are they better than someone visiting your landing page? What about someone participating in your pilot? Is that better than getting a signup for your waitlist?

We need something to assess the evidence of these types of signals. In The Desirability Scale, I show you the ‘empirical weight’ of certain signals. The scale shows which signals weigh a lot when it comes to evidence for desirability and which weigh a little.

Explaining the scale

First I will explain the scale and then show some drawbacks.

Five customer currencies

To better understand the weight of evidence, we need to look at the currency the customer pays in. The first thing to understand is that customers don’t always pay with money. Early-stage (potential) customers you interact with do always spend something on you:

  • 👀 Attention: are they willing to look at you?

  • ⏰ Time: are they willing to spend time with you?

  • ⭐ Reputation: are they willing to their risk reputation for you?

  • 💪 Effort: are they willing to put effort in?

  • 💵 Money: are they willing to spend money on you?

Customer interactions

These currencies come in five different phases. Ultimately, you need proof of money coming in. But at the start there is none.

Identify 5. Each phase has a different composition of currencies that are dominant.

1. People engage with you as a person

Dominant currencies: ⏰ Time, 👀 Attention

  • Problem talks: Are people willing to talk with you? That’s a starting signal that you need. Tips on interpreting these problem talks here.

  • 👀 Social media attention: follows are nice, but unless a huge number, not very impressive. Anyone can get couple hundred friends to join?

  • 👀 Media writing about you: Often vanity metric. It’s good for the ego, often it results in traffic that doesn’t convert well. Learn about the influence of news coverage on product-market fit here (YC story)

2. People engage with you for what you might offer them

Most people are willing to talk at least once or look at you at least once. But are they willing to give you a little more?

Dominant currencies: ⏰ Time, 👀 Attention, ⭐ Reputation

  • Spoken with people that have the problem If you find people that actually have the problem (or JTBD), that’s a great signal!

  • Follow-up talks: if people are willing to talk again, that’s a good signal, that means the problem is important to them

  • Referral to others: if people are willing to refer you to their network, means you are credible enough to do that. Good signal. (this can occur as well during later phases, but in this phase, this is a telling tale).

  • 👀 Ads/visits/signups: in this stage, you can do smokescreen tests and see if you have desirability around your value proposition. They are just committing their email though, so it’s not a lot of evidence. Depending on the volume, I find follow-up talks more convincing.

3. People commit to using your product

In the previous phase, people were engaged but you weren’t reeling them in yet. This phase is about that.

Dominant currencies: ⏰ Time, 💪 Effort

  • ⏰ 💪 Customer commits to free pilot: at the end of your problem-talk (or follow-up talk) or in your email list, ask them if they are interested in a pilot. If not, that’s a signal that you might not be talking to the right customer. Or your solution is not the right one.

  • ⏰ 💪 Customer active in free pilot: Not everyone that says yes will actually be onboarded to your pilot successfully. That is okay, but something to watch out for. Saying yes is easy, but putting in the effort is hard.

  • ⏰ 💪 Customer happy in free pilot: Are your pilot users getting value? This is a key question. There are many ways to measure customer satisfaction, including NPS and CSAT. The best way is to ask them. Dating App Breeze phoned their first 100 dates, so that’s 200 phone calls, just to understand the value they provided.

4. People commit to paying to use your product

In the previous phase, you focused on value generation. This fourth phase focuses on signals of value capture.

Dominant currencies:Time, 💪 Effort, 💵 Money

  • 💵 Paid pre-order: Not all products are easy to prototype. Hardware can be harder to generate value in an MVP way. Pre-orders can give you an idea of how desirable your solution is, including willingness to pay. It usually shows they are willing to wait for a longer period for delivery cause most hardware products have that downside. Good desirability signal.

  • ⏰ 💪 💵 Customer commits to paid pilot: You can ask your free pilot customer to join a paid program. It can help to make this explicit at the start of the free pilot. “This first pilot is on us, we want to learn how to help you”. Or you can ask for money straight away. If you can do that, that’s more evidence. You need to decide if that is appropriate to ask right away, and if you feel comfortable with that.

  • ⏰ 💪 💵 Customer active in free pilot: Not everyone that says ‘yes’ will actually be onboarded successfully. If they pay money, it’s likely they are more committed. Definitely reach out to those who are not active. A goldmine for learning.

  • ⏰ 💪 💵 Customer happy in paid pilot: This is your holy grail before launching. If your paid users are happy, that means that the value generated is high enough for the price you set. This is an extremely good signal. Note: this doesn’t mean your pricing is good and set in stone.

  • 💵 Scalable sales: A pilot is a soft launch with training wheels, pilots don’t scale. You do a pilot to learn. With scalable sales, your primary goal here is not to learn but to earn. You still want to learn, but that doesn’t stop. It’s the sale according to a blueprint that you want to scale. Self-serve, without extensive support that comes with a pilot.

5. Customers champion for you

Dominant currencies: ⭐ Reputation, 💵 Money, 💪 Effort

  • 💵 Happy scalable customers: Customers are happy with your non-pilot solution. It could be that you being there during the pilot brings a lot of the generated value. I have done experiments where I manually onboarded customers on a platform. NPS went up. However, when we rolled out the product with similar features, NPS went down. Make sure you are not the product

  • 💵 Recurring sales: If applicable to your solution type. Recurring sales are even better than selling once. This means you really provided value.

  • 5-star reviews: customers are willing to risk their reputation to spread your gospel. That’s as good as it gets. High NPS or high CSAT counts too.

  • Word of mouth: The holy grail, free marketing. Word of mouth is the reason people measure NPS. There’s nothing better than being recommended by a friend.

Learn why they are not progressing

You should be curious to understand why your customer is not progressing to heavier types of evidence. Why can’t you have a follow-up talk? Why is this customer not interested in a pilot? Why is this customer not happy with the pilot? And so on. Keep on asking why.

⚠️ Four drawbacks

1. B2C / B2B SaaS focus

This scale is developed with a B2C or B2B SaaS focus. This comes from my experience and research focus. The scale could apply to others, but I’m not experienced in other fields such as Deep Tech or MedTech. Let me know.

2. Imprecise

It’s extremely hard to pinpoint how much evidence a signal holds. Therefore there is a lot of overlap between the signals. This is a suggestion by me of how you can interpret the signals. How do you interpret it?

3. Adding numbers complicates things even more

It’s tough sometimes to tell the differences between two signals. Especially if we add numbers. Would you rather have:

  • 1000 genuine Instagram followers or 50 talks with people on the problem

  • 500 email signups or 5 free pilot participants

  • 20 paid pilots or 5 actual scalable sales

I don’t have the answers. This overview should be a starting point for discussions. It is not rocket science.

4. Only version 2

So this is version 2. Meaning, I’m not done. And open to feedback. Let me know what you think, and vote below.


How was this one?

Voters live longer!

Great - Good - Meh


Need 1-on-1 help?

I help (aspiring) founders build products people want. Do you need to get to your first customer, scope your MVP or develop your business model? Schedule a free mentoring intake below.

Get 1-on-1 help