Product Management for Engineers · Page 1
Discovery vs Delivery: The Four Product Risks and the Build-to-Learn Loop
Product work is risk reduction that happens before engineering time is committed. This page defines the four risks (value, usability, feasibility, business viability), shows what the measured failure rate of ideas implies about how much testing is worth paying for, and matches each test (interviews, prototypes, written vetting, randomized experiments, shaping) to the risk it can and cannot reduce. Afterwards you can classify a proposed feature by its least-tested risk, choose the cheapest technique that tests it, and say which risk is yours as the engineer.
1. Two ways to run a team: a roadmap of outputs or a set of problems
The vocabulary this page uses comes mostly from one organization, Silicon Valley Product Group (SVPG), so the first thing to fix is how the argument is built. Moore and Cagan describe a feature team as one that receives a list of features and projects from business stakeholders, each of whom understands a specific obligation to the business and proposes what they believe will meet it, and is then asked to turn that list into a roadmap with dates and deliverables1. An empowered product team receives instead "a set of problems to solve and desired outcomes to achieve", and is tasked with a solution that works for both the customer and the business1. The rest of the page assumes the second model, because the first one has no discovery step to talk about.
1.1 What each team is given and held to
| Dimension | Feature team | Empowered product team |
|---|---|---|
| Input | A roadmap of features and projects handed down by stakeholders | A set of problems to solve and outcomes to achieve |
| Who owns value and viability of a feature | "Whomever requested that feature on the roadmap"; the team designs and builds it | The team itself, which must find a solution that is valuable, usable, feasible and viable |
| Held accountable for | Output: "they are only there to produce output" | Results: if the feature misses the outcome, iterate until it does |
| If the feature does not pay off | The team points out it was not their job; the stakeholder complains it was not what they hoped or came late | The miss is the team's to close |
| Engineers' role | "Simply there to build what is requested" | Also "responsible for helping to identify the right solution" |
| Residue | "Orphaned" features awaiting an iteration that rarely comes; accumulating technical debt | Not claimed |
| Time to a working result (SVPG estimate) | 3-5 iterations, each a build: "on the order of 1-2 years" | The same 3-5 iterations "in 1-2 weeks" of discovery, with the product version live "a few weeks after that" |
1.2 The mechanism is who holds the uncertainty
The article's causal claim is narrower than the slogan. A feature on a roadmap is "a potential solution to some underlying problem", and that problem can belong to the customer (they cannot figure out how to use the product) or to the company (it costs too much to provision the product)1. In the feature-team model the designer on the team designs the feature and the engineers build it, but "the value and viability of the feature itself is the responsibility of whomever requested that feature on the roadmap"1. Two things follow from that arrangement. The people who know the most about what is cheap, slow or newly possible are not asked whether the feature is worth building. And the person who is asked has no mechanism for finding out before the build, so the build is the test.
That is why the same article ties accountability to the input. It states that in this model "you can't hold the feature team accountable to business results", because the team can say the feature was what it was asked for1. The stakeholder, for their part, will "almost certainly complain" that what shipped was not what they hoped, so the model produces mutual distrust as a by-product1. The residue SVPG points to is orphaned features: shipped, producing no real value, and waiting for a second iteration that rarely arrives, while their maintenance cost compounds as technical debt1.
1.3 Where the engineer stands in each model
Three unrelated organizations describe the engineer's position in almost the same words. SVPG calls engineers in a feature team "effectively mercenaries" and claims the engineer's advantage in an empowered team is daily contact with the enabling technology, which "puts them in the ideal position to see what's just now possible"1. The Shape Up method from 37signals, the Basecamp developer, gives "full responsibility to a small integrated team of designers and programmers" who define their own tasks and adjust scope, and contrasts it with methods where "managers chop up the work and programmers act like ticket-takers"3. PostHog's engineering handbook, written by a developer-tools company, lists among its "bad product engineers" those who "only work on things they've been told to work on", "never challenge why they're being told to work on something" and "don't track if the things they've built are being used"4.
All three are descriptions of one organization's practice, and none publishes outcome data comparing the models. What the convergence does give you is a working definition of the engineer's stake in discovery: the engineer is the person who can say, before the build, what is feasible, what has become cheap, and what will be expensive to keep. Who formally decides is the role question that page 5 takes up; this page stays on the risks those decisions are meant to reduce.
2. A feature is a hypothesis: four base rates that are not one number
If a feature is a potential solution to an underlying problem1, it is a hypothesis about that problem, and hypotheses have a hit rate. The best-documented hit rates come from companies that run large numbers of randomized experiments, and Kohavi, Tang and Xu build one of their three organizational tenets on them: the organization "recognizes that it is poor at assessing the value of ideas"5. Their framing is that "features are built because teams believe they are useful, yet in many domains most ideas fail to improve key metrics"5. The same chapter names the management habit that hides this: it is "often easier to generate a plan, execute against it, and declare success, with the key metric being: “percent of plan delivered,” ignoring whether the feature has any positive impact to key metrics"5. That is the feature-team scorecard from Section 1 written as a metric.
2.1 The figures, side by side
| Figure | Population | "Success" means | Provenance |
|---|---|---|---|
| About 1/3 | Ideas at Microsoft that "pass all the internal evaluations" and get implemented6 | Improved the key metric the experiment was designed to improve | Originating paper (Kohavi, Crook, Longbotham 2009), restated in the textbook56 |
| 10-20% | "Some measures" in Bing and Google, "well-optimized domains" | Success rate of experiments on those measures | Textbook cites Manzi (2012); that book was not consulted here5 |
| About 30% | Slack monetization experiments | "Positive results" | A 2019 post by Slack's Director of Product and Lifecycle, quoted in the textbook5 |
| "90% wrong" | What Netflix "tries" | Undefined; "considers ... wrong" | Mike Moran (2007) writing about Netflix, relayed by the textbook and the 2009 paper56 |
| 10-20% | Features and projects on product roadmaps generally | "Generate a positive return on investment" | "Most industry analysts"; no analyst, method or year named1 |
2.2 What the figures do and do not say
Three properties of the experiment-based rows matter more than the headline fractions. First, the population is already filtered. The Microsoft authors describe their statistic as "the percentage of ideas that pass all the internal evaluations, get implemented, and fail to improve the metrics they were designed to improve"6. The two-thirds that failed had survived whatever review the company applied before building, which is a direct measurement of how little that review predicts. Second, "fail to improve" is a statement about one pre-declared metric. The text available for this page does not split failures into flat and harmful, so do not read two-thirds as two-thirds that did damage. Third, the denominators are web and software services where users can be randomized, changes are easy to make, and a metric can be read within one to two weeks (the textbook asks for thousands of experimental units)5. No source gathered for this page measures an idea hit rate for a developer-infrastructure product or an on-device runtime, so the range is a prior borrowed from other domains, not an estimate for yours.
The other figures in the same textbook passage are different instruments again. One practitioner reports he could guess the outcome of a test correctly about 33% of the time, which is a calibration score for human prediction, not a success rate; another that "80% of the time you/we are wrong about what a customer wants"; a third at Etsy that "nearly everything fails"5. The authors add one hedge: "Not every domain has such poor statistics"5.
2.3 The cheap-test counter-case
The textbook opens with the example that shows why a low hit rate and an unreliable prior belong together. In 2012 a Bing employee suggested lengthening ad titles by merging in the first line of ad text. The idea "was prioritized low and languished in the backlog for more than six months until a software developer decided to try the change, given how easy it was to code"5. The experiment raised revenue by 12%, "over $100M annually in the US alone", without significantly hurting key user-experience metrics5. The authors draw the lessons that matter here: it is hard to assess the value of an idea; a return of that size on a few days of one engineer's work "is about as extreme as it gets"; such wins are rare ("Bing runs over 10,000 experiments a year, but simple features resulting in such a big improvement happen only once every few years"); and "the overhead of running an experiment must be small", which at Bing meant an experimentation system the engineer could use5.
Put the two halves together. Most ideas fail, and the people ranking them cannot tell the winners in advance, so ranking by predicted value will bury some of the best ideas along with the worst. What changes the economics is not a better ranking but a cheaper test. How a team should order work given that is page 3's question; this page asks the earlier one, how cheap a test can be made and what it can tell you. The next section turns the base rates into arithmetic.
3. What the base rate implies: derived arithmetic
Everything in this section is derived by the writer from the Section 2 inputs and from SVPG's cost estimates. No source performs these calculations, and the two parameters that matter most (how well a cheap test separates good ideas from bad) are not measured anywhere in the sources collected for this page. The purpose is to see how much a given failure rate can justify spending on testing before building, and which assumption carries the answer.
3.1 Builds per success
Suppose every candidate idea is built and each works with probability \(p\), independently. The number of builds up to and including the first success is geometric, with mean \(1/p\). At the Microsoft rate of one third that is 3 builds per success; at 30% it is 3.33; at 20%, 5; at 10%, 10. The mean hides a wide spread. At \(p = 0.2\) the chance that five consecutive builds all miss is \(0.8^5 = 0.328\); at \(p = 0.1\) the chance that ten in a row miss is \(0.9^{10} = 0.349\). A team that builds five features and sees none move the metric has not necessarily been unlucky or incompetent: at \(p = 0.2\) that happens about one time in three.
The range 3 to 5 builds per success (\(p\) between 1/3 and 1/5) coincides with SVPG's "3-5 iterations" per feature1. The coincidence is not corroboration. The first counts how many different ideas you must try; the second counts how many times you must revise one idea. Treat them as separate estimates.
3.2 The filter model: what a screen is worth
Now put a test in front of the build. Let a build cost 1 unit (a build-equivalent), and let the test cost \(d\) units per candidate. The test passes a good idea with probability \(s\) (sensitivity) and a bad idea with probability \(f\) (false-pass rate); only passers are built. A candidate passes with probability \(q = ps + (1-p)f\), so the build cost per candidate is \(q\), and successes per candidate are \(ps\) (a good idea that passes is assumed to work when built). The expected cost per success is then
$$ \text{cost per success with screening} = \frac{d + q}{p\,s} \qquad\text{versus}\qquad \frac{1}{p}\ \text{without.} $$Screening wins when \(d + q < s\), which rearranges to
$$ d^{*} = (1-p)\,(s - f). $$Read it as an allowance: the most you can spend per candidate before testing stops paying for itself. It grows with the failure rate \(1-p\) and with the test's discrimination \(J = s - f\) (the gap between how often it passes good ideas and bad ones). A test that passes everything (\(s = f = 1\)) has \(J = 0\) and an allowance of zero, which is the correct answer: it tells you nothing, so no spend on it is justified. A perfect test has \(J = 1\) and an allowance of \(1 - p\): at the Microsoft rate, two thirds of a build per candidate; at a 10% rate, nine tenths.
| Base rate \(p\) | Builds per success \(1/p\) | \(d^{*}\) at \(J = 1\) | \(d^{*}\) at \(J = 0.5\) | \(d^{*}\) at \(J = 0.25\) |
|---|---|---|---|---|
| 0.10 | 10.00 | 0.900 | 0.450 | 0.225 |
| 0.20 | 5.00 | 0.800 | 0.400 | 0.200 |
| 0.30 | 3.33 | 0.700 | 0.350 | 0.175 |
| 1/3 | 3.00 | 0.667 | 0.333 | 0.167 |
Figure 3B contains the result that is easy to miss. SVPG's claim is per iteration: each prototype is at least an order of magnitude cheaper than a build1. But a candidate typically needs several. Three iterations at one tenth is 0.3 build-equivalents per candidate; five is 0.5. A middling screen (\(J = 0.5\)) at \(p = 0.2\) has an allowance of 0.4, so three iterations pay and five do not. Only a screen that discriminates well, or prototypes closer to one hundredth of a build, justifies the full 3-5 iterations across the whole base-rate range. "An order of magnitude cheaper" is a necessary condition for discovery to win, not a sufficient one.
Cost per successful feature: build every candidate, or screen first
Formula: cost per success \((nc + ps + (1-p)f)/(ps)\) against \(1/p\); break-even \(d^{*} = (1-p)(s-f)\). Defaults are placeholders inside the Section 2 range, not measurements: no source reports \(s\) or \(f\) for interviews, prototypes or written reviews. A negative saving or a break-even below the spend means building everything is cheaper.
3.3 The iterate model: SVPG's version
The filter model has discovery discard bad ideas. SVPG describes something different: discovery improves one idea until the version that gets built works. If a feature needs \(n\) iterations to reach the result and a feature team builds each one, the cost is \(n\) builds. If a product team runs the same \(n\) iterations as prototypes at a fraction \(c\) of a build and then builds once, the cost is \(nc + 1\). With SVPG's inputs, \(n = 3\) to \(5\) and \(c = 0.1\), that is 1.3 to 1.5 builds against 3 to 5, a saving factor of 2.3 to 3.3. At \(c = 0.01\), which SVPG says holds "in many cases" (prototypes "two orders of magnitude less than the product")1, the factor is 2.9 to 4.8. Those are the writer's combinations of two of SVPG's estimates; the article gives elapsed times instead (about 1-2 years of roadmap-driven builds against 1-2 weeks of discovery plus a few weeks to ship)1.
The assumption doing all the work is that a prototype iteration returns the same information as a built iteration. That is true for some questions and false for others, which is what Section 6 sorts out. A prototype will not tell you the latency of a kernel on silicon it does not run on.
3.4 What the arithmetic leaves out
- Failed builds are not wasted if they are measured: the Microsoft authors call a failed experiment "not a mistake: learn from it"6. The filter model charges the full build against a failure and so overstates discovery's advantage when the build is itself an instrumented test.
- Discovery raises \(p\): iterating on a prototype turns a poor idea into a better one, which the filter model ignores. This understates discovery's advantage.
- The allowance scales with build cost \(B\): in absolute terms it is \((1-p)\,J\,B\). Where a change takes a few days and a measurement system exists, as in the Bing case, the absolute allowance is small and the cheapest test of the idea is to build it. Section 7 returns to this.
- Waiting has a price. Delay on a feature that pays is a cost that neither model charges; cost of delay is page 3's subject.
- Trust is a cost of testing on live users. SVPG warns that customers "start feeling abused because of constant, erratic change"2. Section 5 covers it.
4. The four risks
A feature can fail for reasons that have nothing to do with each other, and the cheapest test for one reason says nothing about another. SVPG's four-risk framing exists to keep those reasons separate. The essay that states it is Marty Cagan's 2017 "The Four Big Risks", the only source here for the history of the framing, so what follows about why it changed is Cagan's account of his own thinking and not independently confirmed8.
4.1 Definitions and how each one fails
| Risk | Definition | How a good-sounding solution fails |
|---|---|---|
| Value | "Whether customers will buy it or users will choose to use it" | "The customer is less impressed, and unwilling to switch"; the commonest reason |
| Usability | "Whether users can figure out how to use it" | "The solution is just too complicated to use" |
| Feasibility | "Whether our engineers can build what we need with the time, skills and technology we have" | Solutions that "sound doable turn out to be much more difficult to build than we initially imagined" |
| Business viability | "Whether this solution also works for the various aspects of our business": fit with the go-to-market or sales channel, partner contracts and legal compliance, cost-effective customer acquisition, monetization, brand promise | Customers love it, but it "would not be compliant, or secure, or legal, or something that we can afford to market and sell, or that can be effectively monetized" |
Two features of the feasibility definition are worth an engineer's attention. It is not binary: it is relative to "the time, skills and technology we have", so the same idea can be feasible with a specialist and a longer schedule and infeasible without them8. And the 2026 FAQ extends the test group to "engineers on product teams we depend on", so dependencies you do not control are part of the risk2.
4.2 Why three became four, and which risk is hardest
In the first edition of Inspired, Cagan says, successful products were "valuable, usable and feasible", with "valuable" defined as valuable to customers and to the business. He later concluded that the three-part version "was obscuring some pretty serious risks and challenges, and making it too easy for product managers to focus on customer value and overlook business value"8. The new edition therefore separates value (will they choose it) from viability (does it work for us). He still holds that "the toughest risk is usually value risk", and that viability risks "are too often under-appreciated and under-estimated (or simply avoided) by the product manager"8. His sequencing advice is the sentence that matters for planning: "Tackle the big risks early - especially value risk and business risk"8.
Moore and Cagan's 2022 article uses the same four tests under the names valuable, usable, feasible and viable, and spells out viability as "constraints in marketing, sales, finance, service, legal and compliance"1. That is repetition by the same author, not independent adoption. The claim that value is hardest is, like the rest, an assertion from experience: neither essay supplies a count.
4.3 Ownership in a trio, and what the engineer is still expected to notice
Cagan assigns the risks to three roles. The product manager is "responsible for the value and viability risks, and overall accountable for the product's outcomes"; the product designer is responsible for usability and "overall accountable for the product's experience"; the product lead engineer is responsible for feasibility and "overall accountable for the product's delivery"8. Torres defines the same three-person unit, "a product manager, a designer, and a software engineer", and calls it the product trio9. The 2026 FAQ is explicit that ownership is not decision monopoly: the product manager is not "the decider", and each member of the team "uses their experience and judgement to make dozens or even hundreds of decisions every day"2. Who has formal decision rights under each team model is page 5's subject.
The test-party column of Figure 4 carries one more practical rule from the FAQ: "normally we do not need to test every risk with every prototype on every constituent. We test the relevant party based on the level of risk"2. A prototype aimed at value and usability goes in front of users; one aimed at feasibility goes in front of engineers; one aimed at viability goes to sales, marketing, legal, compliance, finance or operations2.
What follows for the engineer is a judgment more than a citation. Cagan gives the engineer one risk. But an engineer on a product team is expected to notice trouble in the other three, and the sources that come nearest to saying so are behavioral lists. SVPG says engineers on an empowered team are "responsible for helping to identify the right solution", not only for building it1. Shape Up says shaping does not require being a programmer, but "you need to be technically literate", and it asks the shaper to be critical about the problem: "What are we trying to solve? Why does it matter? What counts as success? Which customers are affected? What is the cost of doing this instead of something else?"3 PostHog's handbook makes the engineer's side concrete in its list of bad product engineers, who "Can't explain who their product is built for", "Don't know their product's competitors", "Don't track if the things they've built are being used" and "Never remove features or complexity"4. Read together, those are value noticing (who is it for, is it used), viability noticing (competitors, what to remove and keep paying for) and usability noticing, which the handbook assigns to the engineer directly: usability tests for new big features are, in its words, "Run by the engineer building that feature"10.
4.4 Feasibility is a tail risk, not a yes-or-no
Shape Up gives the sharpest description of what feasibility risk looks like, even though it is a method about shipping on time and not about building the right thing (Section 6.6). Its author writes that well-shaped work "looks like a thin-tailed probability distribution": there is a slight chance it takes an extra week, but the pieces are "defined enough and familiar enough" that it should not drag on. If the shaping left "technical unknowns, unsolved design problems, or misunderstood interdependencies", the project "could take multiple times the original appetite" and "the right tail stretches out"3. Figure 5 draws that shape; it is a schematic of the authors' description, not data.
Two practices follow, both aimed at moving feasibility risk before the commitment. The first is a question rewrite. "Beware the simple question: 'Is this possible?' In software, everything is possible but nothing is free"; the useful form is "is X possible in 6-weeks?", asked of "technical experts" at a whiteboard while the idea is still rough, with the explicit purpose of "hunting for time bombs"3. The second is that the same chapter's own cautionary case is not a coding failure. Basecamp bet on redesigning how client projects were presented on the home screen without having done the shaping work "to validate that a viable approach existed", and no suitable design emerged inside the six weeks, so the project was abandoned3. Feasibility risk includes not knowing whether a good answer exists at all, and that is the version an engineer is well placed to flag.
5. Build to learn, build to earn
SVPG's 2026 vocabulary names the two activities by what the building is for: "building to learn (known as product discovery) versus building to earn (known as product delivery)"2. The distinction matters because the artifacts look alike. A prototype and a production feature can be written by the same engineer in the same language. What differs is the quality bar, the audience, and the question the artifact is supposed to answer.
5.1 Discovery is defined by its incentive
Moore and Cagan define discovery through what a team held to results has reason to want. For such a team, "while time to market is still important, what is most important is time to money", meaning the time to achieve the necessary outcome. If a shipped feature misses, the team iterates until it does, and so "the incentive moves to being able to quickly determine if a particular product idea or approach will work"1. The team "could simply build their various ideas and see what works", they add, "but this would take much more time and essentially use our customers as guinea pigs"1. The alternative is prototypes: "Mostly the product team creates prototypes, which come in various forms but all are very fast and inexpensive to create, and each type designed to test different risks or assumptions"1.
The cost rule they give is a rule of thumb: any idea tested in discovery "should be at least one order of magnitude less expensive and faster than using the engineers to build, test and deploy an actual product", and "in many cases" two orders of magnitude1. The iteration claim is conditional in the original: "If, on average, it takes 3-5 iterations for a typical feature to reach the point where it generates the necessary business results", and a feature team builds each, "then we are often talking on the order of 1-2 years"; with discovery skills on the team those iterations "can happen in 1-2 weeks"1. Keep both as what they are: one organization's estimates, stated as an "if", with no measurement behind them. The 2026 FAQ is from the same organization and repeats the framing, so it does not count as independent support2.
5.2 What discovery is not for
The FAQ makes a claim an engineer should know before walking into a planning meeting: discovery is not for confirming that the problem is real. "No, and if you're perceived as thinking that's your job, you will almost certainly lose the trust and confidence of your leaders." The argument is that product strategy, usually owned by product leaders, picks the problem; that it is "quite rare" for leaders to prioritize something that is not a problem; and that the hard part is solving it: "The hardest part, by far, is almost always solving the problem." If the team does misunderstand the problem, "we will see this very quickly once we start testing prototype solutions"2. The implicit contract is that leaders choose worthy problems and the team finds solutions that work for the customer and the business; asked how the team knows this is the most important problem, the FAQ answers "We don't, and we can't"2.
This is a position, and it has a precondition: the "problem" handed to the team must be a problem. Many requests arrive as solutions (a feature someone wants). Torres's opportunity-tree method assumes the opposite premise, that trees "are nearly impossible to create when you don't have good inputs" and that opportunities should be drawn from what customers say in interviews, because ones generated "off the top of our heads" bring "our own biases and half-truths into the picture"9. How to turn a request into a problem with a boundary is page 2's subject; the point here is only that the two sources place the uncertainty in different places, and Section 6 and the disagreement section carry that through.
5.3 Learning on live users has a price
Both SVPG articles warn against treating customers as the test environment. The FAQ's principle is "Test Ideas Responsibly": "it's very easy for even the most loyal customers to start feeling abused because of constant, erratic change", and a paying customer who concludes the team is "just throwing things at them in order to see what sticks" can damage revenue, reputation and the customer-success team's work. Opt-in testing is acceptable and counts as discovery; discovery's quantitative and qualitative techniques exist "to protect our general users and customers from this experimentation"2.
The same FAQ is careful not to say learning stops at launch. Once the product is in production the team collects "more actual usage data than ever before", and "the ultimate test of whether we have solved the customer problem and delivered the necessary outcome can only be established once our product is live"2. Discovery reduces the risk of the bet; it does not replace measuring the result. PostHog's engineering handbook goes further and calls shipping "often the best way to validate an idea", with mitigations for the damage a poor early version can do (Section 6.5)4. The two positions differ on how much to pay for evidence before shipping, not on whether shipping is evidence.
6. Matching the technique to the risk
No technique tests all four risks, and each has a characteristic way of misleading you. Table 5 lists the techniques the sources describe, the risk each can reduce, what the source says about its cost, and the blind spot. The cost column quotes each source's own signal; the signals are not measurements on a common scale.
| Technique | Risks it can reduce | Cost signal in the source | Blind spot |
|---|---|---|---|
| Story-based customer interview | Value; some usability signal from past behavior | A conversation; PostHog PMs aim for 1 per week; "response rates are typically low" | Stated interest is not use: "People want to be likable, so they'll often say they want what you're working on, even if they don't"; says nothing about feasibility |
| Prototype shown to the right party | Any of the four, set by who sees it: users (value, usability), engineers (feasibility), stakeholders (viability) | "At least one order of magnitude less expensive and faster" than a build (SVPG estimate) | Non-functional needs such as "scale expectations" are "not easily communicated in a prototype" |
| Written vetting (PR/FAQ) | Value and viability on paper; the internal FAQ lists "technical, financial, legal, or operational" problems | First draft "a few hours, not a few days"; full process "months of work" at Amazon | A document records the authors' belief; no user has reacted to anything |
| Shaping with technical review | Feasibility inside a time budget; checks usage data against assumptions about customers | A whiteboard session with technical experts; a generalist, or one or two collaborators, does the shaping | "surprises might still happen and icebergs could still emerge" (the method's own words); no technique for value |
| Randomized controlled experiment | Value and usability as revealed by behavior on a pre-declared metric; guardrails catch harm | "The overhead of running an experiment must be small"; the change must already be built | Needs enough units, agreed metrics, easy changes; tests tactics, not strategies; not a substitute for formative research |
| Ship a descoped version and watch | All four, in production | "Potentially at the highest cost as you have to build it" | A poor early version can damage the brand; customers become the test environment |
6.1 Interviews, and what kind of evidence they produce
Torres's method collects stories: "Tell me about a time when..." prompts about specific past behavior, from which opportunities "emerge". The reason she gives for insisting on interviews is that generating opportunities "off the top of our heads" imports "our own biases and half-truths"9. Items arriving through sales conversations, support tickets and feature requests are not discarded, but she notes that "most of the time ... it is missing context", so she uses them as "inspiration for what to explore in my upcoming interviews"9. The page is dated 2023-12-06, prescribes the practice without outcome data, and sits on a commercial site that sells courses and discloses a licensing arrangement with a tooling vendor (Vistaly)9.
PostHog's engineering handbook supplies the behavioral edge that makes an interview evidence rather than conversation. Ask open questions, and then "look for evidence that users have actually done anything about the problems they say they have": "Have you tried to solve this problem before? What did you do?" and "Has your company spent money trying to solve this before?"4. The same page is skeptical of upfront validation: "it's debatable how well you can validate ideas up front", the founders "had to pivot 5 times - despite getting positive feedback on new ideas almost every time", and talking to users first "can probably help remove totally stupid ideas fast, but for the majority of ideas 'this could work', it only has a limited amount of benefit in our experience"4. That is an anecdote and a view, not a measurement; but it names the failure that matters, which is positive feedback that does not predict use.
| Source | Cadence and who | Basis |
|---|---|---|
| Torres (Product Talk) | Interview customers "continuously—week over week"; revisit the opportunity space "for every three to four interviews"; the tree is a product-trio activity | Author's practice; sells courses |
| PostHog, product managers | "Our rule of thumb is 1 interview per week per PM"; PMs "organize user interviews" | Company handbook (living document) |
| PostHog, engineers | Ask to join PM-run interviews or message users directly; the engineer building a big feature runs its usability tests | Company handbook |
| Shape Up | No cadence prescribed; "Really important ideas will come back to you"; shaping checks usage data and technical assumptions | Builder's published method |
| SVPG (2022, 2026 articles) | No interview cadence stated; prototypes tested against the four risks | Authors' practice |
Whether a weekly cadence is attainable, or useful, for a product with a small number of integrator customers is not addressed by either interviewing source. The cadence prescriptions should not be assumed to transfer to a runtime sold to a small number of integrators. Treat them as a ceiling to scale down from, and keep the behavioral questions, which cost nothing at any customer count.
6.2 Prototypes, and what a prototype cannot carry
SVPG's prototype is deliberately broad: "various forms but all are very fast and inexpensive to create"1. The 2026 FAQ pushes it further: the prototype becomes the primary specification ("prototype as spec"), and the PRD survives only for what a prototype shows poorly, "usually an enumeration of specific use cases, and non-functional requirements (e.g. what are the scale expectations?)"2. That admission defines the blind spot. A prototype demonstrates behavior and flow; it does not demonstrate throughput, latency under load, memory behavior, or anything else that is a property of the real system on real hardware. For feasibility testing, the prototype that works is a different kind: Shape Up's "R&D mode", in which "a senior team spikes the core features to define the core architecture"3, and the whiteboard technical review of Section 4.4. Calling both "prototypes" is fine as long as you remember that the thing being tested differs: a UI prototype tests a person's reaction, a spike tests the machine.
6.3 Written vetting: the PR/FAQ as a funnel
The authors of Working Backwards describe the PR/FAQ as "a systematic way to vet ideas and create new products", whose "principal tool is a second form of written narrative"7. The press-release half forces the author to name a specific customer and the problem from that customer's point of view; the internal FAQ must "cover all the challenging problems that need to be solved to build the product, whether technical, financial, legal, or operational"7. Their reviewers' checklist asks, among other things, whether the customer and problem are clearly defined, whether customers would be expected "to change their behavior to buy/use the solution", whether the market and payback are big enough, and what business, resource, technical and legal constraints must be faced7.
The cost structure is the point. A first draft "should take only a few hours, not a few days", so a team can write many; the authors call the aim "a funnel, not a tunnel", a process that "accepts and considers many new product ideas at the onset, progressively eliminating lower-potential ideas", as opposed to one in which "all or nearly all new product ideas that enter the process are built and launched"7. That is the filter model of Section 3.2 in prose: a cheap screen in front of an expensive build. The same page is candid that the full process is not cheap: it "required months of work for the most successful products at Amazon", and the team that built AWS "spent more than a year writing, revising, debating, and discussing PR/FAQs"7. Two cautions. The page is an account by two former Amazon executives who sell a PR/FAQ course on the same site, so it is a practitioner's description, not independent evidence7. And a document tests an author's belief against reviewers' skepticism; it cannot produce a customer reaction. Mechanics of the format are page 5's.
6.4 Randomized experiments: the arbiter, and the edge of their reach
Kohavi, Tang and Xu argue that when controlled experiments can be run, "they provide the most reliable and sensitive mechanism to evaluate changes"5, and they list the conditions under which they can. There are four technical ingredients: experimental units that can be assigned to variants with little interference; enough units, where they "recommend thousands"; key metrics, "ideally an OEC", that are agreed and practical to evaluate; and changes that are easy to make5. Where a condition fails, the book says so plainly: you "cannot run a controlled experiment on mergers and acquisitions (M&A)", because the counterfactual cannot coexist; hardware devices "may have long lead times for manufacturing", so experiments with users "are rarely run on new hardware devices"; and software is easier to change than hardware, server-side easier than client-side5. Part III of the book is given to complementary and alternative techniques for those cases, with chapters on logs-based analysis, human evaluation, user experience research, focus groups, surveys, external data and observational causal studies (table of contents only; the chapters were not read)5.
Three further limits matter for the question this page asks. First, an experiment tests a tactic and not a strategy: "a single experiment failing to improve the OEC may be due to the specific tactic being poor, not necessarily indicating that the overall strategy is bad"5. Second, the authors recommend a portfolio: most investments should aim to optimize "near" the current location, but "a few radical ideas should be tried", because "most big jumps fail (e.g., big site redesigns)" while the rare successes can pay for the failures. Their cautionary case is Bing's social-network integration, which ran about two years, cost "over $25 million" and showed no significant impact on key metrics before it was abandoned5. Third, a redesign tested over a short window can mislead, through primacy effects and change aversion, so large changes need longer and bigger experiments or alternative designs5. Google's HEART paper makes the complementary point about metrics in general: they "are primarily useful for evaluation of launched products, and are not a substitute for early or formative user research"12. Experiment mechanics are page 4's.
The authors also state the idea behind the arithmetic in Section 3. They invoke Hubbard's expected value of information, "which captures how additional information can help you in decision making", and disagree with an earlier claim by Sinofsky and Iansiti that uncertainty cannot be reduced ("you don't know what you don't know"); with an experiment on a minimum viable product, they write, you "significantly reduce uncertainty"5. The break-even spend \(d^{*}\) is that principle in miniature: what a test is worth is bounded by how much it lets you avoid wasting on a build.
6.5 Shipping a small version
PostHog's engineering handbook treats the build as the test and manages the downside rather than avoiding it. "Just shipping is often the best way to validate an idea", it states, and it gives "the best evidence (do people actually use it, and what do they think), but potentially at the highest cost as you have to build it". The mitigations are to descope "as much as possible to reduce the cost you incur upfront", and to limit brand damage by "internal use first, a feature flag rollout, messaging a couple of friendly users, not marketing it or limiting the marketing, or shooting for Minimum Lovable Product instead of Minimum Viable Product"4. It adds cheaper signals to run in parallel, such as filing a public roadmap item to "gauge demand" and asking internal teams that talk to customers daily4.
The handbook is explicit about when the approach is safe. It describes three phases and notes that before product-market fit PostHog builds only "things that already have successful competitors with real revenue", so "'just build it' works disproportionately well because other people have already figured out that new product X has product market fit"; the open question is then whether PostHog's own users want it4. The value risk was retired by someone else's market; the team is testing fit and usability. That is a precondition worth stating for any team that adopts "just ship it": it works best when another party has already paid for the value test. Where nobody has, the same handbook says to pay more attention to how a feature is used, "as you are more likely to build something that isn't useful"4.
6.6 Shape then bet, and the risk it declines to address
Shape Up is often cited as a discovery method. Its own introduction says otherwise: "At every step of the process we target a specific risk: the risk of not shipping on time. This book isn't about the risk of building the wrong thing. Other books can help you with that (we recommend Competing Against Luck). Improving your discovery process should come after regaining your ability to ship"3. In the vocabulary of this page, Shape Up addresses feasibility and a slice of viability (does the time budget make sense) and leaves value to the people who decide what to shape. What it does say about value is a filter, not a test: pitches that are not bet on are let go, and "really important ideas will come back to you"; and of something heard once, "If you hear it once and never again, maybe it wasn't really a problem"3. That is recurrence as evidence, a passive and cheap signal that cannot hear from people who do not report, and the book supplies no data on how often it fails. The shaping output is "rough, solved, bounded": rough enough to leave room for designers and programmers, solved enough that "all the main elements" connect, and bounded by an explicit appetite3. Prioritization and the betting table are page 3's.
7. Choosing: how much discovery, and which test first
Torres is the one source here that states a stopping rule for discovery, and she states it as a bet: "Whenever we put a solution in our delivery backlog, we are making a bet. Discovery helps us reduce the risk of that bet." How much to do depends on three questions: how much risk the solution carries, how much risk the organization can bear, and how much time there is to decide. "If there's more risk than our organization can bear or if we have more time, we might choose to do more discovery. If we are running out of time or don't see a lot of risk in a solution, we can do less discovery." She adds that people new to discovery "aren't very good at assessing risk", so they should "do a little more discovery than you think you need"9. That is a qualitative version of Section 3.2: risk carried is roughly \(1-p\), tolerance is the cost of being wrong, time is what waiting costs. The mapping is the writer's. The page prescribes without data, as Torres says herself: "There are no black and white answers in discovery"9.
7.1 A procedure assembled from the sources
None of the sources states one procedure for picking the first test. The following is the writer's synthesis of Cagan's sequencing advice8, the test-party rule2, Torres's three questions9 and the arithmetic of Section 3.
- Write the four risks down for the candidate and, for each, separate what is known (observed behavior, a measurement, a term in writing from a counterparty) from what is assumed. The engineer's contribution is usually the feasibility column, and a skeptical read of the other three.
- Pick the risk whose being wrong costs most and whose evidence is thinnest. Cagan's default ordering is value and business risk first8, because, in the FAQ's account, the most common reason a good-sounding solution fails is that customers are unimpressed and unwilling to switch2. Override the default where feasibility is the unknown that could kill the idea cheaply: finding out quickly that it cannot be built is the cheapest outcome available.
- List the techniques in Table 5 that can reduce that risk and discard any whose blind spot is exactly what worries you. A UI prototype will not settle a latency question; a positive interview will not settle whether someone pays.
- Choose the one whose cost is under the break-even allowance \((1-p)(s-f)\) for your own estimates of \(p\), \(s\) and \(f\). When no estimate is available, use Section 3's ranges and say which assumption you are leaning on.
- After the result, re-rank. A passed test changes which risk is least tested; it does not close the candidate.
7.2 A worked classification (hypothetical)
The three features below are invented for illustration. They are not drawn from any source, carry no figures, and no source in this module covers inference runtimes. The point is to show the procedure, including what each cheapest test cannot tell you.
| Proposed feature | Best understood | Least tested | Cheapest test that reduces it | What that test cannot tell you |
|---|---|---|---|---|
| A quantized KV-cache option | Feasibility: the engineer can reason about the implementation and measure it on the target | Value: whether any integrator's deployment is actually limited by KV memory at the context lengths they use | Behavioral interview questions ("the last time memory limited a deployment") plus a read of existing support requests4 | Whether integrators would accept the accuracy trade-off, which is a second value question |
| A backend for a new accelerator | Value is plausible; the demand side is conventional | Feasibility first (operator coverage, driver behavior, speed on the silicon), then viability (the vendor's SDK and redistribution terms) | A time-boxed spike on the target silicon, asked as "possible in N weeks", and a read of the terms by whoever handles legal32 | Demand: a working backend says nothing about who will use it |
| A one-call API for loading and running a model | Feasibility: the engineer knows what the code behind it is | Usability: whether an integrator gets from install to first output unaided | A rough API sketch given to a developer who tries it while the engineer watches10 | Whether that developer would choose this runtime over alternatives |
7.3 When the build is the cheapest test
The allowance \(d^{*}\) scales with the cost of a build. If the narrowest version of the idea takes a few days, can be measured, and can be withdrawn, then the cheapest informative test may be the change itself. That is the Bing case in Section 2.3, where the enabling conditions are the ones the Microsoft authors list: a small experiment overhead and an easy change5. PostHog's "just shipping" (Section 6.5) adds a third, that someone else has already validated the demand. The first two depend on the product as well as on the team. A change shipped inside a client-side binary is harder to reverse than a server-side one, a point the textbook makes in general terms5; an on-device runtime sits at the hard end of that spectrum. It follows that for such products the build-it-and-see route costs more per test than for a web service, and spikes, prototypes on the target silicon and behavioral interviews carry more of the load. That is an inference from the sources' definitions, flagged as such; none of them measures it.
8. AI and the cost of a prototype
Two practitioner statements, about fifteen months apart, argue that generative AI changes the economics Section 3 describes. Both are assertions from experience and both date quickly; treat them as the current positions of two prominent voices, not as findings.
SVPG's 2026 FAQ distinguishes the roles. In delivery, "generative AI plays very much an automation / code generation role"; in discovery, "AI plays more of a prototyping and decision support role". It also holds that "as the cost of product delivery continues to drop, the bottleneck and our competitive advantage moves to product discovery", which is the FAQ's summary of the article that prompted it2. The same document adds that the product manager "needs to develop what we call strong product sense", with generative AI "a big help in learning this product sense", and it describes the product manager as someone who "is building and testing prototypes in order to learn", not as someone who writes about them2. It also warns that faster output does not automatically mean faster outcomes: the "ready-fire-aim" approach of accelerating output in the hope that outcomes follow collides with the principle of testing ideas responsibly2. The FAQ names no risk specific to AI-assisted prototyping, and none is stated elsewhere in the sources gathered for this page.
Andrew Ng's January 2025 letter reaches a similar premise by a different route: "Writing software, especially prototypes, is becoming cheaper", which he expects to raise demand for "people who can decide what to build". Treating building and deciding as economic complements, he argues that "given a clear specification for what to build, AI is making the building itself much faster and cheaper", and so "this will significantly increase demand for people who can come up with clear specs for valuable things to build". He also expects that "strong product managers" will increasingly "build prototypes for themselves"13. For AI-based products he adds a point that bears on feasibility: "AI's performance is hard to predict in advance", so product managers "need to be comfortable with this and have tactics to manage it"13.
The two agree that building is getting cheaper and differ on what becomes scarce: SVPG says discovery and product sense, Ng says specification and the judgment about what to build. They also differ on the artifact (a prototype versus a clear spec), which is page 5's question. Section 3 suggests what to watch. The break-even allowance is proportional to the cost of a build, \(d^{*} = (1-p)\,J\,B\). If AI lowers \(B\) and lowers the cost of a prototype by the same factor, the ratio SVPG's rule depends on does not move, but the absolute allowance shrinks, and the cheapest test of more ideas becomes building a narrow version (the Section 7.3 route). What does not get cheaper with code generation are the terms that involve people: a customer's hour in an interview, a stakeholder's attention, and the trust that SVPG says live experimentation spends2. Where a prototype must run on specific silicon to answer the question, generating it faster does not change what the silicon allows. All of this is the writer's inference from the two statements and the model; no source measures it.
Where experts disagree
Nobody in the sources argues for building without reducing risk first. The disagreement is over which instrument carries the weight, what counts as evidence before launch, and whether live customers may serve as the test environment. Four positions follow; each is held by an organization that publishes a method, and only one of them (the experimentation group) reports measured base rates.
Talk to customers every week and map what you hear
Torres, Product Talk9; PostHog's product-manager policy of one interview per week per PM11
Opportunities must come from customers' stories of past behavior, not from the team's guesses; interview continuously, revisit the opportunity space every three to four interviews, and compare more than one solution against each target opportunity9.
- Aims at value risk, which Cagan calls "usually" the toughest8.
- Cheap per session, and revisable as the tree grows9.
- The behavioral version (what did you do about it, did you pay) filters out polite interest4.
- A prescription with no outcome data; the author sells courses and licenses AI services to a tooling vendor9.
- PostHog's own engineering handbook says founders got positive feedback "almost every time" and still pivoted five times4.
- Says nothing about feasibility or viability, and assumes customers who will respond; "response rates are typically low"4.
Cheap artifacts before code: prototypes, or a written PR/FAQ
Moore and Cagan, SVPG12; Bryar and Carr, Working Backwards7
Discovery is the activity whose incentive is to learn quickly whether an idea will work; each prototype should cost at least an order of magnitude less than a build and be tested against the risk it targets with the party that can judge it12. The Amazon variant writes the customer announcement and FAQ first and treats the process as "a funnel, not a tunnel"7.
- Covers all four risks, each with the right audience2.
- Attacks the cost of a test directly, the lever Section 3 shows matters most.
- The internal FAQ forces technical, financial, legal and operational problems onto the page7.
- The cost bands and the 3-5 iterations are the authors' estimates, stated as an "if"1; both camps sell training or courses27.
- A prototype does not carry scale or other non-functional requirements2; a document produces no customer reaction.
- Done fully, the Amazon process took "months", and AWS "more than a year"7.
Let behavior decide: randomized experiments and shipping small
Kohavi, Tang and Xu, Trustworthy Online Controlled Experiments56; PostHog's engineering handbook4
Organizations are poor at assessing the value of ideas, so run controlled experiments where they are possible; they are "the most reliable and sensitive mechanism"5. PostHog: "Just shipping is often the best way to validate an idea", with a descoped first version and a limited rollout4.
- The only position with measured base rates, from the authors' own platforms56.
- Evidence is what users did, and experiments surface harms nobody predicted5.
- Cheapest when a platform exists and the change is a few days of work (the Bing case)5.
- Needs enough units, agreed metrics and easy changes; fails for M&A and is hard for hardware and client-side software5.
- Every test costs a build and exposes customers; SVPG calls this treating them as "guinea pigs"12.
- PostHog's safe case is one where others already proved demand4; metrics are "not a substitute for early or formative user research"12.
Shape then bet; do not bolt on a research cadence
Singer, Shape Up (37signals)3
The risk to manage is not shipping on time. Shape each idea into a rough, solved, bounded form; walk it past technical experts; bet a fixed time budget; cancel by default if it overruns; let important ideas return on their own3.
- Moves feasibility risk before the commitment and caps the loss3.
- Honest about its scope: "This book isn't about the risk of building the wrong thing"3.
- Very little process overhead; no backlog to groom3.
- Practitioner assertion with anecdote and one case study; no measured outcomes.
- "Ideas come back" cannot hear from users who never complain.
- Names no technique for value or viability, and its own abandoned redesign shows an unvalidated approach can still reach a bet3.
These are not four answers to one question. Interviews reduce value uncertainty cheaply; the SVPG rule and the PR/FAQ lower the cost of a test; experiments are a measurement instrument with a stated list of preconditions; Shape Up is a commitment discipline for feasibility. The disagreements that are real are three. Whether positive pre-launch feedback is trustworthy: PostHog's founders say it often is not, and the Microsoft figures show the analogous weakness in internal review (about two thirds of reviewed, implemented ideas failing to move their metric)46. Whether customers may be the test environment: SVPG says only on an opt-in basis2, the experimentation group's method is to randomize live users between variants and use guardrail metrics to catch harm5. And whether uncertainty can be reduced at all: the textbook rebuts the claim that it cannot5, and nothing in the gathered sources tests the rebuttal outside its own domain.
Under which conditions each wins follows from Table 5 and Section 3. With many users, an agreed metric and cheap, reversible changes, experiments and small shipments dominate, because the build is the cheapest test. With few customers, a client-side or hardware-bound deliverable, or an expensive-to-reverse release, prototypes on the real target, spikes, behavioral interviews and a written viability check carry more of the load, and experiments are a poor fit. That second profile is the usual one for an on-device runtime sold to integrators, but this is the writer's mapping and no source tests it. What would settle the question is outcome per unit of discovery cost across techniques, controlling for product type, and the unmeasured parameters in Section 3: how well a cheap test separates good ideas from bad (\(s\) and \(f\)). A team could estimate them from its own record by logging each cheap test's verdict against what later happened. Until someone publishes such data, all four positions rest on the experience of the organizations that publish them, with the single exception of the base rates.
What is changing
SVPG's FAQ is dated 2026-04-27 and its claims about AI's role in discovery and delivery sit at the front of a field that is moving; Ng's letter is from January 2025. Expect both to be revised213. The PostHog handbook is a living document: its Q2 2026 product-management themes include "Tracking human vs agent actions, and defining success for each" and giving each product a "reason for being" in an "agent-first world", which changes who counts as the user a team is testing with11. Four questions are open on the evidence gathered here. No source gives a hit rate for developer-infrastructure features. No source measures how well cheap tests (interviews, prototypes, written reviews) predict outcomes. Whether engineering cost or customer trust is the binding cost of learning once code is cheap has not been measured. And the 10-20% Bing and Google figure traces to a 2012 book the textbook cites and this page did not consult5.
References
- blogMoore, J. and Cagan, M. Changing How You Solve Problems. Silicon Valley Product Group (SVPG), 2022-09-20; second of a three-part sequence on defining transformation. https://www.svpg.com/changing-how-you-solve-problems/. Accessed 2026-10-26. Used for: feature team versus empowered product team (Table 1); the 10-20% roadmap-ROI claim attributed to unnamed analysts; discovery's incentive, the order-of-magnitude cost rule, "two orders" in many cases, and the 3-5 iterations estimate.
- blogCagan, M. Build To Learn FAQ. SVPG, 2026-04-27. https://www.svpg.com/build-to-learn-faq/. Accessed 2026-10-26. Used for: build to learn versus build to earn; reasons a good-sounding solution fails (Table 4); who each risk is tested with (Figure 4); "Test Ideas Responsibly"; prototype as spec and non-functional requirements; AI's role in discovery versus delivery.
- officialSinger, R. Shape Up: Stop Running in Circles and Ship Work that Matters. 37signals, web edition. https://basecamp.com/shapeup. Accessed 2026-10-26. Used for: Ch. 1 "Making teams responsible" and "Targeting risk"; Ch. 2 "Who shapes" and the three properties of shaped work; Ch. 5 "Different categories of risk", "Present to technical experts" and the abandoned-redesign case; Ch. 7 "No backlogs" and "Important ideas come back"; glossary (R&D mode, circuit breaker).
- officialPostHog. How to do product, as an engineer (PostHog Handbook, engineering). https://posthog.com/handbook/engineering/product-engineering.md. Accessed 2026-10-26. Used for: good and bad product engineers; "Just shipping is often the best way to validate an idea" and its mitigations; the three PMF phases; how to talk to users; interview response rates. Living document; added by the page writer, not in the source map.
- textbookKohavi, R., Tang, D. and Xu, Y. Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge University Press, 2020. Front matter, table of contents and Chapter 1 from the authors' companion-site PDF. https://experimentguide.com/wp-content/uploads/TrustworthyOnlineControlledExperiments_PracticalGuideToABTesting_Chapter1.pdf. Accessed 2026-10-26. Used for: Bing ad-headline case (Ch. 1, pp. 3-5); necessary ingredients (pp. 10-11); Tenets 1 and 3 and the base rates (pp. 12-14); strategy, portfolio, the Bing social-integration bet and EVI (pp. 21-24); Part III contents (pp. xi-xii).
- paperKohavi, R., Crook, T. and Longbotham, R. Online Experimentation at Microsoft. Third Workshop on Data Mining Case Studies and Practice Prize, 2009. https://exp-platform.com/Documents/ExP_DMCaseStudies.pdf. Accessed 2026-10-26. Used for: the originating statement that about one third of well-designed experiments improved the key metric (Section 5, p. 5); the population "ideas that pass all the internal evaluations"; "a failure of an experiment is not a mistake"; the McConnell 10-100x claim as reported. Added by the page writer, not in the source map.
- textbookBryar, C. and Carr, B. The Amazon Working Backwards PR/FAQ Process. workingbackwards.com, companion site to the book Working Backwards; the book text was not consulted. https://www.workingbackwards.com/concepts/working-backwards-pr-faq-process/. Accessed 2026-10-26. Used for: the PR/FAQ as a vetting tool, "funnel, not a tunnel", draft cost, internal FAQ scope, reviewers' questions, the AWS timeline. The page also sells a paid course.
- blogCagan, M. The Four Big Risks. SVPG, 2017-12-04. https://www.svpg.com/four-big-risks/. Accessed 2026-10-26. Used for: the four risk definitions (Table 4), the move from three to four, role ownership (Figure 4), "toughest risk is usually value risk", "tackle the big risks early".
- blogTorres, T. Opportunity Solution Trees: Visualize Your Discovery to Stay Aligned and Drive Outcomes. Product Talk, page dated 2023-12-06. https://www.producttalk.org/opportunity-solution-tree/. Accessed 2026-10-26. Used for: weekly interviews and the three-to-four-interview revision rule; the product trio; inputs from sales and support lacking context; the three questions on how much discovery to do. Author discloses a licensing tie to Vistaly and sells courses.
- officialPostHog. User feedback (PostHog Handbook, product). https://posthog.com/handbook/product/user-feedback.md. Accessed 2026-10-26. Used for: usability tests for big features "run by the engineer building that feature"; PM-led general feedback. Living document; added by the page writer, not in the source map.
- officialPostHog. Product management at PostHog (PostHog Handbook, product). https://posthog.com/handbook/product/product-team.md. Accessed 2026-10-26. Used for: engineers own sprint planning and spec'ing; PMs' "rule of thumb is 1 interview per week per PM"; Q2 2026 themes. Living document.
- paperRodden, K., Hutchinson, H. and Fu, X. Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications. CHI 2010 (note, 4 pp.). https://research.google.com/pubs/archive/36299.pdf. Accessed 2026-10-26. Used for: metrics "not a substitute for early or formative user research" (p. 1, introduction).
- blogNg, A. The Batch, Issue 284 (letter on AI product management). DeepLearning.AI, 2025-01-15. https://www.deeplearning.ai/the-batch/issue-284/. Accessed 2026-10-26. Used for: prototypes getting cheaper; demand for people who decide what to build; PMs building prototypes; AI performance hard to predict in advance.