The Looking Glass: Chains of Assumptions (Part 2)
Imagining what dream collaboration looks like
Dear readers, instead of my usual short snippets on a theme, this is Part 2 of the essay Chains of Assumptions, a topic near and dear to my heart.
Two extra sections are included behind the subscriber paywall.
Not All Assumptions are Salient
One feature of our smart human mind is that it’s really good at finding shortcuts.
Discontent to spend precious energy on what’s already assumed to be well-known, our mind naturally hones in on what appears salient.
This is why we don’t find ourselves requestioning the laws of physics every day.
We are happy to passively observe that gravity still works, that the sun still rises, that scientists are not raising any alarms upending theories we learned in grade school.
Similarly, if at some point in your youth you have gone through the entire rigamarole of researching and debating Macs versus PCs? and convinced yourself thoroughly of the superiority of Apple products, you no longer have to think much when you start a new job and someone asks you what kind of laptop you want. It’s almost an automatic response. (If this analogy does not resonate, feel free to plug in your preferred programming language, monitor setup, or beverage choice.)
This is one of the benefits of life experience, what we sometimes call wisdom. You don’t question every assumption because many of them you already have high conviction on. These chain links have grown thick after tumbling through the vicissitudes of time.
Another reason you may not question an assumption deeply is because you don’t believe it matters much to the goal at hand (which, by the way, is also an assumption!)
By the wisdom of the 80-20 rule, while thousands of factors might have some impact to your end goal, perhaps only a handful really matter. If Chain 1 is linked simultaneously to Chains 2, 3 and 4, and Chains 3 and 4 break, Chain 1 can still be held if Chain 2 is strong.
Thus, at any given moment, the assumptions worth questioning are the ones that are salient— both a) likely to be important and b) where conviction is weak.
Our job as builders is to spot and test these salient assumptions. We want to spend the most time identifying and strengthening the weakest links with the greatest potential to send our chandelier crashing down.
But how do we find these mysterious salient assumptions?
Not All Salient Assumptions are Examined
One bug of our smart human mind is that it’s really good at finding shortcuts.
It’s so good at this, in fact, that often we don’t even realize we are making assumptions.
What often feels to us as “intuitively right” or “intuitively wrong” — that snap feeling in our gut we get — is based on a chain of assumptions.
But we often feel the conclusion more easily than we can rationally explain the chain.
This is where mishaps lurk. Because if you can’t expose and explain your chain, how do you know whether it’s built on strong links?
This problem is even more pronounced for those of us who pride ourselves on being problem-solvers.
If you are an engineer, you were probably taught that your job is to solve problems. If your title has the word “product” in it, you probably think your job is to… well, invent products.
What happens if you self-identify as a “problem solver” or “product person?”
Naturally, you think your job is to crank out solutions! You pride yourself on the quality and speed in which you can do so. And the more “tangible” your solution is, the better — a product spec, a mock, a prototype or best of all, a feature live on production!
It is fine and well to pride yourself on being a skilled and efficient problem solver. (Hey, I do!) Speed matters, especially if it’s cheap to try out the solution and get feedback on its efficacy.
The challenge comes when the problem at hand is a) difficult to solve and 2) important to solve.
A difficult-to-solve problem means whatever solution you throw at it is likely to be incorrect, which means some assumption(s) in your chain are probably wrong
An important-to-solve problem means it’s worth investing the time, energy and people to get to an effective solution.
If the above criteria holds, I have news for you — the common model of a single person proposing a tangible solution and other people reviewing said solution is unlikely to be the most direct path to a great solution.
The Problem with Presenting Solutions
Imagine you are solving for a very difficult problem (new app idea, new large-scale systems design, new strategy) and you present a solution for review to a group of well-meaning collaborators.
Upon showing them your proposal, you see them furrowing their brows.
What’s going on in their heads?
It’s a good guess that your proposed solution did not immediately click. It doesn’t mean it won’t; but it’s not an instant “YES, this is a great solution!”
What follows furrowed brows is either 1) questions or 2) critiques. Let’s examine these one by one:
Questions — collaborators ask questions for one of two reasons: they aren’t exactly sure how the proposal works, or they aren’t convinced why this proposal is great.
The how questions are generally easy to answer (though if you get a lot of them, it’s worth reflecting on ways to explain your proposal more clearly to begin with.) If answering these questions gets the group to a “YES, this is a great solution!“, then we now have a high degree of alignment (hooray!) which, assuming the collaborators are a bunch of smart people, adds greater confidence that the solution is effective. But! It could still be the case that the solution is wrong because a) folks aren’t thinking deeply or b) everyone holds the same biases.
The why questions are asked because some folks want a convincing explanation for why this proposal is the best proposal. After all, it’s tough to judge the validity of a theorem unless you see its proof. Why questions demand to examine the chain of assumptions a solution is built upon.
Critiques — collaborators critique because they see problems in the solution. Something about the solution seems to them like it might not work, or doesn’t seem as good as it could be. Some implications of critiques:
The harder a problem is to solve, the more likely any proposed solution will garner many critiques. (If it didn’t and the solution were obvious, it probably wouldn’t be considered a hard problem!) Thus, anyone working on a hard problem should expect many critiques.
It is far, far easier to critique than create. Perfection is an illusion. One can still poke numerous holes at the best solutions in our world today (Apple products are too expensive! ChatGPT gets some answers wrong!) So an effective solution doesn’t have to address every possible critique.
The person who put forth the proposal has likely already wrestled with trying to address as many of the most important critiques as she could. Thus, when they hear additional critiques, it may feel deflating or even frustrating — I already thought of that critique, and I’m telling you, this is the best we can do given the constraints!
The goal of collective intelligence: what if the critiquer has unique knowledge about some portion of the proposal that can be improved? They might not have the context to come up with a complete solution that is better. They might not even be able to pinpoint exactly which part of the end solution they would improve. But they have unique knowledge that is additive to discovering a better solution.
Given the above, a far more meaningful question than Do you find my solution acceptable? is How can we utilize collective knowledge to its fullest to come up with the best solution?
For the 2 needs above — Answering Why Questions and Utilizing Collective Knowledge—there is a better method than Study my proposed solution and provide me some feedback.
Particularly for problems that are difficult and important, it’s usually the case that the collective intelligence and skills of a group of people will yield better results than a single brain.
How can we harness this most effectively?
Build a Web, Catch Some Questions
If a group’s goal is to find the most effective solution to a problem that is simultaneously difficult-to-solve and important-to-solve, then a way of working that maximizes each individual’s unique skills and knowledge would be to collectively create the web of assumptions.
A web is different from a chain in that there may be multiple reasonable assumptions for a hard question. Depending on which assumption you believe, you’d traverse down a different branch of deeper assumptions.
For example, take a simple example of the hard problem of Hire a top-notch engineering team. Imagine you asked a smart group of people to brainstorm solutions to this problem. You might hear ideas like the below:
Invest in on-campus recruiting at the top engineering universities
Reach out to employees from companies with great engineering reputations
Hire a recruiting agency with a great reputation
Implement a rigorous take-home coding assignment
Ask open-ended systems design questions in the interview
Start an internship or contractor program and convert the best candidates
Network with a group of CTOs and ask them for leads or references to greater engineering talent
Start a high-quality engineering blog
...etc
All of these are sensible ideas! But we can’t do ALL of them. Which ideas are most likely to get us that top-notch engineering team we’re dreaming of?
One way to make progress is to take each idea and extract its chain of assumptions by asking the question: Under which major assumptions would this idea lead to the outcome we want?
For example, if Invest in on-campus recruiting at the top engineering universities was indeed the best idea of the bunch, then that tells us:
Engineers with industry experience isn’t important for our team
Top engineering universities produce the best engineers
Going on-campus to recruit is the most effective way to reach those engineers
We have the capability to execute an effective on-campus campaign
We can get a large portion of the hires we need from on-campus recruiting
…and so forth
Some of these assumptions we may consider more salient than others. For example, perhaps no one in our group disagrees that Top engineering universities produce the best engineers or Going on-campus to recruit is the most effective way to reach those engineers.
But an assumption like Engineers with industry experience isn’t important for our team is a salient one. Because as we can see, Solution Number 2 (Target reaching out to employees from companies with great engineering reputations) would assume the opposite of that!
So we can start to build our web like the below:
What kind of top-notch engineers do we need?
New grads
→ Invest in on-campus recruiting at top engineering universities
Experienced engineers
→ Reach out to employees from companies with great engineering reputations
As we examine some of the other ideas from our brainstorm, we continue developing the web:
What kind of top-notch engineers do we need?
High potential new grads
→ Invest in on-campus recruiting at top engineering universities
Engineers with 3-5 years of experience who can independently execute
→ Reach out to employees from companies with great engineering reputations
Highly experienced engineers who can serve as leads and managers
→ Reach out to employees from companies with great engineering reputations
→ Hire a recruiting agency with a great reputation
And keep developing…
What’s our biggest blocker to hiring top-notch engineers?
We aren’t getting enough great candidates to talk to us
What kind of top-notch engineers do we need?
High potential new grads
→ Invest in on-campus recruiting at top engineering universities
…
Engineers with 3-5 years of experience who can independently execute
→ Reach out to employees from companies with great engineering reputations
Our response rate is low. Why?
Our company doesn’t sound interesting
→ Start a high-quality engineering blog
…
…
Highly experienced engineers who can serve as leads and managers
→ Reach out to employees from companies with great engineering reputations
→ Hire a recruiting agency
…
We aren’t accurately evaluating top-notch engineers
What’s the best way to assess talent?
→ See their actual work
Implement a rigorous take-home coding assignment
Start an internship or contractor program and convert the best candidates
…
→ Ask better questions
Ask open-ended systems design questions in the interview
…
…
…
…
How you choose to represent this “web of assumptions” doesn’t matter much. You can write a doc listing out the questions and potential answers like I did. You can use a prettier figjam with arrows.
What matters is that building this web can be a deeply collaborative process. It’s not My idea against your idea! or I’m trying to get everyone on board with this one idea.
Rather, the first important goal is identifying as a group: What salient questions are behind our salient assumptions?
The ones from the exercise above include:
What is our biggest blocker to hiring top-notch engineers?
What kind of top-notch engineers do we need?
How many engineers do we need to hire?
What’s the most effective way to assess talent?
The second important goal is then to identify as a group: Who among us is best suited to take on answering each of these salient questions?
For example, the best person to answer What is our biggest blocker to hiring top-notch engineers? is probably the person (or people) on the team with the most experience hiring top-notch engineers! Maybe this is the engineering recruiters + the engineering managers.
If no one on the team has experience hiring top-notch engineers, perhaps the best answer will come from outside the organization. Can we get connected with a handful of experienced engineering leaders and ask them to share their hard-won learnings on the matter?
Similarly, How many engineers do we need to hire? might be a question for the entire leadership team, or at least the CEO, the head of engineering, and the head of finance as they examine the question through the lens of company prioritization and budgeting.
What’s the most effective way to assess talent? might be a good question for the managers + senior members of the engineering team that actually conduct the interviews.
If we agree on the answers to these salient questions, then automatically many of the ideas from our initial list vanish in a poof.
The solution(s) we’re left with may now seem obvious (and if not obvious, at least intentional).
If we execute our solution and it doesn’t work, we should ideally be able to trace it back to an incorrect answer to one of our salient questions. (If it was not a salient question at the time, it is worth asking: Why did we not think it was salient?)
In this manner, we go from a giant list of ideas to the salient assumptions that form the foundation of those ideas.
Then, we go from salient assumptions to salient questions.
Then, as we do our best to answer those salient questions, that knowledge helps us refine and improve upon our ideas.
This is how we harness collective intelligence and move with greater intentionality toward our goals.
But what if we cannot agree on the answer for a salient question?
That One Weird Trick When you Disagree on the Assumptions
Keep reading with a 7-day free trial
Subscribe to The Looking Glass to keep reading this post and get 7 days of free access to the full post archives.