2 posts on Collaboration

Every decision we make involves weighing tradeoffs, whether that is done consciously or not. From evaluating whether an extra bedroom is worth $800 extra in rent, whether being able to sleep lying down during a long flight is worth the $500 upgrade cost, to whether you should take a pay cut for that dream job.

For complex high-stakes decisions involving many competing tradeoffs, trying to decide with your gut can be paralyzing. The complex tradeoffs that come up when designing products ^[1] fall in that category so frequently that analytical decision-making skills are considered one of the most important skills a product manager can have. I would argue it’s a bit broader: analytical decision-making is one of the most useful skills a human can have.

Structured decision-making is a passion of mine (perhaps as a coping mechanism for my proneness to analysis paralysis). In fact, one of the very first demos of Mavo (the novice programming language I designed at MIT) was a decision-making tool for weighed pros & cons. It was even one of the two apps our usability study participants were asked to build during the first Mavo study. I do not only use the techniques described here for work-related decisions, but for any decision that involves complex tradeoffs (often to the amusement of my friends and spouse).

Before going any further, it is important to note a big caveat. Decision-making itself also involves tradeoffs: adding structure makes decisions easier, but consumes more time. To balance this, I tend to favor an iterative approach, adding more precision and structure only if the previous step failed to provide clarity. Each step progressively builds on the previous one, minimizing superfluous effort.

For very simple, uncontroversial decisions, just discussing or thinking about pros and cons can be sufficient, and the cost-benefit of more structure is not worth it. Explicitly listing pros and cons is probably the most common method, and works well when consensus is within reach and the decision is of moderate complexity. However, since not all pros and cons are equivalent, this delegates the weighing to your gut. For more complex or controversial decisions, there is value in spending the time to also make the weighing more structured.

The tradeoff scorecard

What is a decision matrix?

A decision matrix, also known as a scorecard, is a table with options as rows and criteria as columns, with a column in the end that calculates a score for each option based on the criteria. These are useful both for selection, as well as prioritization, where the score is used for ranking options. In selection use cases, the columns can be specific to the problem at hand, or predefined based on certain principles or factors, or a mix of both. Prioritization tends to use predefined columns to ensure consistency. There is a number of frameworks out there about what these columns should be and how to calculate the score, with RICE (Reach × Impact × Confidence / Effort) likely being the most popular.

The Tradeoff Scorecard is not a prioritization framework, but a decision-making framework for choosing among several options.

Qualitative vs. quantitative tradeoffs

Typically, tradeoffs fall in two categories:

• Qualitative: Each option either includes the tradeoff or it doesn’t. Thing of them as tags that you can add or remove to each option.
• Quantitative: The tradeoff is associated with a value (e.g. price, effort, number of clicks, etc.)

Not all tradeoffs are all equal. Even for qualitative tradeoffs, some are more important than others, and the differences can be quite vast. Some strengths may be huge advantages, while others minor nice-to-haves. Similarly, some weaknesses can be dealbreakers, while others minor issues.

We can model this by assigning a weight to each tradeoff (typically a 1-5 or 1-10 integer). But if quantitative tradeoffs have a weight, doesn’t that make them quantitative? The difference is that the weight applies to the tradeoff itself, and is applied the same to each each option, whereas the value of quantitative tradeoffs quantifies the relationship between tradeoff and option and thus, is different for each. Note that quantitative tradeoffs also have a weight, since they also don’t all matter the same.

In diagrammatic form, it looks a bit like this:

These categories are not set in stone. It is quite common for qualitative tradeoffs to become quantitative down the line as we realize we need more granularity. For example, you may start with “Poor discoverability” as a qualitative tradeoff, then realize that there is enough variance across options that you instead need a quantitative “Discoverability” factor with a 1-5 rating. The opposite is more rare, but it’s not unheard of to realize that a certain factor does not have enough variance to be worth a quantitative tradeoff and instead should be modeled as 1-2 qualitative tradeoffs.

The overall score of each option is the sum of the scores of each individual tradeoff for that option. The score of each tradeoff is often simply its weight multiplied by its value, using 1/-1 as the value of qualitative tradeoffs (pro = 1, con = -1).

While qualitative tradeoffs are either pros or cons, quantitative tradeoffs may not be universally positive or negative. For example, consider price: a low price is a strength, but a high price is a weakness. Similarly, effort is a strength when low, but a weakness when high. Calculating a score for these types of tradeoffs can be a bit more involved:

• For ratings, we can subtract the midpoint and use that as the value. E.g. by subtracting 3 from a 1-5 rating we get value from -2 to 2. Adjust accordingly if you don’t want the switch to happen in the middle.
• For less constrained values, such as prices, we can use the value’s percentile instead of the raw number.

Explicit vs implicit tradeoffs

When listing pros and cons across many choices, have you noticed that there is a lot of repetition? First, several options share the same pros and cons, which is expected, since they are alternative solutions to the same problem. But also because pros and cons come in pairs. Each strength has a complementary weakness (which is the absence of that strength), and vice versa.

For example, if one UI option involves a jarring UI shift (a bad thing), the presence of this is a weakness, but its absence is a strength! In other words, each qualitative tradeoff is present on all options, either as a strength or as a weakness. The decision of whether to pick the strength or the weakness as the primary framing for each tradeoff is often based on storytelling and/or minimizing effort (which one is more common?). A good rule of thumb is to try to avoid negatives (e.g. instead of listing “no jarring UI shift” as a pro, list “jarring UI shift” as as con).

It may seem strange to view it this way, but imagine you were trying to compare and contrast five different ideas, three of which involved a jarring UI shift. You would probably list “no jarring UI shifts” as a pro for the other two, right?

This realization helps cut the amount of work needed in half: we simply assume that for any tradeoff not explicitly listed, its opposite is implicitly listed.

Putting it all together

Your choice of tool can make a big difference to how easy this process is. In theory, we could model all tradeoffs as a classic decision matrix, with a column for each tradeoff. Quantitative tradeoffs would correspond to numeric columns, while qualitative tradeoffs would correspond to boolean columns (e.g. checkboxes).

Indeed, if all we have is a grid-based tool (e.g. spreadsheets), we may be stuck doing exactly that. It does have the advantage that it makes it trivial to convert a qualitative tradeoff to a quantitative one, but it can be very unwieldy to work with.

If our tool of choice supports lists within cells, we can do better. These boolean columns can be combined into one column as a list of all relevant tradeoffs. Then, a separate table can be used to define weights for each tradeoff (and any other metadata, e.g. optional notes).

I currently use Coda for these tradeoff scorecards. While not perfect, it does support lists in cells, and has a few other features that make working with tradeoff scorecards easier:

• Thanks to its relation concept, the list of tradeoffs can be actually linked to their definitions. This means that hovering over each tradeoff displys a popup with its metadata, and that I can add tradeoffs by selecting them from a popup menu.
• Conditional formatting allows me to color-code tradeoffs based on their type (strength/weakness or pro/con) and weight (lighter color for smaller impact).
• Its formula language allows me to show and list the implicit tradeoffs for each option (though there is no way to have them be color-coded too).

There are also limitations however:

• While I can apply conditional formatting to color-code the opposite of each tradeoff, I cannot display implicit tradeoffs as nice color-coded chips, in the same way as explicit tradeoffs, since relations can only display the primary column.
• Weights for quantitative tradeoffs have to be baked in the formula (there are some ways to make them editable, but )

You may be familiar with this wonderful illustration and accompanying blog post by Henrik Kniberg:

It’s a very visual way to illustrate the age-old concept that that a good MVP is not the one developed in isolation over months or years, grounded on assumptions about user needs and goals, but one that delivers value to users as early as possible, so that future iterations can take advantage of the lessons learned from real users.

The MVP as a range

While not quite what Henrik intended, I love this metaphor so much, I have been using it to describe shipping goals when writing product specs. I find they can be immediately understood by anyone who has seen Henrik’s illustration, and fit nicely into a fixed time, variable scope development process, such as Shape Up.

1. 🛹 The Skateboard (aka the Pessimist’s MVP): What is the absolute minimum we can ship, if need be? Utilitarian, bare-bones, and somewhat embarrassing, but shippable (only barely). Anything that can be flintstoned gets flintstoned.
2. 🛴 The Scooter (aka the Realist’s MVP): The minimum product that delivers value. Usable, but no frills.
3. 🚲 The Bicycle (aka the Optimist’s MVP): Stretch goals — UX polish, “sprinkles of delight”, nonessential features with high I/E. Great if we get here, fine if we don’t.
4. 🏍️ The motorcycle: Next-phase improvements we want to reach soon after launch.
5. 🚗 The car: Medium-term vision; more complete and refined.
6. 🏎️ The race car (aka the North Star UI): The ideal experience — unconstrained by time, resources, or compatibility. Not meant to ship, but to guide everything else.

The meat is the first three stages, since they directly affect what is being worked on. The more we go down the list, the less fleshed out specs are, since we know they will change once we have input to customers.

The most controversial of these is the last one: the race car, i.e. the North Star UI. It is the very antithesis of the MVP. The MVP describes what we can ship ASAP, whereas the North Star describes the most idealized goal, one we may never be able to ship.

It is easy to dismiss that as a waste of time, a purely academic exercise. “We’re all about shipping. Why would we spend time on something that may not even be feasible?” I hear you cry.

Stay with me for a moment. Paradoxical as it may sound, I hope that by the end of this I will have convinced you that fleshing it out pays dividends in the long run and actually saves you time.

1. It exposes the hidden driver of all product design decisions

Whether you realize it or not, every shipping goal is derived from the North Star, like peeling layers off an onion. In some contexts the process of breaking down a bigger shipping goal into milestones that can ship independently is even called layering.

The process is so ingrained, so automatic, that many product designers don’t realize they are doing it. They go from race car to car to motorcycle so quickly they barely realize there was anything else there to begin with. Thinking about the North Star feels like a guilty pleasure — who has time for daydreaming? We must ship, yesterday!

But the race car is fundamental. Without it, there is no skateboard — you can’t reduce the unknown. Without a solid North Star, your MVP is a confused jumble of design decisions and compromises, so tangled it becomes impossible to tell them apart.

To stick with the transportation metaphor, a skateboard might be a good MVP if your ultimate vision is a race car, but it would be a terrible minimum viable cruise ship — you might want to try a wooden raft for that.

This North Star will likely change a lot down the line, informed by experience and user feedback. That’s okay; having an initial destination does not remove your ability to course correct.

2. It makes impossible product problems tractable

Nearly every domain has a version of divide and conquer: instead of trying to solve a complex problem all at once, break it down into smaller, more manageable components and solve them separately. Product design is no different. The concept of a North Star UI breaks down tough product design problems into three more manageable components:

1. North Star: What is the ideal solution?
2. Ephemeral constraints: What prevents us from getting there right now?
3. Compromises: How close can we reasonably get given these constraints?

Depending on the product problem, this could simplify things a little or a lot. This framework really shines when you’re dealing with really tough product problems, where two or three of these components are hard, so focusing on one at a time really helps cut down complexity.

3. It facilitates team alignment

When the North Star UI is not clearly articulated, it doesn’t mean it doesn’t exist. It just means that everyone is following a different version of it.

Since MVPs are products of the North Star, this will manifest as difficulty reaching consensus at every step of the way, because the root disconnect is never addressed head on.

Here is a story that will sound familiar to many readers: Alice has designed an elegant solution that addresses not just the problem at hand, but several prevalent longstanding user pain points at once — an eigensolution. She is aware it would be a little trickier to implement, but she thinks the tremendous improvement in user experience is worth the very modest increase in effort. She has even outlined a staged deployment strategy that allows it to ship incrementally, adding value and getting customer feedback earlier. Excited, she presents her idea to the product team, only to hear Bob dismiss it with “this is scope creep and way too much work, it’s not worth doing”. However, what Bob is actually thinking is “this is a bad idea; any amount of work towards it is a waste”. The design session is now derailed. At best, instead of figuring out whether Alice’s solution is good, the team is now spending all remaining time discussing how much work it would be and whether it could be reduced. This is a dead end because the amount of work was never the actual problem. Or worse, Alice backs off and agrees to Bob’s “simpler” idea that is overfit to the very specific use case being discussed, because she does not want to be seen as unfocused, or not a team player.

Arguing over effort feels safer and less confrontational than arguing over vision — but it’s often a proxy war. Fleshing out the North Star UI as an explicit goal strips away this noise and brings clarity.

This is not productive. First, if the idea is not good, the amount of work is irrelevant and spending time costing it is putting the cart before the horse. And if it is, often implementation proves to be easier than expected once it is properly investigated.

It is important to answer the questions above in order, and reach consensus on what the North Star is before moving on to the compromises. This way, we are aware of what is an actual design decision and what is a compromise driven by practical constraints. Articulating these separately, allows us to discuss them separately. It is very hard to evaluate tradeoffs collaboratively if you are not on the same page about what we are trading off and how much it’s worth. You cannot do a cost-benefit analysis without being aware of both the cost and the benefit.

Additionally, fleshing the North Star out separately ensures that everyone is on the same page about what is being discussed. All too often have I seen design sessions where one person is discussing the skateboard, another the bicycle, and a third what the car, no-one realizing they are talking cross purposes.

4. It can improve the MVP via user testing

Conventional wisdom is that we strip down the North Star to an MVP, ship that, then iterate based on user input. With that process, our actual vision never really gets evaluated. By the time we get to it, it has already changed tremendously. But did you know you can actually get input from real users without writing a single line of code?

Believe it or not, you don’t need to wait until a UI is prototyped to user test it. For some reason, this seems to be almost unheard of outside usability circles, but you can even user test a low-fi paper prototype or even a wireframe. The user tells you where they would click or tap on every step, and you mock the UI’s response by physically manipulating the prototype or showing them a wireframe of the next stage. This allows you to user test your North Star UI from the start, and adjust your MVP accordingly.

Obviously, this works better for some types of products than others. It is notably hard to mock rich interactions or UIs with too many possible responses. But when it does work, it can have a very high Impact/Effort ratio. It can be especially useful when there are vastly different perspectives within a team about what the North Star UI might be, or when the problem is so novel that every potential solution is low-confidence. Even the best product intuition can be wrong, and there is no point in evaluating compromises if it turns out that even the “perfect” solution is not actually all that great.

5. It paves the way for getting there (someday)

Just like the mythical North Star, a North Star UI serves as a guide to steer us in the right direction. Simply articulating what the North Star is can make it more feasible. It’s not magic, just human psychology.

First, once we have a North Star, we can use it to evaluate proposed solutions: How do they relate to a future where said North Star is implemented? Are they a milestone along that path, or do they actively prevent us from ever getting there?

Prioritizing solutions that get us closer to the North Star can be a powerful momentum building tool. Once we’re partway there, it naturally begs the question: can we get closer? How much closer? it is much easier to convince people to move further along a path they are already on than to start a different path. Even if we can’t get all the way there, maybe we can close enough that the remaining distance won’t matter. And often, the closer you get, the more achievable the finish line gets. In fact, sometimes simply reframing the North Star as a sequence of milestones rather than a binary goal can be all that is needed to make it feasible.

6. It makes the design process robust and adaptable

North Star UIs make the product design process more resilient and adaptable. I have often seen “unimplementable” solutions become implementable down the line, due to changes in internal or external factors, or simply because someone had a lightbulb moment.

Without a solid foundation to fall back on, this means our only options are to either go back to the drawing board, or miss out on the opportunity to adapt. With a solid North Star UI in place, when constraints get lifted we only need to re-evaluate our compromises, which is usually much easier than starting from scratch.

Case studies

In my 20 years of product design, I have seen ephemeral constraints melt away so many times I have learned to interpret “unimplementable” as “kinda hard; right now”. Some examples from my own experience below.

Context Chips

This is a case where I was originally operating under the constraint extremely minimal engineering resources and having to aggressively prioritize, a constraint that was later lifted due to a change in engineering momentum. It is described at length in Context Chips in Survey Design: “Okay, but how does it feel?”.

In a nutshell, the relevant bits are:

• The engineer hated the original low-effort, entirely client-side idea so much, he prototyped a much higher-effort solution which involved a backend and API changes in a day. Previously, this would have been entirely out of the question.
• Initial opposition to the later proposed UI was completely reversed once the engineer observed user testing sessions using a prototype.

Here is a little secret that applies to nearly all software engineers: neither feasibility nor effort are fixed for a given task.

Engineers are not automatons that will blindly implement whatever they are told with the same energy and enthusiasm. If you are content getting Engineering to reluctantly agree to implement your vision, you’re typically getting very poor ROI out of it.

Often all that is needed to make the infeasible, feasible is engineering momentum. Investing the extra time and energy to get engineers excited can really pay off. When good engineers are excited, they become miracle workers. The difference is not small, it is orders of magnitude. Things that were impossible or insurmountable become feasible, and things that would normally take weeks or even months are prototyped in days.

One way to get engineers excited is to convince them about the value and utility of what they are building. All too often, product decisions are made in a vacuum, based on gut feelings and assumptions about user needs. Backing them with data, such as usability testing sessions is an excellent way to demonstrate their basis. And having engineers observe user testing sessions firsthand can be much more powerful than secondhand reports.

CSS Conic Gradients

background: conic-gradient(in hsl,
	red, orange, yellow, lime,
	cyan, blue, magenta, red);

Often, what makes a high effort feature feasible (or even trivial) is a change in the environment. An example that comes to mind is CSS conic gradients. Conic gradients are the type of gradient that is created by (conceptually) rotating a ray around a center point.

I originally proposed adding conic gradients to CSS in 2011, and they first shipped in 2018 (in Chrome 69)! Someone observing this timeline without context may just conclude “pffft, standards just take forever to ship”. But there is always a reason, either technical, human, or both. In this case, the reason was technical. Browsers do not implement things like shadows and gradients from scratch, they use graphics libraries such as Skia, Cairo, or Core Graphics, which in turn are also abstractions over the OS-provided graphics APIs.

At the time these libraries did not support any primitive that could be used to render conic gradients (e.g. sweep gradients, mesh gradients, etc.). In the years that followed, one after another these libraries added support for some kind of gradient primitive that could be used to easily render conic gradients, which took the proposal from prohibitively high to low effort. I also created a polyfill which stimulated developer demand, increasing Impact. These two things together took the Impact/Effort ratio from “not worth it” to “let’s do this, stat” and in 2 years the feature was implemented across all major browsers.

Relaxed CSS Nesting Syntax

Sometimes high effort things just take a lot of hard work and there is no way around it. Other times, feasibility is one good idea away.

One of my favorite examples, and something I’m proud to have helped drive is the relaxed CSS Nesting syntax, now shipped in every browser. It is such an amazing case study on the importance of having a North Star UI, I even did an entire talk about it at Web Unleashed, with a lot more technical details that I have included here.

In a nutshell, CSS nesting was a (then new) CSS syntax that let CSS developers better organize their code by reducing repetition through nesting CSS rules inside other CSS rules.

table.browser-support {
	border-collapse: collapse;
}
table.browser-support th,
table.browser-support td {
	border: 1px solid silver;
}
@media (width < 600px) {
	table.browser-support,
	table.browser-support tr,
	table.browser-support th,
	table.browser-support td {
		display: block;
	}
}
table.browser-support th {
	border: 0;
}
table.browser-support td {
	background: yellowgreen;
}
table.browser-support td:empty {
	background: red;
}
table.browser-support td > a {
	color: inherit;
}

table.browser-support {
	border-collapse: collapse;
@media (width < 600px) {
&, tr, th, td {
display: block;
}
}
th, td {
border: 1px solid silver;
}
th {
border: 0;
}
td {
background: yellowgreen;
&:empty {
background: red;
}
> a {
color: inherit;
}
}
}

This is one of the few cases where the North Star UI was well known in advance, since the syntax was already well established in developer tooling (CSS preprocessors). Instead, the big challenge was navigating the practical constraints, since CSS implemented in browsers has different performance characteristics, so a syntax that is easily feasible for a preprocessor may be out of reach for a browser. In this case, the North Star syntax had been ruled out by browser engineers due to prohibitive parsing performance ^[1], so we had to design a different, more explicit syntax that could be parsed more efficiently.

At this point, it is important to note that CSS Nesting is a feature that once available, is used all over a stylesheet, not just a couple times here and there. Both conciseness and readability are paramount, especially when conciseness is the sole purpose of the feature in the first place!

Initial attempts for a syntax that satisfied these requirements introduced a lot of noise, in the form of an awkward, noisy @nest token that needed to be placed in the beginning of most nested rules. Worse yet, these attempts were actively incompatible with the North Star syntax, as well as other parts of CSS (namely, the @scope rule). This meant that even if the North Star UI became feasible later, CSS would need to forever support syntax that would then have no purpose, and would only exist as a wart from the past, just like HTML doctypes.

This proposal sat dormant for a while, since implementors were not exactly in a hurry to ship it. This all changed when State of CSS 2022 showed Nesting as the top missing CSS feature, making Google suddenly very keen to ship it.

A small subset of the CSS Working Group, led by Elika Etemad and yours truly organized a number of breakouts to explore alternatives, an effort that produced not one, not two, but four competing proposals. The one that the group voted to adopt ^[2] was the one I designed with the North Star in mind, by asking the question: If the North Star syntax is out of the question right now, how close can we get and still be compatible with it in case it becomes feasible later on?

Once we got consensus on this intermediate syntax, I started exploring whether we could get any closer to the NSUI, even attempting to propose an algorithm that would reduce the number of cases that required the slower parsing to essentially an edge case. A few other WG members joined me, with my co-TAG member Peter Linss being most vocal. This is a big advantage of NSUI-compatible designs: it is much easier to convince people to move a little further along on the path they are already on, than to move to a completely different path. With a bit of luck, you may even find yourself implementing an “infeasible” NSUI without even realizing it, one step at a time.

We initially faced a lot of resistance from browser engineers, until eventually Anders Ruud and his team experimented with variations of my proposed algorithm and actually closed in on a way to implement the NSUI syntax in Chrome. The rest, as they say, is history.

Conclusion

Hopefully by now you’re convinced about the tremendous value of spending time reaching consensus on an explicit North Star UI that has buy-in from the entire product team. However, it is important that everyone is on the same page and understands the steps and benefits of this framework. Trying to design the North Star with people that see that as a pointless thought experiment does not confer any of these benefits, and just adds friction.