How to make better technology decisions

From macro decisions like the languages and frameworks we choose for our teams to the thousands of micro decisions we make with every line of code, software development is a never ending series of choices.

While this sentiment is true in all areas of software, the sphere of web development might be the most exemplary.

Two years ago I was tasked with transforming our Angular 1 (also known as AngularJS, but I can never keep the Angular names straight) web stack to something more modern. At the time we could choose from React, Angular (maybe version 2 or 3 back then?), Vue, and a few others that I can't remember but those were the big 3.

We avoided a heavy framework decision process by quickly choosing React for a few reasons but that just left us with more decisions...

Do we start with Create React App or another meta framework like Next.js?

What do we use for styling? Raw CSS, CSS modules, Styled Components, Emotion, etc.?

What do we use for state management? Redux, MobX? Plain React?

Do we use an open source component library like Bootstrap or Material UI? (hint: we went with Semantic UI, whoops)

What about TypeScript? (this one was not controversial AT ALL /end sarcasm)

There were maybe a dozen or more decisions we had to make before we felt like we could even get started.

It has been over 2 years and in retrospect, we feel good about many of the decisions we made but there were still a few (looking at you Semantic UI) that we regret. With the sheer volume of decisions made, it can be expected that a few decisions don't go your way. There are many factors at play, many of which are completely out of the control of developers like you and I.

In many ways, technology choice is an exercise in hedge theory. You need to simultaneously balance what you need now with what may or may not happen in the future, where many decisions are outside your control.

While there will always be some risk that any technology choice can come back to bite us, it is important that we minimize risk as much as possible. Investing in an open source package that is abandoned 6 months after you start using it can be detrimental. You either pay the hidden cost of carrying the weight of stale technology or you pay to move away from it. In either case you, your team, and your stakeholders won't be happy to deal with any of that.

After trying many forms of technology evaluations, I landed on a single, sturdy process that I use for every technology decision I participate in and created a tool called Help Me Decide, which is free for you to use for any type of evaluation.

For the remainder of this article, I will walk you through the principles behind this evaluation process and how to use the Help Me Decide tool to start making better technology decisions today.

Why technology decisions are hard#

Going back even further in time, I experienced a "tale of two teams" so to speak where a small company I worked for had two teams adopting React in parallel at the same time. We were moving so fast we weren't really communicating with each other on what we were doing nor sharing any of our learnings. Keep in mind this was back in 2014-2015 when React was very fresh we were still hanging onto bits of Backbone. Actually, I was surprised to see React's docs still explain how to use Backbone models in React components, whyyyyyy would you do that today?

A month or two into both teams having some React experience, the difference in sentiment toward React was staggering. One team used terms like "nightmare" and "a huge mess" and another team used terms like "delightful" and "intuitive". Being on the nightmare/intuitive side of the equation, I was intrigued by the stark difference in experience. Why are some developers loving it and why are some hating it?

As I dug in, I realized the other team was breaking every rule in the book and there weren't even that many rules back then. Directly manipulating props values, frequent calls to forceUpdate and other really nasty things across the board. It was a clear case of people trying to do jQuery with React and it just didn't work.

This story points out the biggest problem with any technology choice: humans.

But humans are also responsible for making all of these decisions, so how do we acknowledge our flaws but still press on and make the best decisions possible?

How not to make decisions#

Let's explore decision making further, starting with the most common ways I see technology decisions made and why they are not enough.

Pros and cons#

Most often, when I see people making "weighty" decisions (you know the decisions that really matter) they list the pros and cons for various options like which framework to choose, which open source package to install, which component library to use, which error service provider to use, etc. While pros and cons can surface useful information, they do not provide a good structure for making quality decisions. Often one or two items in either the pro or con category heavily outweighs the others but a simple list of bullet points doesn't necessarily capture that.

Internet research#

If you searched the internet on your technology choice, you will surely find many helpful resources out there, but no single resource should be the end of your research.

Most importantly, the authors of those articles are not aware of you nor your situation. One example where I see this lack of awareness cause problems is when evaluating Flutter and React Native on their respective learning curves. The learning curve for any framework depends entirely on the experience of you and your team. It is impossible for any of these authors to know what the learning curve will be for you.

This is not to say that these resources should be ignored. In fact, I 100% advocate for you reading as many comparisons as you can on your topic.

Comparison charts#

Not too long ago, my team needed to choose a headless CMS solution. If you've ever had to choose a headless CMS, you are probably already laughing because you know there are countless options available.

As part of this evaluation, somebody on our team put together a spreadsheet that listed 82 different features across 16 different CMS providers with data sourced from this headless-cms-comparison repo.

Again, feature comparison charts might surface useful information but just like you don't choose your dinner meal based on the number of ingredients in the dish, you shouldn't choose your technology based on the number of features it supports.

1-1 Criteria comparisons#

Ok I admit I sort of love these [Framework] vs. [Framework] posts that you find on the internet. They are especially entertaining when they use the word "battle" or include fighting graphics like the arm wrestlers in this Dart vs JavaScript post I found when I was researching my Flutter vs. React Native post. The thought of these inanimate frameworks coming to life and battling each other is so captivating I just need to read them.

Often what you see in these comparison posts is a list of categories or criteria and then a declared winner for each one. This Flutter vs. React Native post (with battling chess pieces, yes! ⚔️) is exactly what I mean. Because React Native uses the more popular JavaScript language, it wins that category and so on until you add up all the points and declare the final verdict.

As much as I love these kinds of posts and recommend you read as many as you can, they are still missing something. What if you are already proficient with Dart? Would you be so quick to declare React Native as the winner?

A sane decision making process#

Decisions are a process. After making many technology decisions, both large and small, over the years I distilled this decision process down to 3 simple steps:

1. Identify your options
2. Identify your criteria, applying weight to each
3. Score each option for every criteria

Identify options#

This is probably the easiest step in the process. Especially for very common choices like choosing a web development framework, some quick internet searching should turn up several options. At this stage, you can likely also use your intuition to remove a few options. While Backbone.js is still technically an option, I recommend you dismiss it.

Identify and weight criteria#

Identifying the criteria for your evaluation is considerably more difficult. This is where you need to be honest with yourself and you need to force your stakeholders to be honest too. Everybody comes into every technology evaluation with bias. Even non-technical C-levels will have an opinion on the technology choices you make because some founder one time told them a nightmare story or their child in high school was raving about the hottest new tech at the dinner table. Bias is important to acknowledge and understand. You can never remove bias from your decision process, but understanding its influence will help you make better decisions overall.

While you want to list your criteria early in the decision process, it is not necessarily set in stone. It is common to add or remove criteria as you discover new information during your research.

When identifying criteria, think about the material impact of you, your team, and your stakeholders.

Cost is a criterion that should be considered in most of your evaluations because it has a direct material impact on your business, though it may not be applicable if you are choosing between two free options.

Developer Experience has an indirect material impact. Unlike with cost, a poor developer experience will not hit your bank account immediately, but a poor developer experience can lead to productivity loss, attrition, defects, etc.

Other criteria I typically consider:

• Community support
• Performance
• Resource or team impact (another type of cost)
• Adoption
• Learning curve

Synthesizing criteria from stakeholders is a challenge in and of itself. I hear a lot more "Company A down the street is using X technology so shouldn't we too?" than I do "Y is the most important metric for us right now". As you pull criteria from various stakeholders, be sure to drive them toward the root of their thinking by identifying the material impact they are getting at.

Going back to the "Company A down the street..." scenario, it is important to acknowledge that individual reputation is at play here. No company leader wants to be in a situation where they made a bad bet on a risky technology decision and were the only one to do so. If everyone on the proverbial "street" makes the same bet and loses, the blow to individual ego's is softened. Obviously you don't want to list C-Level Ego as a criterion but if you sense this is influencing your decisions you can incorporate it into a more palatable category like Adoption or Risk.

Once you have criteria, it's time to assign weights to each one.

For the sake of simplicity, I typically use a 1-5 weighting scale.

The most difficult part of this step is assigning weights on behalf of all of your stakeholders. You can try sending out a form to each stakeholder to have them select a weight for a list of criteria but in my experience stakeholders avoid explicit rankings like the plague. I am no psychologist, but my speculation is this aversion stems from the fear that declaring something as less important means it won't get done or receive as much attention as people want it to.

This leaves you with some guesswork trying to figure out the appropriate weight that represents the best interests of all of your stakeholders.

While this leaves open the opportunity for you to boost your influence by weighting things higher based on what you want, manipulation is a losing tactic. No matter how thorough or sophisticated your analysis, if you deviate too far away from the best interests of the group, you are not only botching the current analysis but you will lose trust for future decisions. Don't manipulate the results. Generally I try to think like an owner and elevate that thinking transparently in the analysis. Stay focused on what your key stakeholders need, even if it goes against your own personal interests.

Score options#

Where identifying and weighting criteria might be the most challenging step, scoring options is the most time consuming. There are many ways to come to a score for each option's criteria and often the quality of your decision grows proportionally with the amount of time spent.

This is the step where you need to dig in and research. If you are in a position where you need to make such a decision, you very likely don't have significant experience with any of the options and almost certainly do not have enough experience with all of them. If you did have that experience, you probably already made your decision and don't need to go through this process.

So how do you balance time? Surely you don't have enough time to build your entire solution twice and wait 6-12 months to see if it stood the test of time with your team and your users. Depending on your situation, the amount of time you have to research your options probably ranges from a few days to a few months.

Start with prototyping#

Even if I had just a few days to evaluate my options, I still try to build a prototype with each option. The amount of time I have dictates how large the prototype is. For example, when researching Flutter vs. React Native I started my prototyping by integrating with a GraphQL backend because the most common development activity in my project would be wiring up UI with a GraphQL backend. Obviously there were many other things to explore that I could prototype, but I could get the so-called "80% of the value with 20% of the effort" by focusing on the core development activities and my biggest unknowns.

I also need to emphasize that it's important to start with prototypes. If you start by reading docs and various blog posts, you are going to miss important details that you can only acquire by getting your hands dirty. Just like you won't learn how to rock climb by reading about rock climbing technique, you won't learn technology without using technology.

Use prototyping to feed your curiosity by building up a list of questions to answer and knowledge gaps to fill. Why does React Native need a FlatList component? Why does Flutter need a separate class for state? Just take raw notes as you go, you can revisit them later. The goal here is to focus on doing rather than reading (though you will need to read a few things to get started).

Now that your curiosity is fed, you are ready to move into research.

Research#

At the beginning of this guide, I took a little jab at the flaws of internet research but I'll reemphasize this point:

Read as many comparisons as you can on your topic

Depending on the decision you are trying to make, there can be endless articles out there on your options. Just search for "React vs Vue" and you will be quickly overwhelmed. I encourage you to read as many of these as you can and continue to take raw notes on the points that matter to you. You will find after reading just 3-4 articles, the information will begin to repeat itself so it shouldn't take too long for you to have enough information to begin to make conclusions in certain areas.

I also recommend you reach out to people. Your colleagues or others in your network may have experience with some of the technology you are evaluating. Don't be afraid to ask for a 30 minute interview to ask about their experience. If there is one thing that developers love, it's talking shop and sharing their opinions. I find these conversations far more valuable than what you can read on the internet. They allow you to better understand context, ask clarifying questions and dig into the most pressing topics for your situation.

Direct interviews also add credibility to your analysis. If you can point to a specific person's statement and share that you know them and their background, it carries more weight than a random person on the internet. Even the most influential developers in your field are probably not known by your non-technical stakeholders, so your direct relationships with other developers carry importance.

These conversations almost always end with the interviewee saying something like "Keep me posted! I want to know how it goes for you!" so you are also opening up opportunities for somebody to ask questions to later as you get deeper into implementation.

In all of these forms of research, you should be able to validate presuppositions you took note of in your prototyping. If you had a note like "Hmmmmm this seems overly complicated" and multiple sources bemoan the same complications, then you likely made a noteworthy observation.

Likewise, you will also discover areas where your first impression was wrong. I'll sometimes have a note like "Whyyyyyyy does this not all me to do X? -20 points for Gryffindor!" only to realize later that I could do X all along, I just didn't read the docs thoroughly enough.

Lastly, your research may lead you to revisit certain areas of your prototypes. If it seems like an important avenue to explore or you still don't quite get it, absolutely go back and do some additional tinkering if you have time.

Scoring#

This is it, you are on your way to decision glory! Scoring is the final and most important step in the process.

Now that you've done your prototyping and research, you are ready to score each option.

After going through this process many times over the years with different spreadsheet formats, I created Help Me Decide, which is free for you to use for any type of evaluation. With this tool you can follow the exact process I laid out in this article to list options, add weighted criteria and score your options. The final result will look something like this:

With the share link, you can share your analysis with colleagues or save your results to revisit them later.

How do you make technology decisions?#

I'd love to hear your feedback on this process, and if you have questions on this process or the Help Me Decide tool, find me on Twitter @seanconnollydev.