20.  When is good enough, good enough?

Drew Freeman: Hey, Joe, how's it going?

Joe Marcantano: I am well, Drew. How are you?

Drew Freeman: I am doing well. We've been recording a couple of episodes. My voice is starting to get a little bit tired, so we'll see how this one goes.

Joe Marcantano: Well, we'll do our best to get wrapped up in time for you to watch football on this Sunday.

Drew Freeman: I love my football. Watching Sundays, I watch the Red Zone Channel. I don't know if you've heard of it, but for those who haven't, it's where they take all of the games that are happening on Sundays and basically switch back and forth to whatever is most interesting. It's pure chaos, and I love it.

Joe Marcantano: See, I'm much more of a, of an NHL hockey guy, and my team is off today, so today will not be filled with watching sports for me.

Drew Freeman: All right, well, enough about our sports conversation. That's a. That's a different podcast. There are many out there already. So for ours, the topic and the question that I bring to you today is, in UX research, when is good enough, good enough? And how do I know that I've reached good enough?

Joe Marcantano: I. There's something I love about this question, and it's kind of in its vagueness, because before we started recording this, I asked you, like, when you're talking about good enough, good enough, like, what area do you mean? And we kind of had this quick conversation about how it kind of touches lots of things and they all make sense.

Drew Freeman: And to go behind the curtain a little bit, my answer was basically yes.

Joe Marcantano: Yeah. And I like that because it certainly fits in with everything. You know, when I read this question, my first thought was, like, rigor of the study. You know, when is certain enough in your outcomes, certain enough? But there are lots of other ways to take this question.

Drew Freeman: Okay, so let's go down the research rigor path first. I know that for a lot of researchers, they want to be as certain as they possibly can be. They want to provide as much evidence as possible. They want to prove it beyond a reasonable doubt. What do you say to those researchers?

Joe Marcantano: I say to those researchers that you are misdirecting your resources. So there are certain questions, existential to the business that you need to be on that, you know, level of certainty scale at an 11 that you, you must be right. The consequences are critical if you are wrong. There are other questions where a directional idea is good enough and the consequence of being wrong is low and the cost of switching after the fact is low, that we can give a directional answer and save a bunch of time.

Drew Freeman: To give some vague answers or some vague situations to that answer. The kind of directional cost of switching is low kind of thing might be what where should this button be on the screen? Should it be in place A or place B? It might be should this text to be bolded and made a different color so that it stands out, or should it be, you know, the same color as the rest of the text and blend in a little bit more? Those are things where the consequences are not huge, typically. And even if they are big, and even if you get it wrong, they're relatively simple to switch.

Joe Marcantano: Yeah, you know, this is kind of specific to digital products in that. And it's really a superpower of digital products. You know, for a lot of things, you build it, you test it, you release it. That's it. It's in the world. If I am making a widget and I test the widget with people who are going to use it and they seem to get it and the widget goes out into the world, I can't then update the widget. I have to release a new version. But all of the old widgets still exist and they're all still out there creating potentially poor user experiences. If I have a software, if I have an application based product and I release it and the color is wrong or the button's in the wrong spot, I can release an update and that retroactively updates all of the previous ones, or it should. That is

00:05:00

Joe Marcantano: something that, you know, in the grand scheme of products is fairly new. And we should be leaning into this iterative process so that we can get good products out there faster. And then if we need to tweak them to make them great, we can do so.

Drew Freeman: As long as that first iteration that you put out there is, not to use industry jargon, as long as it is a mvp, a minimum viable product, which means as long as it is not so bad that it just turns people off from your company in the future, you can make it better, you can tweak it.

Joe Marcantano: And an MVP is actually the perfect example of something that's an existential threat. Especially if you're a startup, you're not an existing company releasing A new product. You are a startup and this, this product is your company. The MVP is critical for you. That is an existential threat to the business. If you get that wrong, that is the perfect example of something that needs to be the proof beyond a reasonable doubt level of Ricker.

Drew Freeman: I love that as an example, the example that I had in my head was even more foundational and less evaluative, which was, you know, more along the lines of are there even, does this problem even exist for enough people that there's a market there? Does our potential solution even solve that problem well enough that people will buy it? Those are the kinds of existential problems that if you get wrong, you've just sunk a ton of money into a company that's going to go nowhere.

Joe Marcantano: So Drew, when we're thinking about existential business threats versus low consequence things, how do we kind of evaluate that? How do we know when good enough is good enough?

Drew Freeman: So I think there's two major prongs to my answer. The first is how do you know if you're dealing with an existential threat, a quick and easy change later or somewhere in the middle? And most things are going to be somewhere in the middle that involves really deep and meaningful conversations with your product stakeholders and your leadership and your executives to understand where on that spectrum it falls. Those conversations are relatively easy to have when you're an embedded researcher. When you are an outside consultant or an agency researcher, you might not be involved in those conversations at all. But you should definitely make sure that you are aware of the outcome of those conversations.

Joe Marcantano: And sometimes, especially if you know, you're an agency researcher, you're an outside freelance researcher, whatever, and they're not willing to share those things with you. The factor that's going to determine when good enough is good enough, or the factors are budget and timeline. You know, if they if your client comes to you and says, this is my research question, this is what I need to know and I have three weeks and X amount of dollars and that's it. Your response to that should be, this is what I can get you in that timeframe under that budget. Does this level of rigor meet your need? The conversation need not be, well, what level of rigor are you looking for? The conversation can be, this is what I can deliver to you in that timeframe. Does this suit your needs?

Drew Freeman: Yeah. I love that. As a follow up question, okay, so then the second prong to that how do I know when I've reached good enough? Question is pretty simple, which is have you answered your research questions? And if you've answered your research questions, then, and you've met that level of rigor that was required based on the first part of that answer, then you've reached good enough. Any time that you spend going beyond that, trying to perfect it, trying to make it even better, is not worth the time and the money that you're spending to do it.

Joe Marcantano: Totally agree. Why don't we talk for a second about, we opened with this question having so many different contexts. What are some other contexts you were thinking about when we say when is good enough good enough?

Drew Freeman: For me, the next most obvious one is slide decks and presentations. I could, I am not a designer, I am not a slide expert. I could spend all of my time tweaking and trying to make this slide deck as good as it could possibly be. And would I eventually make it better and better and better? Absolutely. Is that a good use of my time? Heck no. That's the next obvious area

00:10:00

Drew Freeman: for me where good enough is good enough. Is this visual slide deck helping me to convey information better. If this change that I'm thinking about making isn't going to do that, then it's probably not worth my time.

Joe Marcantano: You know, there's ah, an anecdote and a really great story about I'm sure you've heard of the song Hallelujah by Leonard Cohen.

Drew Freeman: of course.

Joe Marcantano: So that song, and somebody's going to fact check me here and that's fine, but it has something like 25 or 30 verses. Leonard Cohen apparently kept iterating, kept adding, kept writing this song for years and years and now most people hear a version that's you know, three verses, four verses. But the idea that like when is it done? He continued to work on it and work on it and it was never done. It was this kind of living piece of art that continued to grow and expand. That's not what you want from your decks. Like you want to get it to a point where it accomplishes the goal and then stop working on it. It is done. Do not continue to work on it.

Drew Freeman: And I think something that a lot of people have a hard time with is letting go of, that well, I need to make it as good as it can be and I need to get it to 100% of whatever it could be and whatever I can make it. 100% is not the number that you should be shooting for. This is going to vary based on, you know, how important it is, who is seeing it, etc, etc, but the kind of Baseline rule of thumb number that I always have in my head is 80%. Can I get this to 80% of as good as it could possibly be? Okay, that's good enough.

Joe Marcantano: Yeah. And there are some things that, you know, need to be at 100% when you're thinking about, like, are there typos, are there misspellings? You know, am I, am I fully answering the research questions? But then especially for researchers, when you're thinking about like the visual aesthetics of the deck, that does not need to be 100%. You know, it can be 80% and that's totally fine.

Drew Freeman: I'm a, I'm a big fan of good enough is good enough. Perfection is not what we are shooting for. What we are shooting for is answers to our questions and the ability to the information that we need to make the next decision. That's what we're actually shooting for. We're not shooting for 100% here. We are not in school. We're not trying to get an A. We're trying to make the next decision.

Joe Marcantano: Yeah. And of course there are going to be instances where you're going to be reading out to a larger group, you're going to read out to executives, you're going to present on something that is extremely important. In those instances, you want to be closer to 100%, but you need not be there every time.

Drew Freeman: It's always a sliding scale and you have to understand where you are on that scale in that given moment.

Joe Marcantano: You know, Drew, another place I think good enough is good enough could apply to is when we're talking about like study plans and discussion guides.

Drew Freeman: Oh, absolutely. You know, we've had, we've had an entire episode talking about how a discussion guide, that word guide is really specific and really important. They are not discussion scripts. They are discussion guides. You don't want. In a world where you were looking to get to 100% for a discussion guide, you could theoretically, script out and choose every single word that you're going to say with the utmost importance and care. But we know that to like, run them, run a session as well as you possibly can, you need to react to your participant and you need to think on the fly and you need to ask follow up questions. So all of that means that your script is going to go out the window. So that 100% as good as it could be guide is now not really 100% useful anymore. So you might as well just stick with, well, I've got my solid outline and that's good enough. And I can then come back to it and I can use that to center myself, but I can also bounce off of it when I need to.

Joe Marcantano: The thing I thought about when this kind of came up was, do you remember the old choose your own adventure books where you read three or four.

Drew Freeman: Pages and you died on every other.

Joe Marcantano: Page, but you'd flip around, it was, you'd read a page and a half and it was, if this, go to this page, if this, go to this page. If this, go to this page. And that's kind of the thought I had. I could, in theory, if I had enough time, map out a discussion guide that followed a choose your own adventure book that kind of accounted for

00:15:00

Joe Marcantano: everything. And if the participant says this, skip to here. And if the participant says that, continue, but that's not useful. that's not a wise investment of my time. I need to cover the two most common things, have a general idea of where I want to go with everything and move on. It doesn't make sense for me to try and plan out a discussion guide for every single edge case. My plan for the edge cases could simply be something like, tell me more about that.

Drew Freeman: Think about how many hours you would have to spend in your discussion guide planning to come up with all of those edge cases, especially knowing that participants are unpredictable and you're not going to.

Joe Marcantano: Think of everything you brought up number of hours. You know, if you don't work at an agency, if you're embedded, if you're in house, it's easy to forget that your time is also money for the business. And so when you're thinking about budget for a project, try to think about your time as the resource too. And you know, a real easy back of the napkin way to do that is take whatever your annual salary is and divide it by 2080. That is roughly the salary cost of your work per hour. Now, there are other things that increase that number like benefits and bonuses and whatnot, but that'll give you a good idea of like, Is this worth $2,000 of the company's money for me to continue to work on this? Is this a wise investment of my time from the business's perspective?

Drew Freeman: Even more than that, or maybe I think about it slightly differently. I think about it as, what else could I be using this time for?

Joe Marcantano: Oh yeah, the lost opportunity cost.

Drew Freeman: The opportunity cost is that to me is what actually gets me to stop working on something when I should have five hours ago. Because like you said, for most projects, time is the most costly and the most valuable resource that you have, what else could I be spending this time on rather than polishing up this slide deck for the third time, fourth time, fifth time.

Joe Marcantano: You know, I think that's one of, like, the reasons that agencies get as hired as often as they do by, UX researchers. Right. Like the UX researcher that hires an agency is saying, I have two or three big projects that are, like, require X number of hours. I could also do this other project. But the fact is that's not the wisest investment of my time.

Drew Freeman: I really do think that more researchers should understand and learn about opportunity cost. I come from, I have an economics educational background. So that is one of those things that gets drilled into your head. But that is something that as an everyday person, I think more people need to know about.

Joe Marcantano: Yeah. The real easy way to kind of break down opportunity cost from kind of an economics perspective into a layman's term perspective is just there are X number of hours in a day and you need to prioritize your tasks. And anytime I am sitting on the couch watching tv, that's time that I am not using to do the dishes, help somebody with their homework, whatever. Anyone who's managed a household has dealt with opportunity costs. They just haven't thought of the words opportunity costs.

Drew Freeman: Right, Exactly. So the idea that I want people to be like, thinking of is, is it better for me to spend this next hour, 30 minutes, 10 minutes, 1 minute, however small a chunk of time you want to think about it, is it better to spend that next piece of time doing this thing that I've been doing, or should I move on to something else? Is that going to be better for me?

Joe Marcantano: When is good enough good enough? And you know, one of the phrases, and I know that there are some people who don't like this phrase, but one of the phrases that I really like that helps me realize that good enough is good enough is perfect is the enemy of good.

Drew Freeman: I was just thinking that phrase myself.

Joe Marcantano: You know, I could spend 10,000 hours on something and get it to be perfect, but is good good enough? Another place that somebody might think, about this or see this, where, you know, this perfect is the enemy of good. If you ever look at professional endurance athletes, marathoners, triathletes, ironman competitions, you'll notice that when they get to mile, you know, when they get most of the way done, they're at mile,

00:20:00

Joe Marcantano: like 23 in the run. It doesn't necessarily always look like they're going all out anymore. Their objective in that day is not to get the best possible time. Their objective is to win the race and to win the race without hurting themselves so that they can go out and train in a few days, so, so that they can compete in the next event. So that is a really good example of they need not be perfect. They need to lock in their spot, lock in their prize money, and once they're good enough, they're already thinking about the next event, the next thing they're going to train for.

Drew Freeman: Okay, so how do we take that idea where it's super easy to understand. I don't need to be, I don't need to be going all out to win this competition. I just need to be 1 second better than second place. How do we take that idea where it's really easy to understand and really easy to see and translate that to something like user experience research where you're not directly competing with other people? There's not that easy to see competition.

Joe Marcantano: You're right, you're often not competing with other people or you're almost never competing with other people, but you are competing with yourself and you're balancing competing priorities. So, for example, let's say I'm working with a stakeholder who I have worked with dozens of times before. It's just going to be that stakeholder who sees the results. I don't need to burn a bunch of time on a deck. A, one pager might be good enough for them. You know, you need to right size your output for whatever the need is so that you can then take whatever time I would have used building a deck and dedicate that to the next problem, to the next need that the business has.

Drew Freeman: So I just thought of this and I'm a little bit embarrassed that neither of us brought it up before. But when you're wondering when is good enough, good enough, it is entirely okay. And you absolutely should ask your stakeholders, what do you need? How much do you need to see what does a good outcome look like for you? Because they might say, oh, I just need, like, I just need to be able to understand how to make this decision. And a, one pager will be, will do great. Or they might say, well, we need to present this to executives and we need to be able to quote, unquote, prove this. And that requires an entirely different level of work. There is no shame in either obliquely or just directly asking, what does good enough look like for this project?

Joe Marcantano: For sure, you know that you don't need to spend 10 hours building a highlight reel if it's going to one stakeholder who, you know, the outcome kind of already jives with what they were thinking. But if I'm going to convince an executive and what I'm going to present is going to be counter to their expectation or to what their previous belief was, that extra time in that highlight reel might make a difference, and that's going to be important then.

Drew Freeman: So I think to wrap up the thing that I want to come back to and have people really think about when it comes to specifically the research process. Good enough is good enough when you have answered the research questions and given your stakeholders the information they need to make the decisions that they were trying to make.

Joe Marcantano: Yeah, exactly. And when you're thinking about that, also consider your time, your budget, the severity of getting it wrong. All of these things kind of play into it. And ultimately it's through your conversations with your stakeholders that you're going to get the information you need to decide with preponderance of the evidence versus proof, beyond a reasonable doubt, where you need to be on level of certainty meter.

Drew Freeman: Thanks for this, thanks for this conversation, Joe. I know both of us are big, big fans of team Good Enough is Good Enough. So we could, we could talk about it for a little while longer, I'm sure.

Joe Marcantano: Yeah, this is, this has been a really good. A good chat and something that I think that is easily overlooked because it's, it's really easy to shoot for perfect all the time.

Drew Freeman: It's almost like we were trained for that as children a lot of the time.

Joe Marcantano: Yeah. And through school.

Drew Freeman: Yeah. Yeah. Don't get. I mean, there's a lot of formerly very academic. There's a lot of children who are academically gifted who are now burnt out. And, it's good enough is good enough conversation could really do them a lot of good.

Joe Marcantano: Agreed.

Drew Freeman: All right, well, on, that note, I'm about to go ready to get

00:25:00

Drew Freeman: to, watch some football and have my brain be scrambled by the amount of football going on. So thanks everyone for listening today. Please give us a like and a subscribe on your podcast platform of choice and special request today. Please tell one friend who's interested in user experience research about us. Your word of mouth and your recommendations are how we grow and it's really, really meaningful and important to us. If you have questions that you would like to hear us talk about and hear us answer, you can send those to us at InsideUxR@gmail.com. If you'd like to support the show, there's a link in the show notes where you can do that. I'm Drew Freeman.

Joe Marcantano: And I'm Joe Marchantano.

Drew Freeman: And we'll see you next time.

00:25:43