29.  Should I use a mock or prototype for my study?

Joe Marcantano: Hey, Drew, good to talk to you again.

Drew Freeman: Yeah. Ah, welcome back for another episode.

Joe Marcantano: Yeah, I'm excited about this episode. When you suggested this, I thought this is really great. And then as we were outlining it was like, oh man, I didn't even think of that. This is awesome.

Drew Freeman: And that's why researchers should talk to each other.

Joe Marcantano: Exactly. Collaboration is always going to provide better answers than one person working in a room by themselves.

Drew Freeman: But before we get into the episode, Joe, I know that you've got some exciting news to share with folks.

Joe Marcantano: Yeah, I've got two big announcements. The first is, Drew, you and I are no longer coworkers. I've accepted a new role and I am now working for a, in house research at a, larger tech company.

Drew Freeman: Yeah, obviously I said this to you in person when you told me, but I'm really excited for you to see where this goes. But I'm really sad that I'm not going to get to work with you on a day to day basis anymore.

Joe Marcantano: It was very bittersweet and it really came down to this was an opportunity that would be foolish for me to pass up.

Drew Freeman: So do you want to talk a little bit about what you're going to be doing or do you want to let that kind of come out in its good time?

Joe Marcantano: Let's let that come out. I'll say that I've been hired to kind of start a new, program within the research department. And there is a part of me that doesn't want to talk about it because I'm not 100% sure what it's going to look like yet. I've only just started and I'm still kind of learning the problem space. And I don't really. I've got a couple of ideas written in pencil, but I don't really have a solid plan yet because I'm still kind of learning the problems and what success looks like.

Drew Freeman: But I will give folks a little bit of a teaser and say that this problem space is already. Is something that we already have an episode on in this podcast feed. I'll leave it to the listeners to go listen back and see if you can figure out which episode I'm talking about.

Joe Marcantano: Yeah, I think that's a good place to leave it for now. definitely more details to come, but.

Drew Freeman: You'Ve got another exciting announcement.

Joe Marcantano: I do. certainly was not planning on these two things happening at the same time.

Drew Freeman: But life doesn't care about our plans, does it?

Joe Marcantano: Exactly. So I have, I, applied and have been chosen to be a speaker at the UX Insight festival, in 2025.

Drew Freeman: So obviously, folks should go to the UX Insight Festival, if only to hear you speak. But how else might people be able to hear this if they can't go?

Joe Marcantano: Yeah, so there are online options for online tickets. you can find some more details@uxinsight.org. the conference is April 14th through the 16th, and I did get my speaking slot time the other day. I will be speaking on Tuesday, April 15th. I'm notnna say the exact time because I don't want to do the time zone translation in my head. the conference is being held in the Netherlands, so it is going to be a, European centric time zone. So by the time this episode comes out, the full schedule should be posted on their website. You can go to their website, uxinsight.org do you can see the time of my speaking slot if you are so inclined and kind of do the translation into your local time zone.

Drew Freeman: And if we have our act together, we'll put a link to the schedule in the show notes and we'll maybe even give a couple of time zones in the show notes as well.

Joe Marcantano: Well, you are asking a lot there, Drew.

Drew Freeman: Why? I said if we get our act together, who knows?

Joe Marcantano: Yeah, but it is very exciting. you know, this is going to be my first time speaking at a conference in person, so I'm very excited about that. And, I'm still working out. I'm still ironing out the final points on my talk. I know the big thing that I want to talk about, but I'm still kind of ironing out some of the story I'm going toa tell. But I'm excited to share that story with everyone and hopefully lots of folks will show up in person or online.

Drew Freeman: Okay, so

00:05:00

Drew Freeman: with those exciting announcements out of the way, what are we talking about today, Joe? What's our question?

Joe Marcantano: So today's question is, should I use mocks or prototypes for testing?

Drew Freeman: I love this question. This is something that I really think researchers and, design teams should spend a lot more time thinking about than they really do.

Joe Marcantano: I think a lot of times this decision ends up being made for researchers because it's what does the designer have time to put together before fielding? But I think we should be more intentional about when should we use a mock, when should we use a prototype, when is one better than the other?

Drew Freeman: So the place that I want to start this conversation is the place that I think researchers should start when they are deciding which one to use. And side note, it is also completely acceptable and sometimes the best option to use neither and to have no visual as you're doing your testing. The place that I want to start is by thinking about and by asking, what are we trying to learn, what are we testing and what I mean by that. Let me give you some examples. Are we trying to learn how people use this tool or can people use the tool easily? Well, in that case, I probably want to use a prototype that has that level of interactivity built in, because I need to see that those behavioral interactions. And of course to do that you need interactivity. But if I'm testing something like can people understand the information presented to them on this informational dashboard or in this report, in that case I probably don't need that level of interaction. And a mock screenshot would be perfectly acceptable. Or maybe a case where nothing, you know, no visual is probably the best answer. Maybe I'm trying to learn what are people doing in this space today? What challenges do they face? In that case it might be distracting to have any sort of visual.

Joe Marcantano: Yeah, I think as a good general rule, if you are testing what is basically one screen, a mock is gonna be fine. You can do that. But you are testing anything that is going to involve a change in screens or, a flow of any kind, you are going to kind of want to default to the prototype.

Drew Freeman: So I think it's worth for just a second, you know, defining those terms because I think, ah, good call. I think mostly people know and use those terms the same, but I know for a fact that not everybody uses those terms in the same way that we just did. So when we say mock, you know, you heard me call them mock screenshots before because typically when we're talking about a mock, we're talking about, a static, unchanging screenshot that doesn't have any level of interaction. It's just a visual representation of what folks are going toa be looking at. Whereas when we're talking a prototype, we're talking about something that does have that level of interaction. I can go from one screen to the next. I might be able to go forward and backwards. You know, that's what we're talking about. And there are lots of different formats that a prototype can take. It could be an HTML prototype, it could be something that is actually coded in whatever coding system you're using and maybe just isn't released to the public yet. It could be, you know, something that's in a, testing environment, for example, is something, you know is a kind of prototype that I've made use of a ton. Or there are also different prototypes that you can basically string together or tie together static screenshots with something as simple as PowerPoint. And you can use shapes and invisible buttons and whatnot to create hyperlinks between the different PowerPoint slides to give the feeling of some level of interaction and interactivity without having something that rises to the level of a prototype. So it's a scale. There isn't a hard and fast. This is a mock. This is a prototype. But that's what we're talking about when we say these words.

Joe Marcantano: And I think we should point out that we're not today gonna talk about kind of the differences between high and low fidelity prototypes in mocks. Well, maybe that'll be a separate episode.

Drew Freeman: That needs to be a separate episode. Cause that is also a very important distinction.

Joe Marcantano: Yeah. So we're not gonna make any distinctions between wireframe versus a very high fidelity, mockeer prototype today. But just keep that nugget in your head that there are also differences there.

Drew Freeman: Okay. So I posited that the place that everybody should start

00:10:00

Drew Freeman: when they're thinking about this question is what do you need to learn? I think that the next place to go is Joe. Is one of these better than the other?

Joe Marcantano: So the short answer is no. It totally depends on what you need to learn, what it is you're trying to test. And then really there are other considerations as well. There are considerations of your timeline, of the importance, the business consequences. All of these things kind of inform which one you should use. And one is not better than the other.

Drew Freeman: So some of those things that you have to think about that will inform whether a prototype or a mock is most appropriate for your situation are things like, how much time do we have? It can often take more time to create a prototype than it than it does to create a mock. What level of risk tolerance are we okay with? if we have tolerance for a. Like, this is a very, very critical decision that is going to go all the way up to the C Suite, for example. There might be a necessity for a prototype at that point simply to kind of minimize some of the risk that's involved that, well, Maybe this feedback we got isn't actually going to translate to the real world. A prototype can do that because it more closely emulates what the real world situation will be as opposed to a mock drew.

Joe Marcantano: I know, I said that we were going to save this for another episode, but maybe this is the right time to dive into high versus low fidelity.

Drew Freeman: You know, I was just thinking the same thing. So let'okay so let's define what is high fidelity and what is low fidelity. You'll also, of course, hear this called high fi or lo fi.

Joe Marcantano: The easiest way to think about this is on the extremes. Low fidelity is going to be wireframes. It's going to be either computer drawn or hand drawn. It's going to look like scribbles and drawings. High fidelity on the other side is going to mimic as closely as possible the real thing.

Drew Freeman: Yeah, I was even thinking of it, I was thinking of it, more in how closely do these things mimic the real product? So the highest fidelity that you can possibly get is the live product that people are actually using. That is the highest fidelity that you can get. And then as you go down the spectrum, you move further and further away from that full live working product to, like you said, think of a drawing on a cocktail napkin. That's about as lo fi as you can possibly get.

Joe Marcantano: Drew, I don't know if you have, but I've even done low fidelity testing with things drawn by hand on eight and a half by eleven sheet of paper. And I have hundred percent papers.

Drew Freeman: In the right circumstances, that is far more beneficial than actually working through a live product. Okay, so let's talk about whether one is better than the other. Spoiler alert. Spoiler alert. No, they're right. They're each right in different situations. So let's talk about some of the pros and cons of each one.

Joe Marcantano: So let's start with the wireframes. You know, the pros here, and this.

Drew Freeman: Is to be clear, this is the low fidelity.

Joe Marcantano: Low fidelity, yes. the pros here are that m it is faster to make. Right. Your designer can presumably bang out some wireframes much faster than a working interactive prototype. It also is a little less reliant on you. The copy the text in there, you can kind of put some squiggly lines and say, this is text, this is filler text.

Drew Freeman: So I think something that is a benefit that is maybe even more important than either of those and honestly doesn't get thought about all that often. But I can guarantee you, when I was embedded within, directly within a development team. This is something that we talked about all the time. A big benefit to these low fidelity wireframes mocks drawings is that participants feel like it is an unfinished product and with it being an unfinished product, are often more comfortable and more willing to give critical feedback or feedback that would result in big changes. And I'll use the counterpoint to kind of help illustrate that if I'm doing a really high fidelity mock up, people will often feel like,

00:15:00

Drew Freeman: oh, well, this is so close to done that I can't tell them that they got it completely wrong. But if it's just some squiggles on a piece of paper, people feel like, oh, this is still very much a work in progress. I can tell them that this is not the right path. And they feel much more confident giving that and giving us what we actually need to hear.

Joe Marcantano: That's such a great call out. Yeah.

Drew Freeman: And like I said, that's something that you wouldn't think about unless you had made that mistake and actually live that in the real world. And I can raise my hand and promise you I have lived that. I have absolutely realized only after we did a study, well, shoot, we did this, but because we did this in a testing environment, for example, people said, you know, I can see now. People said. And people thought, well, I can't tell them that they got it 100% wrong and that they need to start over because they've clearly already put a bunch of work in.

Joe Marcantano: Drew, are there any other pros or times you can think of where it's more appropriate to use a lower fidelity mock up or prototype?

Drew Freeman: I think if you are, I mean you can use low fidelity for any different method or tool, but they are particularly appropriate and particularly important when you're talking about those kinds of co design sessions or co participatory sessions where you're really working together with the participant to create something. Those are really where your lower fidelity kinds of tools become extra important.

Joe Marcantano: I totally agree. And I think that the places where a higher fidelity, example or prototype or MOC is more important is when you really need the participant to be fully kind of in that headspace in that moment. Re, maybe you're closer to final deployment or final production and you really want them to feel like whatever it is they're testing is the real thing.

Drew Freeman: Absolutely. So I think of high fidelity for questions like can people use, can people complete this task or complete this workflow easily, effectively without errors? I want that to be as close as possible to the real world and that means high fidelity. So another example of times when I've used high fidelity is that this was kind of a showcase. This was not only a, you know, a, Pure. Pure is the wrong word. This was not only a usability test to gather feedback. This was also kind of a, you know, a demo and an announcement and a showcasing to the world of our users. Like, look at this thing that we've done. And then finally there is some practical realities that are kind of like internal politic realities. So I think about, you know, maybe you've got some really high level executive sponsors who are involved in this project and maybe they don't really, you know, they probably don't have a deep understanding of usability and maybe to avoid the conversation of. Or to mitigate the impact of. Well, this was just a mock up. This was just a, you know, this was just scribbles. This data that we got can't be any good. I might do it. I might do the test. I might skew a little bit more high fidelity than I otherwise would simply so that I don't have to, you know, combat that problem, which can absolutely derail implementing usability learnings for sure.

Joe Marcantano: The one other example I'm thinking of, of when a low fidelity might be more important or a lower fidelity is if you are trying to do some blind testing, you do not want the. Maybe you're testing a new feature on an existing program and you do not want the reputation of the existing program to influence the testing. So you might do some lower fidelity to kind of conceal, hey, this is a, you know, making stuff up. This is a meta program or this is a Google program, or this is a Microsoft program. Whatever you're trying to kind of conceal, the fact that this is a product of whatever company is you're testing, that.

Drew Freeman: Is such a good call out that I hadn't even thought of. That's such a great example.

00:20:00

Joe Marcantano: I want to kind of bring us back to MOX versus prototypes. And one of the things that I said at the start was a lot of times this decision is made for us because of the time constraints or the demands on the designer. So Drew, what if we've looked at the question, the thing we want to learn and we as the research have determined, hey, a working prototype is what is best to test this. And then we go to the designer and the designer says, I don't have time, I have too many competing priorities. The best I can do is get you some static screenshots. I can get you three or four Screenshots. What do we do then?

Drew Freeman: Couple of different things. And that reality. First of all, this reality happens all of the time.

Joe Marcantano: All the time.

Drew Freeman: And it's something that you're gonna have to have tools and things that you have in your toolbox and in your toolkit to be able to work around, because we are almost never going to get 100% of the situation that we want in any research. So what I like to do is I'm thinking of two really big things. One, I need to then set expectations in my readout with the team that, hey, I understand that we do not have the time to make, a prototype here. That is fine. But I need you to know upfron, before we start doing the research, before we start running with participants, there might be X, Y and Z impacts from that. Those can be things like, you know, there might be a little bit less certainty in the data that we have. There might be some questions that we just can't answer, and we have to be okay with that. Those are kinds of, the, those are the things. Those are the expectations that I need to be very clear about setting once, once it comes out that, okay, we've determined a prototype is, in the ideal world, what we would use. We don't have the time, we don't have the resources to do it. Okay, that's fine. I just need to set some expectations and some ground rules with you. The other thing that I am thinking about is specifically within the moderation of this test or of this session then. So I'm thinking about how can I do as much as possible to tie these mocks together for the participant? And I'm talking about things like, as I'm transitioning from one screenshot to the next, I'm going to say, okay, so before we move on to the next screenshot, I want you to think about this is what happens when you click this button. It takes you to this screen. And now I'm going to ask you some questions about that. The other thing that I'm doing is I'm slightly tweaking how I ask my questions rather than setting up my task as a. Or asking my questions is like a. I want you to do this. I'm going to ask things like tell me where you expect to click to do this. That sort of thing. Those are the kind, those are kind of the two big things that I'm thinking about in this situation.

Joe Marcantano: I totally agree. Those were the two things I had in mind as well. Anything else you want to cover on mocks or Prototypes.

Drew Freeman: So said we said that we were gonna, you know, we teased doing, an episode about lower Hii previously and then decided to actually do it. I am actually going to tease another episode that we're going to record later. And that is kind of once you've decided whether to use a mock or a prototype. We've talked about that fidelity and we've talked about how important the like, content, the text, the images, whatever is. So we're go goingna deep dive into that next week and we're going toa talk to you about the importance of the realism of that content.

Joe Marcantano: Yeah, this is definitely an area that I have, you know, metaphorically picked fights over before. And I think that it is something that is extremely important.

Drew Freeman: 100%. This is an area where I have essentially thrown away studies because I realized I didn't do this well enough.

Joe Marcantano: Yeah, for sure. So I think that's gonna wrap us on MOCX versus Prototypes though, and even high and low fidelity. Drew, thanks for sharing your thoughts and your expertise on this and thank you to everyone for joining us. If younna help or support the show, give us a like and a subscribe. Wherever it is you get your, your podcasts, give us a shout out. share the show with a friend and then send any questions you have over

00:25:00

Joe Marcantano: to us at insideuxr@gmail.com. if you want to support the show, you can do so through the link in the show notes. Hopefully I've gotten my act together and there's also a link to the UX Insights Festival and the schedule as well as where you can get tickets and the exact time slot that I'll be speaking with that. I'm Joe Marantano.

Drew Freeman: And I'm Drew Freeman M And we'll.

Joe Marcantano: See you next time.

00:25:26