31. Why should I use realistic data in my prototypes?

Joe Marcantano: Morning, Drew. How you doing today?

Drew Freeman: I'm doing just fine, Joe. the listeners won't have to deal with this, but, I just rambled for about six minutes and we're scrapping and starting over, so hopefully I do better this time.

Joe Marcantano: I mean, I wasn't gonna call you out for that in front of everyone, but you just called yourself out, so.

Drew Freeman: Oh, I am the first person to call myself out. I am always happy to do that.

Joe Marcantano: Yeah, you like anything else. We certainly iterate and we did do a restart on this one as we realized the flow of the episode wasn't quite going the way we wanted it to.

Drew Freeman: Take two. We will ll be better than take one.

Joe Marcantano: That's right. So I've got a really cool question that I think a lot of non researchers don't think about. And I know you and I have picked fights over this before, and I think that I've definitely said I told you so when I lost the fight over this. And it wouldn't surprise me if you have as well.

Drew Freeman: So what's our question?

Joe Marcantano: Our question is, what is the importance of using realistic data in your prototypes?

Drew Freeman: You are 100% correct that this is a question that I love. This is the topic that I have harped on for years and years. And yes, I have definitely had my fair share of I told you so moments. So the importance of using realistic data is primarily because we want our participants to be focusing on the things in the usability tests that we need them to be focusing on in order to be able to give us the feedback that will help us answer our questions. For example, we want, we want participants to focus on the interactions within the prototype, the actions that they need to be taking. We don't want them to be focusing on the text or the content necessarily. I say necessarily because there are absolutely times when we are testing the content and testing the text itself. But for these situations, for this conversation, think about we're testing more than just content.

Joe Marcantano: Yeah. The breakdown I was going to do there was if we're doing usability test surrounding a dashboard, and it's, you know, the functionality of pulling a report, for example, we're talking about in this instance, the data that the Dashboard displays, not the functionality of the dashboard. What is the importance of the, actual numbers and data being reported? Why does it matter that that is realistic?

Drew Freeman: Yeah, it matters that it's realistic there because we need our participants to be engaging with and indulging us with their imagination. So's there's a term in movies, for example, that that goes the. It's called the suspension of disbelief. James Bond movies are a really good example. Really like any war movie is a good example of this. Those movies aren't realistic, they're not accurate, but we enjoy them because we're able to suspend our disbelief as audience members and just go with the, go with the world and go with the story that this movie is telling.

Joe Marcantano: Drew, I'm wondering if maybe you have like, a story or an example of a time where this didn't go the way you planned that might, like, really illustrate this for folks.

Drew Freeman: I sure do. And this is a story that I have told to m many people many times. So in my last job, I did a lot of testing with, with doctors, with nurses, with other medical staff. And so obviously we had medical data and fake patients that we were using for this testing. I was moderating a session that I, not, I did not write the study plan, I did not create this fake patient, but I was moderating the session. And we had medications prescribed to this fake patient. And we weren't testing. We weren't testing anything about the fake medications. We weren't asking the, asking the doctor to review them and make decisions, anything like that. They were just there to fill out our fake patient and try to make this more believable and more full bodied, for lack of a better word.

00:05:00

Drew Freeman: Well, turns out we didn't do our due diligence. And at one point in the session, after looking over that list of prescribed medications, the physician turns to me and says, well, this patient isnna be constipated for a week based on the different medications that we had prescribed and how they interacted with each other at that moment. I groaned internally, put on a brave face, and kept going. But I did not get anything useful. I did not get any usable feedback from that participant for the rest of the session because they were no longer engaging with the world that we had created for them, and instead they were trying to pick holes and find other mistakes that we had made.

Joe Marcantano: I've had countless similar scenarios where the data that was put in was simply not realistic and it pulls the person out of the world we're trying to create out of the situation. And then we know moving forward that rather than evaluating the prototype or the flow, they're evaluating the realisticness of the data. And that exactly really reduces our level of certainty and the value of our insights.

Drew Freeman: However, there are times when using placeholder text, dummy text, non realistic data is valuable and appropriate in a usability test. And I'm going to start with what is probably the most common and the most well known, and that's Lorem Ipsum text. And Lorem ipsum is dummy or placeholder text that is most commonly used in graphic design, in publishing, it's even used in typesetting for books and book publishing. But basically what it is, it's text that is used to fill in empty spaces where real text is going to go. But we don't have that real text yet.

Joe Marcantano: I think this is a good place to point out that, you know, for someone who's not familiar with Lorem mpsum, it is essentially not English, it.

Drew Freeman: Is nonsensical, it is not even English. It is based off of a Roman text, by Cicero, but it's not even real Latin. So it's based off of Latin, but it's been garbled and corrupted purposefully. So it's not even Latin. You can't make any sense out of it.

Joe Marcantano: Yeah, and there are several websites online where you can tell them, you know, I need two paragraphs of five lines each of Lorem Ipsum and it will essentially generate some, some random garbled text that you can't make sense of.

Drew Freeman: It can be super useful when you are, when you're just trying to fill space that would otherwise be blank.

Joe Marcantano: Why don't we dive into a couple of instances where, where it's right, where it's appropriate to use Lorem ipspsum as opposed to real text or you know, I'm, using air quotes. Real text or real data.

Drew Freeman: The first thing that comes to mind is when I wantn to test, kind of the foundational architecture or layout of a website, for example, you know, do I want to put the menu in the top left, do I want to make the menu in the bottom left, for example? That kind of, that kind of question can be answered with, you know, by including Lorem ibsum text because it doesn't matter what is actually presented there. I just need filler so that it doesn't look weird.

Joe Marcantano: I think the only thing I would add to that is it's, you know, when you're testing, doing a test that the realism of the scenario is a little less important. So that those kind of structural things. Right. Is the menu in an intuitively easy to find place, that kind of stuff as opposed to what happened? What's your workflow? For whatever reason, folks who work in power plants came to mind. So what is your workflow when the temperature gets too high, what do you do to check the safety points or whatever? Right. So, like, there, the realistic data is really important. But if I were to tell you, where might you go to, you know, we're using an audio software, where might you go to save the recording? The details of the audio file or the audio wave that's being displayed isn't really important there.

Drew Freeman: Right. an example that you brought up earlier is a dashboard that is showing various different reports and various different data. If I'm trying to understand if that data is

00:10:00

Drew Freeman: useful and understandable, I absolutely need realistic data. However, if I'm just trying to understand, is this too much data? Is it overwhelming? Do I have, Do I have the various widgets in the right place? I may not need realistic data and I may be able to use something like lorem ipsum in that case.

Joe Marcantano: So true. Let me ask what seems like a silly question then. why is it important to use lorem ipsum instead of just copying and pasting the, you know, my favorite paragraph from my favorite book and sticking it in there? Why does that matter?

Drew Freeman: That's a good question. So lorem ipsum specifically because it is not English, it is not Latin, it is, it is gibberish, but it is created in such a way that you get, you know, realistic lengths of words. So what it does well is that it looks like it's real text, but no one can read it and get meaning from it. Whereas if you picked the favorite paragraph from your favorite book, that text does have meaning and a participant might get lost in that meaning and in those words in a way that they can't really get lost in. Lourm ipsum because it's just gibberish.

Joe Marcantano: Yeah. The last thing you want to do is provide a distraction. You. We talked about the importance of real data and how that kind of keeps people, in the scenario in the context you've created. If I were to just put in my favorite paragraph from my favorite book that you're intentionally pulling them out of the context and that's the last thing you want to.

Drew Freeman: Exactly.

Joe Marcantano: What about when we're super low fidelity? Maybe we're testing wireframes? Do I really need to put lorem ipsum or real data or any kind of dummy text in there? How Do I proceed there?

Drew Freeman: So I think this is these kinds of lo fi, super early drawings, they're not even prototypes, just drawings are super valuable for demonstrating why Lorem Ipsum or other filler text is, is valuable in certain situations. Because those wireframes are even another level more extreme. Because usually in those wireframes you don't even have Lorem Ipsum you just have squiggles that represent where writing or where text will go. And at that stage, even having something like Lorem Ipsum is going to look out of place with the hand drawn, not perfect. Everything else that's going on in a wireframe or in a drawing, in a sketch. And so having something as low fidelity as squiggles fits in and doesn't pull people out of the, the world that you're building for them.

Joe Marcantano: You know, it reminds me of, I want to say it was Neil Degrasse Tyson who said, it's really easy for me to suspend disbelief so long as the rules for that universe are consistent. And that kind of fits in here. It's really easy for me to believe that these squiggles are an app or a computer program or whatever as long as the rules within that little universe that we've created are consistent. If you show me hand drawn outlines and boxes and menus and then we have perfectly written Lorem Ipsum or perfect dummy or filler text, there's this separation in my brain that becomes really hard for me to accept that this is all one world, and the context that I've created.

Drew Freeman: That's a really good way of explaining it. You need to adhere to the rules of the universe that you've created. Whether those rules are super low fidelity and sketchy like in a wireframe, or if they are super accurate and real world adjacent or real world like as they would be in a high fidelity prototype.

Joe Marcantano: Now, I'm sure there are some folks out there who have listened to everything we've said and said. Yeah, this is super obvious, like why would this even be a thing? And I think you and I have kind of right now talked through perfect world. You know, if everything's perfect and the designers have time. But I think that it's realistic, if not fairly commonplace, for us to look at a, prototype and realize that the data is not consistent with the world or it'they're using Lore or Ipsum when they should really pull in some realistic dummy text or whatever.

Joe Marcantano: Let'why

00:15:00

Joe Marcantano: might that happen? Why are we even picking this fight to begin with? Why does the fight exist.

Drew Freeman: As is often the case, the reason the fight exists is that we don't have infinite resources and it takes longer, it takes more resources to create that realistic fake data than it does to just plug in Lorem Ipsum or dummy text. So you have to, you have to pick your fights and figure out where is it actually crucial to have that realistic data and where is it not crucial and where can we get away with squiggles or Lorem Ipsum or whatever it may be.

Joe Marcantano: Going back to the examples, the story you told. Presumably all of the developers and designers involved in creating that prototype were not doctors or clinicians. And so the research that they would have done had to do for somebody without a medical background, might have been fairly extensive and might not have been the wisest use of their time. whereas maybe they just needed to consult an expert real quick and say, hey, what are two or three medications we can put in? You know, it'in the story you told, it was clearly worth it, they should have done it. but you know, they've got to balance the time it takes to create that versus whatever it is they're pulling time away from whatever projects, other projects they're involved in.

Drew Freeman: Absolutely. And the process that we ended up going with and this process was in place before this example that I gave. So it's not a foolproof process. But what we, what we always had folks do and we required in certain situations was you have to get sign off from a subject matter expert, whether that is a physician who understands the area that you are testing or whether that's a very senior developer when it comes to developer tools, whatever you need us me to sign off and say, yep, this is realistic, this is good to go. And that can be kind of a good balance between spending too much time that isn't worth it in doing that research. As someone who's writing the, writing the study plan and coming up with the data versus going the other direction and having Laurm Ipsum in a place where you really need fake but realistic data. another solution that not everyone is going to have access to, but if you do have access to is really, really valuable, is actual customer data. And then you can take that actual customer data and de identify it so that there isn't a single organization or a single person that's involved. That's the best of all worlds, because then you've got actual real data in all its messiness and weirdness and it is repeatable because it's not real. You're not messing with anyone's actual profile or whatever. That's the ideal solution. But the ideal solution isn't always what comes up.

Joe Marcantano: So we've talked about the importance of using real data and we've talked about why this fight exists, why we need to pick it at all. The final thing I want to talk about for our last section here is what should I do if it goes wrong or if I can't win the fight? What should I do if my lore Ipsum, my dummy text, whatever is not appropriate given the testing and because of the timeline in the budget and whatever, I have to move forward? How do I handle that?

Drew Freeman: That is a good question that doesn't have an easy answer for me. The first thing I'mnn the first thing that comes to mind is I'm going to try to take on that work myself. I will go to the smeeze and say, hey, what are, what's realistic here? I will try to take on that effort and shift that time to me because I know it's so important. I'm going to turn the question back on you. Let's say that you can't do that. You can't take my approach. What would you do at that point?

Joe Marcantano: So what I have done in the past is I address it head on with the participants and I just let them know, hey, I don't want you to focus on the words on the screen. I don't want you to focus on the numbers. That is dummy data. Pretend it makes sense to you. How would you do this?

Drew Freeman: The other important thing to do at that point then is to really make sure that your stakeholders, as you're communicating results, know that this was a limitation of the study and what we learned might be slightly tweaked and colored by that.

Joe Marcantano: Yeah, exactly. You know, I would never sit here and suggest that you should not let a test be the best it could be. But

00:20:00

Joe Marcantano: I do think there is some value in at some point letting stakeholders understand that there is a lower degree of certainty that is delivered with these results. And it is because of X, Y and Z.

Drew Freeman: It goes back to, a phrase that I love and I know that you're a proponent of, which is sometimes good enough is good enough and you have to make a decision of, let's say, is 60% of this test better than 100% of no test.

Joe Marcantano: Exactly. Drew, is there anything else we need to cover when it comes to realistic data, Lore Ipsum and dummy text?

Drew Freeman: I think the thing that I would go back to is realistic data is the always the platonic ideal of what you should be going for always is strong. Almost always the platonic ideal of what you should go for. With that being in mind, aim for that and know that you're not always going to reach it. But really do think about what am I trying to learn? How does the text, how does the content, how does the data play into that? That'll help lead you down the right path.

Joe Marcantano: I completely agree. I think that like any testing, the biggest thing to understand and to convey or to know ahead of time is the limitations. And just make sure that you know, if you're running a test for stakeholders and there simply isn't time and it's not possible to get the appropriate filler text, just make sure they know ahead of time, hey, I'm gonna do the best I can, but these are the limitations, these are the potholes we could.

Drew Freeman: Run into and think, I think I'll leave us with the final, final piece of advice, which is what you said about the rules of your universe should be consistent. Think about that. You know, think about what is the, what is the level of realism that surrounds this text? If it's high level realism that surrounds the text, you should have highly realistic data. If it's a low level of realism that surrounds the text, text, a low level of realistic text is actually what you should be going for.

Joe Marcantano: I think that's as good of place as I need to land this plane for the day.

Drew Freeman: Hopefully I did better on this second time around than I did on the first. That you will never hear.

Joe Marcantano: I have zeroed out that you did. I want to thank everybody for joining us today. If you like the show and you want to help it out, tell a friend, let somebody know, tag Drew or I or the show on LinkedIn and give us a shout out. If you have a question that you wantna hear us talk about, send that over to inside uxrmail.com. we'd love to talk about your questions and what you wantna hear us talk about. And if you'd like to support the show, there's a link in the show notes where you can do that with that. I'm Joe Marantano.

Drew Freeman: And I'm Drew Freeman.

Joe Marcantano: And we'll see you next time.

00:23:00