Paper summary - Preparing Future UX Professionals: Human Skills, Technical Skills, and Dispositions, by Emma J. Rose, Cynthia Putnam, Craig M. MacDonald

What I read

In this text, the authors describe the output of a set of interviews with practicing user experience design managers. The results show an expectation that user experience designers have a broad set of skills, and these skills include hard, soft, and “disposition” capabilities.

First, the authors briefly describe a background of UX as an interdisciplinary field that attracts people from a variety of backgrounds; these people do a variety of activities in their professional roles. They indicate a focus on two academic disciplines as a source for educating designers—human computer interaction and “TPC” training (technical and professional communication.) The authors cite examples of how exploring industry practices has been used to influence educational strategies in other contexts.

The authors describe the way skills are typically considered in the context of career success: organized into groups of either technical abilities and knowledge, or interpersonal and “soft” skills. They add a third category, “dispositions,” which they define as “a person’s motivation for acting in certain ways.” Previous research studies have explored these skills, and the authors cite other studies that show user experience designers require a breadth of skills and crossover between these skills. They cite other research that indicated the importance of persuasion in the professional design process, in order to “forward particular arguments in a strategic and savvy way that helps to forward particular arguments and design decisions within an organization.” The authors then indicate the goal of their research is to identify skill sets and qualities of new user experience practitioners.

The researchers conducted 71 interviews with practitioners, who had at least five years of experience, and at least 6 months in a leadership role. Participants had, on average, 13.5 years of industry experience. They then describe their approach to data analysis. First, they individually inductively coded the interviews, and then, using the most common categories, deductively coded the remainder. They merged their coded results, and identified the most common categories of results.

The authors then describe their findings, mapped to the categories of technical skills, human skills, and dispositions. The top four technical skills were research (n=30), design (n=21), process thinking (n=15), and information architecture (n=10). Each skill category is briefly described, and supported by a participant’s quote. The top four human skills were approaching problems (n=29), communication (n=26), collaboration (n=21), and storytelling (n=16). Again, each category is briefly described and then supported by one participant quote. Finally, they follow this pattern to show the top four dispositions, which included independent/self-starter (n=18), open-minded/flexible (n=17), curious (n=15), and passionate (n=14). Curious and passionate are identical, indicating that there was a copy/paste error.

Next, the authors identify two unique elements that “set candidates apart.” 7 participants described the value of a “T-shaped” candidate, someone who has depth in one area and a broad set of general skills as well. 49 participants indicated that human skills were important, and 40 indicated that dispositions were important.

Finally, the authors discuss their findings and how those findings might impact how user experience design is taught. First, they indicate that there is a need for designers, but graduates “face fierce competition for UX jobs.” Next, they describe the broad finding from their interviews—that user experience practitioners are expected to have a wide variety of skill sets. This is in line with other industry reports. The authors describe that this research contributes a new perspective on desired or required industry skills—the perspective of dispositions, and that a person’s “character or personality makes them more suited to be successful in securing and succeeding in a UX position.”

The authors end with a “two-part call to action for educators.” The first part asks educators to “work together and across traditional disciplinary silos.” They describe that there is a lack of places for a student to study in this cross-disciplinary way. The next “call to action” is an observation that students can learn the soft skills and dispositions (that they are not innate.) They question—“What would it look like to craft our learning outcomes to engage in helping students develop as self-starters, to be humble or curious, and so-on?”

What I learned and what I think

In terms of interview content, I am not surprised that a major conclusion is that there is a wide perspective in industry on what a user-experience professional is and does, and a broad view of the skills that are required to succeed or gain early employment.

On the technical side, I think this is because the phrase caught on without a clear definition, but as a pointer loosely to design in the context of software; it then became abused by the “UX/UI” labeling (easy/lazy mnemonic) from product managers as a shortcut for “wireframes/comps” or “interaction/visual”; and then further diminished by the bootcamp approach to a set of dumb set of steps across generative research, interaction design, usability testing, and visual design.

On the soft-skill side, I don’t think the interpersonal skills described are unique in any way to design; I have a feeling that any strategic or “thinking” role in a large company requires those capabilities, and hiring managers in those other fields would say the same thing. Yes, it’s important that a developer can code, but they need to describe what they are doing, work together, and so-on. Yes, it’s important that someone in marketing can… market… but also that they can drive consensus, make friends, and influence people.

These base findings are based on valuable data, in that the participant sample size was large enough that the generated data will feel trustworthy by a skeptical audience, and that patterns can emerge towards the types of groupings that were described. And, that the research supported existing research (cited) is also useful.

The method is the same accepted inductive/deductive/coding/confirmation analysis that keeps reappearing in this type of literature, and I’m increasingly convinced that it’s just a not-useful approach. I think my main critique of it is emerging as a cat chasing its tail: the method proposes something, and then uses it to find itself. I don’t in any way fault the researchers for using it, because it seems to be a publishable approach.

But…

The authors started by indicating they don’t think the soft/hard skill binary distinction is valuable, and they offer a third category of dispositions. They offer no rationale for including this, but then use it as a container to organize their results, and indicate that it is a major finding: “As our findings show, UX practitioners are expected to have a wide variety of skill sets that cross the technical and human binary and extend into character traits, as represented by the category we call dispositions.” The findings show the first half—having a wide variety of skills crossing the two categories. The findings don’t support the dispositions at all; the interpretation was circular. The introduction of it seems appropriate, although an explanation would be valuable. The use of it as an organizing principle seems completely appropriate, too, if it emerged from the data. But it didn’t.

I think this is important because there are many ways to slice interview data, and since they started looking for skills in three containers, they found skills in three containers. What if the containers were identified bottom-up? What if there were no containers? What’s the point of doing research if the conclusions, drawn from the data, are based on the method and not the data—why not just talk to some people and then offer a polemic or opinion piece?

I’m also back to publish-ability. Why is this a valuable contribution to the body of knowledge around user experience education and hiring? They cite a number of studies (including their own) that already described the different types of skills and capabilities people need to be successful in user-experience. How is this different?

The two implications of the work at the end seem entirely disconnected from the data. The data most directly shows that there is a broad set of skills that make up user experience design, and that there is little agreement between participants about which are important. But the first finding calls for a working-together approach in academia, that extends across disciplines. I don’t see how one connects to the other. From the data, it more closely follows that we need to better understand if there’s a hidden relationship between skills as they map to parts of an organization, and teach to those relationships (which are probably highly specialized.) Or, it more closely follows that a student needs to be a master of all, which then implies that education needs to be longer in duration. Or, it most closely follows that we need to find ways to engage with industry at a more discrete level, so when we send students out to get jobs, their portfolios are better aimed at the unique needs (that probably map to verticals, size of company, and maturity of design organizations.) Or, or or.

The second finding makes very little sense to me. The authors say that they “push back on the notion that it is not possible to teach soft skills and the qualities identified as dispositions.” Push back on whom? They only mention the idea of innateness twice in the paper, and briefly; the first is a subset of the importance of human skills: “Further, participants believe that the technical skills, like design or usability research are much more straightforward and something that can be taught, whereas the human skills and dispositions are both harder to teach and key to a UX person’s success…” and next in the implications section: “Throughout these interviews, we heard participants say a variation on the statement, ‘I can teach someone to wireframe, but I can’t teach them how to be passionate about UX.’ This perspective implied that there are some people who are just better suited based on some innate qualities to be in a UX career.” If the sentiment of this is important enough to be one of only two findings, then where is the data evidence above, and why isn’t it featured in any depth?

The repeated paragraph used for both curiosity and disposition also raises a question for me—was this peer reviewed, or even read by the accepting conference? People make mistakes, and it’s unexpected but reasonable that the authors may have missed this. But how did everyone else miss it? I saw it right away, and I’m not reading this at the level of rigor I would expect from a reviewer. [The group that accepted this, called SIGDOC (ACM’s Special Interest Group on Design of Communication), holds a yearly conference; the conference proceedings are “double-blind peer-reviewed.”]

I guess my own takeaways are—there are people researching this topic, which is great; the codebook/theming/whatever approach is suspect, at least to me (not in this paper specifically—as a method that's broadly used and accepted); findings need to map to data; and what it takes to get something published is different than I expected.