Phase 2 of the Engineering CAReS Study

 by Denise Wilson and Jennifer VanAntwerp, February 8, 2025

Much of what human beings do is done in the service of belongingness.

(Baumeister & Leary, 1995)

A square peg in a round hole. 

Most of us have felt like that square peg at some point. In a college class. In a high school club. Meeting up again with that group of friends who has stayed close over the past years while you have drifted away. The oldest person in the room. The youngest. We tried to enter that space but discovered that we were not going to make it in through that round hole unless someone whacked us very hard on top of our square peg head to cram us inside. If we do make it inside, we will be a bit bruised – or maybe even find that some of our edges have been shaved off.  And so, we debate…how much do we really want to be in that room? Is it worth it?  

Engineering can be one of these round hole places. But we want to see a greater diversity of people working in engineering – diverse in thinking patterns, leadership styles, vision, perspective.  True diversity of thought and perspective is not only good for employees and individuals.  It's good for business and global competitiveness.  

To support more productive and more nourishing engineering workplaces in the future, we want to know more about how that sense of belonging comes alongside feelings of being competent and having some say-so in our daily work lives (autonomy). Hence the name of our study, Engineering CAReS: Engineering Competence Autonomy Relatedness (a.k.a. Belonging) Study.

The U.S. has spent decades trying to increase the diversity of people in engineering. Of course, most of those efforts focus on the pegs - how engineering can do a better job of shaving those square pegs into round ones. Our research team wants to flip that question around. How can we carve out engineering fields so that they form a new shape, one that allows more people to enter – and to stay – without needing to change their own shape?

We think this question is important. Every person fundamentally wants to feel that they belong. Belonging matters. Psychology researchers report a laundry list of problems linked to unmet needs for belonging. 

So, then, what about engineering? Do engineers feel a sense of belonging within their workplaces? Do those that work with engineers enjoy a sense of belonging as well? What aspects of the workplace best support – or hinder – an engineer’s ability to feel like they belong?

We are excited to be in the second phase of our Engineering CAReS study. With a shorter and more focused survey emerging from Phase 1 of our study, we hope to gather a truly broad range of perspectives from those who are engineers or computer scientists and those who work with them, whether in corporate, government, academic, or nonprofit positions.   

Thanks for joining us in this journey to a new, and better, engineering future.

Here is the link to your Phase 2 online research survey:  Engineering CAReS Survey

The Elusive Mere Belonging

Gregory Walton and Geoffrey Cohen, researchers at Stanford University, have conducted a wide range of controlled experiments on students to study the effect of "mere belonging" on student persistence, task performance, GPA, etc.   They found that little things that established a little bit of social connectedness among people (like sharing a birthday or some other small thing in common with at least one other member of the team) increased the motivation and persistence that team members showed on a shared task.  

In simple terms:

Create a little bit of social connection, and Voila!  

Along comes better performance.

Walton and Cohen's work resonates with me, because I'm very familiar with how little it takes to help me feel more comfortable and motivated in a meeting, whether it be at work, at a social gathering, at church, etc.  In a work meeting, the words "Like Denise, I .... " or "I am on the same page as Denise." or "I also teach in that style." are music to my ears. The words don't necessarily have to be positive or affirming. They just have to provide, at a minimum, some temporary relief from feeling like the alien in the room.   

Especially in a job that requires a lot of teaching and leading, it is easy to go without "mere belonging" for days, if not weeks at a time.  In most of our meetings, we are expected to be the leaders in the room, if not the experts.  Students often believe that we are so old we couldn't possibly relate to their experiences and while wanting the same sense of mere belonging that we do, they often remain firmly convinced that such belonging can and never will come from something as ancient and disconnected as a tenured professor -- I also believe students think that when such tenured professors are not in class, we are locked up in a closet until such time that we are released once again into the blessed intellectual space that is the college classroom.  After all, if professors had a life outside of class, they would not be spending sufficient time on how, what, and how much to teach.   During the semester, these matters of teaching must be the central and only focus of our lives.

But I digress.

In our research meetings, we are often the only experts in the room in the area of expertise that prompted our contribution to and collaboration with the project at hand. By sorrowful definition, then, we often share that expertise in common with no other researchers in the room and mere belonging remains elusive once again.  Student researchers may share a common interest in our area of expertise, but the authority or experience barrier often prevents mere belonging from germinating between us. 

And so it goes, in higher education.   

Many meetings. Many interactions.  Much time spent with others.  

But, too often, strangely lonely.  

Mere Belonging remains elusive.

Interested in helping out? Complete our short survey on Workplace Belonging.  

Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the engineering classroom.   

How we compare ourselves to Others

by Denise Wilson, March 6, 2024

Often when researchers are trying to study how one group of people behaves compared to another group or groups, we code our data to support a wide group of statistical techniques called regression analyses. Regression simply tries to come up with a mathematical relationship (typically linear) between input (independent) variables and an output (dependent) variable.

Without getting into the math of it all, the hands-down, most popular way to code individuals by their demographic characteristics is a process called dummy coding... which BTW does not imply that anyone is a dummy.  

Dummy coding works by identifying a reference group and then giving everyone who doesn't belong to that reference group a label and a category of their own.  For example, in a population of students who are White, Asian, Black, Multiracial, or of "other" races, we might choose the reference group to be White people.  To find a place for all races in the statistical analysis, we could then dummy code the five categories of race into four variables:

• Asian:  this variable would code all White students as "0" and all Asian students as "1"
• Black:  this variable would code all White students as "0" and all Black students as "1"
• Multiracial:  this variable would code all White students as "0" and all Multiracial students as "1"
• Other:  this variable would code all White students as "0" and "other" race students as "1"

Dummy coding, whether intended or not, inherently implies that the reference group is "normal" and explores whether there is something not normal about the remaining racial groups.  Results in studies that use dummy coding often sound like: "Asian students experienced less belonging than White students," or "Black students had higher test scores than White students," and so on. Dummy coding, intentionally or not, often sets us up to aspire to what White people do.   

Effect coding, on the other hand, works similarly to dummy coding in that it codes demographic data into integer numbers, but unlike dummy coding, it does so in a way that compares each group to the grand mean (the unweighted average of the outcome variable among all groups). In plain English, this means that effect coding allows us to compare results to the norm across the entire population rather than to a particular reference group. This leads to statements like "Asian students experienced less belonging than was the norm in the larger student population in this study," or "Black students had higher post-test scores than was the norm among all students enrolled in the course." Using the same example as for dummy coding of race, effect coding would also code five categories of race into four variables, but a little bit differently than for dummy coding:

• Asian:  this variable would code all White students as "-1", all Asian students as "1", and all non-White and non-Asian students as "0"
• Black:  this variable would code all White students as "-1",  all Black students as "1", and all non-White and non-Black students as "0"
• Multiracial:  this variable would code all White students as "-1", all multiracial students as "1", and all non-White, non-Multiracial students as "0"
• Other:  this variable would code all White students as "-1", all "other" race students as "1",  and all Black, Asian, and Multiracial students as "0"

Without getting into the math of it all, the above (effect-coded) approach to coding demographic data allows us to refrain from judging what is normal and to simply compare what certain groups are feeling or doing to the average across the whole sample population we are studying. Reaching the norm or (unweighted) average may still not be the ultimate goal, but it prevents us from devising strategies or designing interventions whose goal is to get everyone acting like White people.  

Reference:

Mayhew, M. J., & Simonoff, J. S. (2015). Non-White, no more: Effect coding as an alternative to dummy coding with implications for higher education researchers. Journal of College Student Development, 56(2), 170-175.

UCLA Advanced Research Computing:  Statistical Methods and Data Analytics.  Coding systems for categorical variables in regression analysis

UCLA Advanced Research Computing:  Statistical Methods and Data Analytics.  Interpreting the coefficients of an effect-coded variable in a regression model.  

Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the engineering classroom.  She is also invested in engineering workplace research focused on understanding belonging and inclusivity.     

The Messy Measurement of Belonging

While most agree that belonging is a fundamental human need, few agree on one reliable and accurate way to measure belonging.  Belonging has been measured as a sense of community, a feeling of connectedness to others, a sense of being accepted, valued, or included by others, perceptions of social support, feelings of being respected, a sense that one's presence in a group matters, and on and on.  Belonging has also been measured directly using the word belonging in survey items. 

Of the many different approaches to measuring belonging, one thing is clear -- the words we use to drill into what it means to belong in any organization or setting -- are imperfect at best.

The Engineering CAReS study has not set out to identify which of these measures is the best, the most accurate, or the most applicable to engineering, but instead to ensure that items we use to measure belonging are conceptually related to common themes among these different definitions of belonging. While statistically exploring which survey items from the CAReS study are suitable for understanding belonging and which are not may seem like an necessary but mundane exercise in data analysis, interesting insights can emerge from such exploratory factor analysis. 

In Phase 1 (our tool development phase of Engineering CAReS), we used the following survey items to measure belonging:

While these items seem to cover many of the ways that belonging has been defined and measured (in previous research studies) and seem like they should come together as a single measure, they do not -- at least when engineers and computer scientists are reporting their experiences about their jobs.   Instead, only three of the above items seem to capture belonging:

In our Phase 1 analysis, sense of community within an organization is distinctly different from sense of belonging.  While sense of belonging may influence sense of community or vice versa, they remain distinctly different measures.   The items that measure sense of community are:

Feeling supported (Item #1) and perceiving that people in an organization are friendly (Item #5) had significant cross-loadings in our Phase 1 analysis. This means that both of these items are measuring more than thing and couldn't be used in future surveys.   

So, what's the big deal here?  

Well, for one, no matter what we label it, feelings of being accepted and comfortable at work are different from feeling a sense of togetherness and community at the organizational level.  While they may be correlated to one another, they are distinctly different.   

Put another way, while an organization may draw its employees together toward a common goal and put together regular evens to bring employees together, that does not necessarily mean that all employees will develop a sense of belonging as a result.  

Have you ever gone to a company party that is well attended and still feel like a fish out of water? Our data says that's a perfectly normal possibility.  

Great Progress in Research Study Phase 1!

Thanks for your interest in the progress of the Engineering CAReS research study! We are trying to find out what the technical workplace feels like to those who work there, so that we can provide solutions to make it better for everyone.  If you are curious about this project, check out this early blog post for an introduction.

      Status Update 

Completed surveys continue to come in, and we are excited to be approaching our goal for Phase 1.  

Some Exciting Early Results...

If you have been keeping track, you might notice that our target has changed from 360 to 200 surveys for Phase 1. What's going on?

Well, it is very good news! We have preliminary results from the 155 completed surveys that indicate we can complete the necessary Phase 1 statistical analysis with fewer surveys. Why is that? Because the data is very good! What do we mean by that?

Well, we are learning that the questions you answered on the survey are very self-consistent. This means that we will be able to remove many of the questions on the Phase 1 survey, and still get reliable and valid results. 

Are you wondering how this works? Well, when researchers study people, we often want to measure an abstract concept, such as a person's "sense of belonging" in their workplace, or how much a person feels that their workplace supports (or hinders) their professional development. We do this by constructing measurement scales, which are a list of questions that each person who completes the questions seems to always answer in a similar way. We try to reduce an overall survey to the minimum number of questions that can reliably measure a particular concept of interest.

For example, suppose we want to know an individual's tendency to like eating sweet foods. We might, in Phase 1, ask these questions:

For each food listed below, indicate how often you would choose to eat it if you could (Answer 1 = never or not at all, 2 = once in a while, 3 = sometimes, 4 = often, 5 = any chance I get):

1. Ice cream
2. Birthday cake
3. Chocolate chip cookies
4. Hard candy
5. Chocolate truffles
6. Marshmallows
7. Apple pie
8. Pecan pie
9. Jello
10. Banana bread

If we have many people answer this question, we are looking for a set of questions that any one individual will tend to answer not the same, but similarly.  If we find that the answer to "jello" has no correlation to the answer for other foods, then we would eliminate it on future surveys, because it does not seem to measure the same concept of "like to eat sweet foods." If any one person will give a similar number answer to all the other 9 foods, then we have some confidence that they all measure the same thing. In that case, we could ask fewer questions. If we only asked about ice cream, birthday cake, and chocolate truffles, we would still have a good estimate of how much that one person likes sweet foods. Now, we have a much shorter survey!

And so, back to our survey about the engineering workplace. We have a number of different ideas or concepts that we want to measure. The Phase 1 survey has multiple questions about each concept. From the preliminary data of the first few hundred people, we are starting to find questions that we can eliminate without losing information. 

This Phase 1 "Tool Development" allows both us and other researchers to better understand both engineers themselves and their experiences in the technical workplace. Once we know how to measure the important concepts, then we can collect data from many more people, and then confidently build models to understand what factors are significant to the workplace experience. It will also allow us to identify which groups of people are thriving - or not - as engineers or computer scientists (including those who work closely with them).

Uh-Oh: As a teacher, am I doing more harm than good?

by Denise Wilson, July 6, 2022

It's a teacher's worst nightmare, right? You see a student in your class -- looking a little isolated. You want to help. You reach out. You try to connect. You want the student to feel included, accepted, and welcome in your class. You reach out again. Offer help. And then, the unthinkable happens. Good intentions lead to bad outcomes. Instead of feeling more included, the student feels even more left out. Isolated. Unwelcome.   

Unfortunately, this may not be "just a nightmare" but rather is grounded in reality. In our recent six-year study of engineering undergraduates, the data suggests that this fear may actually ring true. For those students that we as teachers may see as isolated or less integrated with peers, more interactions with the teacher go hand-in-hand with a lower sense of belonging -- more isolation, less sense of acceptance -- exactly the opposite of what most faculty want when they work with and interact with students. This is scary. 

To be fair, our data is cross-sectional -- taken only at a single point in time. But to add fuel to the fire, we do know that, more so than K-12 teachers, college faculty are intimidating to students -- even scary. Going to office hours may be the last thing that many students want to do. Interact with the Professor? No way.  And so, it is indeed possible that although correlation does not prove causality, some, many, or most interactions with faculty may actually be impairing students' sense of belonging -- pushing students out rather than drawing them in.   

Fortunately, we have some good news from that global bearer of bad news over the past several years -- the COVID-19 pandemic. Virtual office hours reduce the barrier between faculty and student. This effect is similar to the way that holding office hours in a student space (rather than in the professor's office) can reduce that barrier. Interactions held in spaces that are more comfortable, familiar, and overall safer for students can counteract the intimidation effect that those with the three letters PhD after their last name seem to unintentionally have on students. Students who talk to professors in more familiar spaces feel less like imposters, are more likely to ask frequent questions, and are likely to see faculty as more approachable.

This is good news for our worst nightmare.  Faculty can overcome the potentially negative impacts on student belonging that the mere act of interacting with students can invoke -- by making simple changes in how, when, and where interactions with students take place.

Whew.  Time to spruce up my home office for this evening's virtual office hours.     

Reference:

Misra, S., Kardam, N., VanAntwerp, J., and Wilson, D.M. (2022). How Did the Landscape of Student Belonging Shift During COVID-19? Journal of Engineering Education, in review.  

Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the engineering classroom.   

Building Belonging and Inclusion through Effective Performance Reviews

by Jennifer VanAntwerp, June 12, 2022

Does your employer include you in the performance review process? Does the way in which your performance is evaluated, regardless of the "rating", make you feel a part of the organization?  Or does it leave you feeling left out?  

If your feelings of belonging at work seem tied to how your performance is evaluated, regardless of whether you are performing well or poorly, you are not alone.  Employees who are part of a structured evaluation system that includes effective management of each employee's performance and clear communication are over 1.4 times more likely to feel a sense of inclusion at work compared to those who are subject to less transparent performance evaluations.   

Recently, one of my colleagues, an engineering manager, related the performance review process that he was expected to implement at work. Every time performance reviews came around, he was expected to rank all his team members in such a way that their "performance" conformed to a nice, neat bell-curve, zero sum game. 

Image Source:  psychology.org

Despite the fact that the number of people in my workgroup were far below the typical sample size that one would expect for a reliable normal distribution and there was no reason to think they were a random representation of engineers, someone in upper management had gotten hooked on thinking that all workgroups could be molded into a bell curve.  The above and below average performers would have to balance within each work unit, regardless of what the actual distribution of their performance looked like. 

During one review cycle, this method of performance evaluation seemed especially disrespectful of his team, prompting my colleague to approach upper management, pleading for leniency: 

"This is the best team of people I have ever worked with. They are each truly exceptional individuals, and they are truly exceptional in how they work together synergistically to achieve even more. I think they all deserve an above average rating this year."

Despite pleading for a more humane (and accurate) performance review approach, his request was refused. This manager was forced to choose which star employees to disappoint, demoralize, even antagonize. He had to create unhealthy competition among a team that had previously been working well to support each other.   After he completed the reviews, the morale in his workgroup declined and attrition soon followed.   

Certainly, a business has to find ways to allocate resources, incentives, and rewards appropriately. But a zero-sum attitude might in fact lead to a less-than-zero result. Which leads to the question...are there better ways to include employees in their performance evaluations? Let's consider this through the recent events of tech giant Google.

Google has had its share of bad press as an employer in the last few years. In 2018, more than 20,000 employees held a walk-out to protest the inadequate response of upper management to discrimination, racism, and sexual harassment. Employees were not only unhappy about the specific incidences of quiet and cushy exit packages for those who had harassed, but also because of what employees believed to be an inadequate reporting system for offenses. Employees were not universally satisfied with Google response, but it appears that the executive leadership is learning. Since these protests, Google CEO Sundar Pichai expressed in a recent interview that he understands the strong employee voice at Google to be an important asset:

"You trust your employees to get it right at scale.... So I view it as a strength of the company when employees speak up. I think it's important for us to take it seriously."

Technology columnist Jason Aten applauds Google for a recent change that seems to indicate Google is walking this talk: how Google reviews its employees. In the old process, employee performance reviewed happened twice a year, and each individual review involved hours of work by multiple personnel. More significantly, over half of Google employees felt that these reviews were mostly a waste of time. In other words, all of the time invested was even more frustrating because it did not seem to be adding any value. Going forward, those reviews will be cut in half, which already reduces the time investment. Taking the concerns of employees seriously benefits everyone.

But perhaps even more significantly, the rating process has been revamped to reflect a different point of view about its employees: 

As Google now tells its employees: 

Our new rating scale will reflect the fact that most Googlers deliver significant impact every day.

Think of the (unfortunate) novelty of this concept. The employer allows for the possibility that every team member is making important and distinct contributions. They don't force their team leaders to smash a small group of employees into an artificial bell curve. 

Or as Jason Aten put it: 

What is your workplace like? Do you find yourself forced into a competitive relationship with the very teammates who should be your best supporters (and vice versa)? Or does your workplace encourage the idea that a rising tide lifts all boats? Does your empower everyone to both develop and work from their strengths while simultaneously having opportunities to grow new skills? 

Jennifer VanAntwerp is a professor of chemical engineering at Calvin University in Grand Rapids, Michigan. She researches how engineers learn, work, and thrive, beginning in college and extending throughout their professional careers