Monday, October 17, 2022

The Messy Measurement of Belonging

While most agree that belonging is a fundamental human need, few agree on one reliable and accurate way to measure belonging.  Belonging has been measured as a sense of community, a feeling of connectedness to others, a sense of being accepted, valued, or included by others, perceptions of social support, feelings of being respected, a sense that one's presence in a group matters, and on and on.  Belonging has also been measured directly using the word belonging in survey items. 

Of the many different approaches to measuring belonging, one thing is clear -- the words we use to drill into what it means to belong in any organization or setting -- are imperfect at best.

The Engineering CAReS study has not set out to identify which of these measures is the best, the most accurate, or the most applicable to engineering, but instead to ensure that items we use to measure belonging are conceptually related to common themes among these different definitions of belonging. While statistically exploring which survey items from the CAReS study are suitable for understanding belonging and which are not may seem like an necessary but mundane exercise in data analysis, interesting insights can emerge from such exploratory factor analysis. 

In Phase 1 (our tool development phase of Engineering CAReS), we used the following survey items to measure belonging:

While these items seem to cover many of the ways that belonging has been defined and measured (in previous research studies) and seem like they should come together as a single measure, they do not -- at least when engineers and computer scientists are reporting their experiences about their jobs.   Instead, only three of the above items seem to capture belonging:

In our Phase 1 analysis, sense of community within an organization is distinctly different from sense of belonging.  While sense of belonging may influence sense of community or vice versa, they remain distinctly different measures.   The items that measure sense of community are:

Feeling supported (Item #1) and perceiving that people in an organization are friendly (Item #5) had significant cross-loadings in our Phase 1 analysis. This means that both of these items are measuring more than thing and couldn't be used in future surveys.   

So, what's the big deal here?  

Well, for one, no matter what we label it, feelings of being accepted and comfortable at work are different from feeling a sense of togetherness and community at the organizational level.  While they may be correlated to one another, they are distinctly different.   

Put another way, while an organization may draw its employees together toward a common goal and put together regular evens to bring employees together, that does not necessarily mean that all employees will develop a sense of belonging as a result.  

Have you ever gone to a company party that is well attended and still feel like a fish out of water? Our data says that's a perfectly normal possibility.  





Tuesday, August 9, 2022

Great Progress in Research Study Phase 1!


Thanks for your interest in the progress of the Engineering CAReS research study! We are trying to find out what the technical workplace feels like to those who work there, so that we can provide solutions to make it better for everyone.  If you are curious about this project, check out this early blog post for an introduction.

      Status Update 

Completed surveys continue to come in, and we are excited to be approaching our goal for Phase 1.  If you have already completed a survey, then thank you! If not, would you consider joining us? The survey can now be found at: 

https://forms.office.com/r/X12q7D3dH5

The survey must be completed before you close your browser.


Some Exciting Early Results...

If you have been keeping track, you might notice that our target has changed from 360 to 200 surveys for Phase 1. What's going on?

Well, it is very good news! We have preliminary results from the 155 completed surveys that indicate we can complete the necessary Phase 1 statistical analysis with fewer surveys. Why is that? Because the data is very good! What do we mean by that?

Well, we are learning that the questions you answered on the survey are very self-consistent. This means that we will be able to remove many of the questions on the Phase 1 survey, and still get reliable and valid results. 

Are you wondering how this works? Well, when researchers study people, we often want to measure an abstract concept, such as a person's "sense of belonging" in their workplace, or how much a person feels that their workplace supports (or hinders) their professional development. We do this by constructing measurement scales, which are a list of questions that each person who completes the questions seems to always answer in a similar way. We try to reduce an overall survey to the minimum number of questions that can reliably measure a particular concept of interest.

For example, suppose we want to know an individual's tendency to like eating sweet foods. We might, in Phase 1, ask these questions:

For each food listed below, indicate how often you would choose to eat it if you could (Answer 1 = never or not at all, 2 = once in a while, 3 = sometimes, 4 = often, 5 = any chance I get):
  1. Ice cream
  2. Birthday cake
  3. Chocolate chip cookies
  4. Hard candy
  5. Chocolate truffles
  6. Marshmallows
  7. Apple pie
  8. Pecan pie
  9. Jello
  10. Banana bread
If we have many people answer this question, we are looking for a set of questions that any one individual will tend to answer not the same, but similarly.  If we find that the answer to "jello" has no correlation to the answer for other foods, then we would eliminate it on future surveys, because it does not seem to measure the same concept of "like to eat sweet foods." If any one person will give a similar number answer to all the other 9 foods, then we have some confidence that they all measure the same thing. In that case, we could ask fewer questions. If we only asked about ice cream, birthday cake, and chocolate truffles, we would still have a good estimate of how much that one person likes sweet foods. Now, we have a much shorter survey!

And so, back to our survey about the engineering workplace. We have a number of different ideas or concepts that we want to measure. The Phase 1 survey has multiple questions about each concept. From the preliminary data of the first few hundred people, we are starting to find questions that we can eliminate without losing information. 

This Phase 1 "Tool Development" allows both us and other researchers to better understand both engineers themselves and their experiences in the technical workplace. Once we know how to measure the important concepts, then we can collect data from many more people, and then confidently build models to understand what factors are significant to the workplace experience. It will also allow us to identify which groups of people are thriving - or not - as engineers or computer scientists (including those who work closely with them).

Wednesday, July 6, 2022

Uh-Oh: As a teacher, am I doing more harm than good?

by Denise Wilson, July 6, 2022

It's a teacher's worst nightmare, right? You see a student in your class -- looking a little isolated. You want to help. You reach out. You try to connect. You want the student to feel included, accepted, and welcome in your class. You reach out again. Offer help. And then, the unthinkable happens. Good intentions lead to bad outcomes. Instead of feeling more included, the student feels even more left out. Isolated. Unwelcome.   

Unfortunately, this may not be "just a nightmare" but rather is grounded in reality. In our recent six-year study of engineering undergraduates, the data suggests that this fear may actually ring true. For those students that we as teachers may see as isolated or less integrated with peers, more interactions with the teacher go hand-in-hand with a lower sense of belonging -- more isolation, less sense of acceptance -- exactly the opposite of what most faculty want when they work with and interact with students. This is scary. 



To be fair, our data is cross-sectional -- taken only at a single point in time. But to add fuel to the fire, we do know that, more so than K-12 teachers, college faculty are intimidating to students -- even scary. Going to office hours may be the last thing that many students want to do. Interact with the Professor? No way.  And so, it is indeed possible that although correlation does not prove causality, some, many, or most interactions with faculty may actually be impairing students' sense of belonging -- pushing students out rather than drawing them in.   

Fortunately, we have some good news from that global bearer of bad news over the past several years -- the COVID-19 pandemic. Virtual office hours reduce the barrier between faculty and student. This effect is similar to the way that holding office hours in a student space (rather than in the professor's office) can reduce that barrier. Interactions held in spaces that are more comfortable, familiar, and overall safer for students can counteract the intimidation effect that those with the three letters PhD after their last name seem to unintentionally have on students. Students who talk to professors in more familiar spaces feel less like imposters, are more likely to ask frequent questions, and are likely to see faculty as more approachable.

This is good news for our worst nightmare.  Faculty can overcome the potentially negative impacts on student belonging that the mere act of interacting with students can invoke -- by making simple changes in how, when, and where interactions with students take place.

Whew.  Time to spruce up my home office for this evening's virtual office hours.     

Reference:

Misra, S., Kardam, N., VanAntwerp, J., and Wilson, D.M. (2022). How Did the Landscape of Student Belonging Shift During COVID-19? Journal of Engineering Education, in review.  


Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the engineering classroom.   


Sunday, June 12, 2022

Building Belonging and Inclusion through Effective Performance Reviews

by Jennifer VanAntwerp, June 12, 2022

Does your employer include you in the performance review process? Does the way in which your performance is evaluated, regardless of the "rating", make you feel a part of the organization?  Or does it leave you feeling left out?  

If your feelings of belonging at work seem tied to how your performance is evaluated, regardless of whether you are performing well or poorly, you are not alone.  Employees who are part of a structured evaluation system that includes effective management of each employee's performance and clear communication are over 1.4 times more likely to feel a sense of inclusion at work compared to those who are subject to less transparent performance evaluations.   

Recently, one of my colleagues, an engineering manager, related the performance review process that he was expected to implement at work. Every time performance reviews came around, he was expected to rank all his team members in such a way that their "performance" conformed to a nice, neat bell-curve, zero sum game. 

Image Source:  psychology.org

Despite the fact that the number of people in my workgroup were far below the typical sample size that one would expect for a reliable normal distribution and there was no reason to think they were a random representation of engineers, someone in upper management had gotten hooked on thinking that all workgroups could be molded into a bell curve.  The above and below average performers would have to balance within each work unit, regardless of what the actual distribution of their performance looked like. 

During one review cycle, this method of performance evaluation seemed especially disrespectful of his team, prompting my colleague to approach upper management, pleading for leniency: 

"This is the best team of people I have ever worked with. They are each truly exceptional individuals, and they are truly exceptional in how they work together synergistically to achieve even more. I think they all deserve an above average rating this year."

Despite pleading for a more humane (and accurate) performance review approach, his request was refused. This manager was forced to choose which star employees to disappoint, demoralize, even antagonize. He had to create unhealthy competition among a team that had previously been working well to support each other.   After he completed the reviews, the morale in his workgroup declined and attrition soon followed.   

Certainly, a business has to find ways to allocate resources, incentives, and rewards appropriately. But a zero-sum attitude might in fact lead to a less-than-zero result. Which leads to the question...are there better ways to include employees in their performance evaluations? Let's consider this through the recent events of tech giant Google.

Google has had its share of bad press as an employer in the last few years. In 2018, more than 20,000 employees held a walk-out to protest the inadequate response of upper management to discrimination, racism, and sexual harassment. Employees were not only unhappy about the specific incidences of quiet and cushy exit packages for those who had harassed, but also because of what employees believed to be an inadequate reporting system for offenses. Employees were not universally satisfied with Google response, but it appears that the executive leadership is learning. Since these protests, Google CEO Sundar Pichai expressed in a recent interview that he understands the strong employee voice at Google to be an important asset:

"You trust your employees to get it right at scale.... So I view it as a strength of the company when employees speak up. I think it's important for us to take it seriously."

Technology columnist Jason Aten applauds Google for a recent change that seems to indicate Google is walking this talk: how Google reviews its employees. In the old process, employee performance reviewed happened twice a year, and each individual review involved hours of work by multiple personnel. More significantly, over half of Google employees felt that these reviews were mostly a waste of time. In other words, all of the time invested was even more frustrating because it did not seem to be adding any value. Going forward, those reviews will be cut in half, which already reduces the time investment. Taking the concerns of employees seriously benefits everyone.

But perhaps even more significantly, the rating process has been revamped to reflect a different point of view about its employees: 


As Google now tells its employees

Our new rating scale will reflect the fact that most Googlers deliver significant impact every day.

Think of the (unfortunate) novelty of this concept. The employer allows for the possibility that every team member is making important and distinct contributions. They don't force their team leaders to smash a small group of employees into an artificial bell curve. 

Or as Jason Aten put it



What is your workplace like? Do you find yourself forced into a competitive relationship with the very teammates who should be your best supporters (and vice versa)? Or does your workplace encourage the idea that a rising tide lifts all boats? Does your empower everyone to both develop and work from their strengths while simultaneously having opportunities to grow new skills? 



Jennifer VanAntwerp is a professor of chemical engineering at Calvin University in Grand Rapids, Michigan. She researches how engineers learn, work, and thrive, beginning in college and extending throughout their professional careers

Friday, May 13, 2022

Are Women Engineers Left Out?

Does feeling like you "belong" in your workplace matter? If you feel left out, shouldn't you just be able to tough it out? Too often, a sense of belonging is regarded as a luxury or a bonus to an otherwise acceptable workplace and culture. Unfortunately, belonging is not a luxury; it is not a want; it is, in fact, a need that all human beings have and seek to fulfill.  And, to that end, a lack of belonging has been proven to hurt work satisfaction, well-being, career advancement, and myriad other outcomes that define a successful and fulfilling career.    

In the big picture, not just as reflected by anecdote but as proven in social science research, women are often left out of engineering work and left without opportunities to belong. And many women face intersectional factors (e.g., women of color, young women, lesbian women) that make it even more difficult to belong.  A recent review of the literature on belonging has shown that a majority of research studies on the subject have demonstrated that the need to belong often goes unmet for women in engineering work. Consistent, positive interactions with coworkers and a relational space that is safe and stable is required to develop the reliable social bonds that are a hallmark of a sense of belonging. Unfortunately, these are more frequently lacking for women engineers. Instead, many women engineers find the workplace to be a place where they experience isolation, are not valued, and do not feel free to be themselves. 

Six out of seven engineering workplace studies of belonging reported that isolation or lack of belonging were major concerns among the female engineers. For example, according to a multinational survey of over 4,400 professionals conducted in the early 2000's, 44% of female engineers feel extreme isolation in their workplaces. More recently, a qualitative 2016 study showed that women continue to report feelings of isolation, and much more often than men. Among these workplace studies, the only one that did find belongingness among engineering women intentionally studied only women who had happily persisted in civil engineering work into mid-career. And, in this one study, the researchers inferred that those women who did not experience belonging had already left engineering!  


Women engineers often experience a tension between being seen as a woman or an engineer.  The fact that so many engineering workplaces are highly male dominated means that engineering work has come to be defined, conducted, and perceived as masculine.  This leaves women constantly negotiating their own identity and struggling to be recognized as engineers while simultaneously sticking out as the only one or one of only a few women in their workgroups. This in/visibility paradox often creates a space where women engineers rarely feel that they can simply be themselves, which all but guarantees that belonging needs will never be fully met in engineering work.   

The current #MeToo era adds its own wrinkles. Obviously, eliminating sexual harassment is an absolutely critical step in allowing everyone to feel belonging. However, this is a necessary but not sufficient step - and it must be done thoughtfully to avoid causing even more trouble. As harassment training continues to proliferate in the workplace, we are (thankfully!) likely to see a continued decrease in egregious or overt acts of sexual and gender harassment. Yet, in the process of "being certain not to harass," male engineers may end up withdrawing from appropriate and necessary interactions with women - further compounding the problem of isolation and lack of belonging. This calls for a need for trainers and trainees alike to be aware of the unanticipated consequences that may emerge from sexual harassment training. Further, it calls for, at the very least, raising awareness at the local workgroup level of what isolation and lack of belonging looks like. To go a step further, adding additional, well-designed, practical training to the organizational toolbox on how to support belonging for a diverse workgroup would go a long way to help women working in male dominated engineering fields.  

Interested in the belonging conversation? Follow our blog, Belonging in Engineering, where you can also learn about and keep up with the progress of Engineering CAReS, a research study of the climate and culture of engineering and computer science workplaces. While you're there, please consider clicking the link to complete the online survey yourself and be a part of the study!

More about Belonging:

Cornell University Diversity and Inclusion (2022). Sense of belonging

Huang, Steven (2020). Why does belonging matter at work? Society for Human Resource Management (SHRM). 

University of Washington. Understanding and evaluating belonging in higher education

References:

Ayre, M., Mills, J., & Gill, J. (2013). ‘Yes, I do belong’: the women who stay in engineering. Engineering studies, 5(3), 216-232.

Faulkner, W. (2011). Gender (in) authenticity, belonging and identity work in engineering. Brussels economic review, 54(2/3), 277-293.

Hewlett, S. A., Luce, C. B., Servon, L. J., Sherbin, L., Shiller, P., Sosnovich, E., & Sumberg, K. (2008). The Athena factor: Reversing the brain drain in science, engineering, and technology. Harvard Business Review Research Report, 10094, 1-100.

Wilson, D., & VanAntwerp, J. (2021). Left Out: A review of women’s struggle to develop a sense of belonging in engineering. SAGE Open, 11(3), 21582440211040791.

Yonemura, R., & Wilson, D. (2016, June). Exploring barriers in the engineering workplace: Hostile, unsupportive, and otherwise chilly conditions. In 2016 ASEE Annual Conference & Exposition.


Tuesday, May 3, 2022

Leadership for Building Belonging

by Jennifer VanAntwerp, May 1, 2022

There are piles of evidence that an organization benefits when its members feel a strong sense of belonging. These employees are more creative, more productive, more engaged and involved, happier, and likely to stick around with the organization longer.

As leadership consultant Alexis Zahner puts it

"IMAGINE showing up to work everyday and not needing to defend your worth because you're the 'only' person like you at the table." 

It doesn't take much, then, to imagine what it might feel like to show up to work everyday if this were not the case.

So how does this culture happen? It isn't by accident. An inclusive culture where everyone, including the organization, thrives, requires the members to embrace it. And that is much more likely to happen if the leadership, at every level, actively seeks and supports this. Again, from Alexis Zahner, 

"Actively creating a culture of inclusion and belonging by helping others to feel seen, heard and valued is a core competence of Human Leadership."

Wherever you fall in the organizational chart, you are a part of shaping the workplace culture. Next time you evaluate your professional goals, consider adding one goal related to improving the culture of belonging within your own corner of the organization.  Will you focus on listening more?  Withholding judgement?  Understanding your own biases? Helping others to collaborate more? Becoming more attentive to the culture of others? Restructuring meetings to give attendees equal opportunity to speak and be heard?  These are just a few of the proven techniques for supporting greater inclusivity and building belonging for everyone in the workplace. There are plenty of strategies out there - feel free top pick one below or customize one of your own to get started!

A few basic places to build your knowledge base...



Jennifer VanAntwerp is a professor of chemical engineering at Calvin University in Grand Rapids, Michigan. She researches how engineers learn, work, and thrive, beginning in college and extending throughout their professional careers

Sunday, May 1, 2022

Moving Forward with Phase 1

We are making steady progress toward the Phase 1 research study participation goal.


Phase 1 requires a few hundred people so that we can have results with statistical significance. So what will we do after we reach that goal?

First, we will determine which questions from the Phase 1 survey we can eliminate without losing information. When social scientists want to measure an abstract concept by asking the person involved, it is often necessary to ask the questions in several different ways. This rephrasing might make the concept clearer to the person being queried. Or the additional questions, when used together, might capture the concept more fully. Whichever the case, the job for statistics is to consider a series of related questions and determine which are the fewest ones that will reliably (and reproducibly) measure that abstract idea. 

Our Phase 1 survey is attempting to measure a number of abstract concepts. We may discover that some of these just aren't relevant to the issues of belonging in engineering workplaces. We will drop those altogether. We will likely discover that some of the abstract concepts can be measured with, say, only 2 or 3 questions instead of 8 or 9. For either case, the result will be a good thing: a much shorter survey for Phase 2.  

But the data from Phase 1 is still important in and of itself. With just a few hundred engineers sharing their experiences, we will also begin to analyze that data and report results that can be statistically supported as being meaningful.

That will be the beginning of the exciting part of the project! We can start to answer some of the questions that prompted our study. Who feels like they belong in the technial workplace? Who doesn't, and why? What is it about a workplace environment that makes it easier or harder to feel belonging? How common is it to (or isn't it) to feel a sense of belonging as a working engineer or computer scientist?

Want to be a part of it all? Take the survey!


https://forms.office.com/r/X12q7D3dH


The survey must be completed before you close your browser.










The Messy Measurement of Belonging

While most agree that belonging is a fundamental human need , few agree on one reliable and accurate way to measure belonging.  Belonging ha...