CAREER READINESS COMPETENCIES CAREER READINESS Development and Validation of the NACE Career Readiness Competencies
2 Table of Contents Introduction 3 Importance of Career Readiness 4 Definition of Career Readiness in Literature 5 Core skills Gap in the Literature 8 Theoretical Underpinnings of the NACE Career Readiness Competencies 9 Career readiness overall Career and self-development Communication Critical thinking Equity and inclusion Leadership Professionalism Teamwork Technology Methods and Results 15 Original task force work in 2015 Second task force work in 2017 Launch of current and ongoing career readiness project in 2018 Factor analysis #1 Third task force review, 2019-20 Factor analysis #2 Content validity survey Student perceptions of the revised competencies’ importance Discussion 22 Future research References 23 Appendix A: Full Set of Revised Definitions and Associated Behaviors 25 Appendix B: Full Set of Initial Competency Titles and Definitions 29 Appendix C: Factor Analysis Results #1 30 Appendix D: Factor Analysis Results #2 34 Appendix E: Content Validity Results 40 ©2022 National Association of Colleges and Employers. All rights reserved.
3 Introduction As the cost of higher education has risen, scrutiny on graduate outcomes has increased as well. Students, families, higher education administrators, and policy makers all want to understand the value of a college education. According to students and families, the reason most cited for enrolling in higher education is to improve the student’s job prospects (Gallup & Strada, 2017). As a result, students’ career readiness is a construct of utmost importance as it reflects the underlying workplace skills that have been presumably acquired through their college education. Unfortunately, as of 2018, only 43% of seniors felt prepared for their future careers (McGraw-Hill, 2018). Similarly, in a survey of 217 employers, 46% of respondents reported they had to provide remedial training to recent graduates to get them up to speed, and about one in five employers said graduates with four-year degrees needed what they referred to as workforce readiness training (Casner-Lotto, Rosenblum, & Wright, 2009). This state of affairs is quite costly for employers; expenditures of workforce training reached $82.5 billion in 2020 (Statista, 2021). On average, organizations spend $1,252 per employee to train and develop their workforces (ATD, 2016), and estimates indicate they are spending one-fifth of that on remedial training specifically for workforce readiness (Casner-Lotto, Rosenblum, & Wright, 2009). Promoting students’ career readiness, thus, is a pro-active, comprehensive and yet flexible approach to addressing this important issue for college students, their institutions, and their would-be employers. The purpose of this technical report is to share the theoretical and empirical underpinnings of the NACE Career Readiness Competencies. The following sections will review the literature, providing context and support for each competency, followed by the methods used and results obtained during their development and validation; the technical report concludes with a discussion of future directions.
4 Importance of Career Readiness Decrying the use of resources in providing what they considered remedial workforce readiness training, employers have expressed need for entry-level professional employees to integrate quickly into their respective organizations (Casner-Lotto, Rosenblum, & Wright, 2009). The need for a more focused form of readiness, one that is specific to college graduates entering the workforce, could not be more pressing. To guide this effort, NACE proposes the construct of Career Readiness as the solution. Individuals, Institutions of Higher Education, and employers would all benefit from having a shared understanding, vocabulary, and guideposts. At the national level, career readiness has been recognized as a critical developmental piece in securing an efficient economy. In 2014, the U.S. Department of Education’s Office of Career and Technical Education (OCTAE) released a fact sheet entitled Employability Skills: Supporting Opportunity Youth to Be Successful in Their First Job to help guide employers, state and federal agencies, as well as families and job seekers as they prepare to support and hire individuals who are career ready. Responding to the desires for enhanced college graduates’ preparation for joining the workforce, institutions of higher education around the country have been recognizing the importance of career readiness by instituting campus-wide initiatives. In fact, Virginia Commonwealth University (VCU) recently introduced a new minor degree granting program in their interdisciplinary studies program called “Career Readiness Skills” that is designed to help students navigate the workplace when they leave the university (Efetie & Oetjens, 2021).
5 Definition of Career Readiness in the Literature Career readiness, while defined and referred to somewhat variably in the literature, generally refers to the skills required to begin a career. According to Conley (2011), career readiness reflects the knowledge, skills, and learning strategies that are required to begin a career pathway, including common expectations about workplace conduct. Traditionally, particularly for the K-12 setting, that set of skills and knowledge has been defined in terms of math (reasoning), reading, and writing skills. Camara (2014) noted that “career readiness has not been defined as a measurable construct” (p. 21). Since then, Camara and colleagues have helped the assessment company ACT develop their College and Career Readiness suite of assessments referred to as WorkKeys. The assessment suite rests on four core areas: core academic skills, cross-cutting capabilities, behavioral skills, and education and career navigation skills (ACT, 2022). While most of the assessments in the WorkKeys suite assess traditional reading, writing, and arithmetic, the suite also measures soft, or what NACE calls core, skills through their set of competencies they have labeled as cross-cutting, invoking the transferable nature of these skills. WorkKeys also assesses behavioral and personality traits. Encompassing the range of academic, cognitive, and non-cognitive factors that affect one’s entry into the workforce, ACT has taken a wide approach to defining career readiness. OCTAE refers to the idea of career readiness as employability skills, which signals a wider audience than the college educated. As such, the office’s conception is broader than reading, writing, and arithmetic. OCTAE defines employability skills as a set of workplace skills (e.g., information use, systems thinking, technology), using applied knowledge (e.g., critical thinking, applied academic thinking), and having effective relationships (i.e., interpersonal skills and personal qualities). Based on OCTAE’s framework, ETS developed an assessment of career readiness, called HiSet, for adult learners entering the workforce without a high school degree (ETS, 2022). OCTAE’s framework is designed to address the entire spectrum of the national workforce, while NACE’s Career Readiness focuses on core, or transferable, skills for jobs that require at least a two- or four-year college degree. Though there is certainly overlap in the extant definitions, their variability stems from which skills are considered critical for beginning a career. Part of the reason for disagreement is due to different industries requiring different skills. For example, manufacturing jobs and sales jobs involve different kinds of activities and, as such, require somewhat different kinds of skills. Manufacturing positions may rely more on being able to follow instructions and read graphics for information, while a sales job may rely more on interpersonal skills, which are not usually measured by traditional reading, writing, and math assessments.
6 Developed in collaboration with experts in career development and talent attraction, the NACE Career Readiness Competencies are unique in that they focus exclusively on core, or transferable, skills. These eight competencies are applicable to career-oriented jobs for two- and four-year graduates, across the spectrum of industries, company types, and job level. Core skills Indeed, as our economy has shifted toward a service economy where information is paramount, interpersonal skills and other core, or transferable, skills have become of key importance. Demonstrating the importance of core skills, in a survey of 343 C-level executives conducted by The Economist Intelligence Unit, 72% of respondents cited critical thinking/problem-solving and 63% cited collaboration/teamwork as skills that are most important in their workplace. The first hard skill, technical skills associated with job, was tied in third, endorsed by 54% of respondents. Reading for information, ranked 8th, was endorsed by only 10% of respondents, and applied mathematics, ranked 11th, was endorsed by 5% of the executives in this sample (Labi, 2014). These results suggest that on average employers recognize higher value in core skills compared to the hard skills of reading, writing, and arithmetic. These results are very similar to those reported by the NACE in the 2019 Job Outlook survey of 172 employers of similar size and similar industries as in the survey by the Economist Intelligence Unit. Technical skills are only the 10th most endorsed skill; the top 9 were all core skills, e.g., communication, problem-solving, teamwork, initiative, work ethic. Based on these and other surveys conducted around the country, it is evident that core skills are in high demand. Core skills are defined as skills that can be transferred across jobs and include both skills and personal attributes. The skills and traits that are included vary by author, but there is much overlap across the definitions. They usually include interpersonal communication skills, critical thinking and problem-solving, an ability to work in teams or collaborate, and cultural awareness. For example, in a 2006 survey of 431 respondents that represented a workforce of over 2 million employees conducted in partnership by the Conference Board, Partnership for 21st Century Skills, Corporate Voices for Working Families, and Society for Human Resource Management, respondents indicated that core skills were more important in entry-level employees because job-specific hard skills can more easily be trained on-the-job. In this order, the soft skills that employers cited as most important were: oral communication, teamwork/collaboration, professionalism/work ethic, written communication, critical thinking/problem-solving, ethics/social responsibility,
7 leadership, information technology application, creativity/innovation, lifelong learning/ self-direction, and an ability to work with and learn from people representing diverse cultures. Similarly, the World Economic Forum (2015) conducted a literature review almost a decade later to synthesize the core skills that are needed in the 21st century. Their findings are very similar; they found that critical thinking/problem-solving, creativity, communication, collaboration, curiosity, initiative, persistence, adaptability, leadership, and social/cultural awareness were the key soft skills needed in the workplace. Given the decade in between these studies and that the results barely changed, NACE, in concert with the expertise of its members, concluded that focusing efforts on improving individuals’ core skills will be the most fruitful path forward in preparing college graduates to enter the workforce.
8 Gap in the Literature In terms of defining career readiness for college graduates entering the workforce, there are important gaps in the literature that necessitated the development of NACE Career Readiness Competencies for college graduates beginning their professional careers. Most of the career readiness literature focuses on college and career readiness, with emphasis on the college aspect for the K-12 students. For example, the US Department of Education provides guidance on employability skills for career and technical education as well as college and career readiness for high school students. Similarly, state governments and agencies that have adopted college and career readiness standards generally connect these educational goals with their offices for career and technical education as did the federal government with OCTAE. Moreover, the two dominant assessment companies in the space focus on the general workforce, which generally address the competencies (e.g., literacy, numeracy, and reasoning) needed for front-line jobs. While the same or related non-cognitive competencies generally remain relevant across the different segments of the workforce, more specific guidance can and should be provided for the college student who is preparing to enter the workforce post-graduation; in fact, this population is most in need of this guidance given its proximity to beginning their careers. To help facilitate the diffusion of career readiness as a construct across the nation, NACE launched its Career Readiness Initiative in 2015 to address the fundamental needs of new college graduates, including both the professionals who serve their career development needs and those who recruit them into the workforce. This effort aimed to provide a shared understanding of what is needed to launch and develop a successful career, a common vocabulary by which to discuss the relevant issues, and a basic set of competencies upon which a successful career is launched. The following section will provide a brief discussion of the theoretical underpinnings of each competency’s definition, including how they align or diverge from existing frameworks.
9 Theoretical Underpinnings of the NACE Career Readiness Competencies Based on the work of Casner-Lotto and colleagues as well as NACE members’ expertise in career development and talent attraction, the NACE Career Readiness Competencies reflect the eight competencies required to launch a successful career. The following sections briefly review their theoretical basis. Career readiness overall With few established definitions of Career Readiness, NACE relied on the available literature while developing a construct that was specifically focused on the competencies college graduates need to enter the professional workforce. The seminal work put forth by the Partnership for 21st century was used heavily by the first task force. Around the same time as NACE, OCTAE developed its employability framework, and the parallels are clear. Designed for a slightly different audience, OCTAE’s framework for employability skills is more focused on preparing people to be employable, which is a bit broader and more inclusive of the entire workforce, including high school graduates and non-graduates. As such, OCTAE’s definition is generally used for the Career and Technical Education (CTE) audience. In defining the construct, OCTAE notes that “Individuals require many skills to be college and career ready, including academic knowledge, technical expertise, and a set of general, cross-cutting abilities called ‘employability skills’” (OCTAE, 2021). Similarly, Casner-Lotto and Silvert (2008) define workforce readiness as the combination of basic skills and knowledge, e.g., reading, writing, and arithmetic; applied skills, e.g., teamwork, critical thinking; and emerging content areas, e.g., career management. Both of these definitions are comprehensive of the knowledge, skills, and abilities required to enter the workforce. Rather than define career readiness as encompassing all types of skills, the NACE task force chose to focus on a core set of competencies that are transferable across occupations and industries, instead of the direct or hard skills involved in the range of occupations and industries. Drawing on the core purpose of the construct, the definition reflects the importance of navigating one’s career toward success. Aligned largely with the work done of the U.S. Department of Education through OCTAE and the work of Casner-Lotto and colleagues, e.g., 2006, 2011, NACE offers the following definition of career readiness in 2020: Career readiness is a foundation from which to demonstrate requisite core competencies that broadly prepare the college educated for success in the workplace and lifelong career management. Please see Appendix A for the full set of definitions and associated behaviors for the following eight competencies. Appendix B contains the initial set of eight competencies.
10 Career and self-development The NACE Career Readiness task force debated how to revise the career management competency from the initial list of competencies, as it did not resonate strongly with students or employers. Self-awareness and career development were considered, as each are relevant to early career readiness and have their foundations in the literature. Jarvis (2003) defined career management as composed of personal and professional development, learning and work exploration, and life/work building. Similarly, Hirschi, Freund, Hermann (2013) included six dimensions within this idea: Career planning, Career self-exploration, Environmental career exploration, Networking, Voluntary human capital/skill development, and positioning behavior. As the idea of self-awareness and career development were embedded in these conceptions of career management, the task force landed on a definition of a competency that was aligned with these definitions and combined the benefits of self-awareness and career development. As such, the NACE task force adopted a title of Career and Selfdevelopment, defined as: Proactively develop oneself and one’s career through continual personal and professional learning, awareness of one’s strengths and weaknesses, navigation of career opportunities, and networking to build relationships within and without one’s organization. Communication Both oral and written communication have long been held as a core competency that is critical for entering the workforce. In fact, 96% of respondents indicated it was “very” or “extremely” important for recent graduates (NACE Recruiting Benchmarks, 2022). Initially published as oral/written communication, the most recent revision simplified the competency to just Communication, including its various forms: verbal, nonverbal, and written. Though difficult to assess, Casner-Lotto and colleagues (2006) as well as the U.S. Department of Education (OCTAE, 2014) have argued for the importance of being able to communicate in the workplace. Because workplaces generally consist of multiple people and teams working together to achieve common goals, communication is required for those efforts. As such, the NACE task force adopted a revised definition of: Clearly and effectively exchange information, ideas, facts, and perspectives with persons inside and outside of an organization. Critical thinking Similarly, critical thinking is always included as a core competency that is required for the 21st century workplace. Some frameworks include it as problem-solving, decisionmaking, or critical thinking. The NACE task force took the position that critical thinking
11 is required to solve problems and make decisions, thus, placing critical thinking as the foundational competency of this closely related trio. Casner-Lotto and colleagues defined critical thinking/problem solving as “exercise sound reasoning and analytical thinking; use knowledge, facts, and data to solve workplace problems; apply math and science concepts to problem solving” (p.16). ACT takes a broader view where critical thinking is a type of thinking skill that includes problem-solving and decision-making, while OCTAE defines critical thinking skills as ones that “enable employees to analyze, reason, solve problems, plan, organize, and make sound decisions in their work” (OCTAE, 2022). In an attempt to focus on the critical thinking skills most in need for early career employees, the NACE task force adopted a revised definition of: Identify and respond to needs based upon an understanding of situational context and logical analysis of relevant information. Equity and inclusion Though not widely recognized by the academic literature as critical to workplace success, the ability to be equitable and inclusive are increasingly essential. Not only do Equal Opportunity Laws require employers to not discriminate in their hiring process, indicating a need to act equitably, but the workforce, like the American general population, is diversifying at a rapid rate. For the first time since its inception, the U.S. Census reported that the number of people of color that are aged 18 and under rose from 46.5% in 2010 to 52.7% in 2020 (Frey, 2021). This development portends that workplaces will become increasingly diverse, further necessitating all employees to engage with each other equitably and inclusively. Without acting equitably and inclusively, workplaces will not maximize the productivity from their workforces. Employees may not feel comfortable or that they belong in their workplace, stifling their motivation, creativity, camaraderie, and, ultimately, their productivity. Additionally, employers are interested in cultivating diverse perspectives; they need employees that not only reap the rewards of an equitable and inclusive workplace, but also appreciate and know how to develop such environment. Thus, the NACE task force chose to emphasize the importance of these attitudes, behaviors, and mindsets by delineating it as its own competency. The 2018 task force initially developed this competency, calling it intercultural/ global fluency, which has its foundation in the literature, and is related to cultural competence. This initial version of the competency focused more on respecting diverse views and cultures; however, this definition received pushback from the practitioners who taught this competency to their students. Moreover, during a public comment period in the summer of 2020, the largest number of comments addressed
12 the shortcomings of this initial definition. Upon review of updated literature and the public comments, the task force adopted a definition closer to cultural competence, while adding important anti-racist language, adopting a definition of: Demonstrate the awareness, attitude, knowledge, and skills required to equitably engage and include people from different local and global cultures. Engage in anti-racist practices that actively challenge the systems, structures, and policies of racism. Leadership Just as colleges pride themselves on developing the leaders of the future, employers seek to hire them. Though employers tend to rate the importance of leadership as least important of the eight competencies for early career hires, still 65% of employers rated it as a very-to-extremely important competency in candidates (NACE Recruiting Benchmarks, 2022). In fact, more employers now believe the competency is important, compared with the first time they were surveyed on this competency in 2015, when only 56% cited it as very or extremely important. Even in entry-level positions, recent graduates work in teams in which they may act in some situational type of leadership capacity, though they may not serve as primary leaders in general. Nonetheless, learning leadership skills in college can set graduates on a path to career advancement, where they learn to manage innovation, change, and how to develop team members’ skills. Casner-Lotto, Rosenblum, and Wright (2006) defined leadership as: “Leverag[ing] the strengths of others to achieve common goals; us[ing] interpersonal skills to coach and develop others” (p. 16). The NACE task force adopted a similar leadership competency, not mentioning the interpersonal skills referenced above, to reduce the inherent overlap these competencies have with each other. The task force adopted this revised definition: Recognize and capitalize on personal and team strengths to achieve organizational goals. Professionalism Professionalism has long been a hallmark of an important workplace competency; OCTAE includes it under their conception of personal qualities. ETS includes it in their model as initiative and being a good workplace citizen. Casner-Lotto, Rosenblum, and Wright (2006) defined professionalism/work ethic as the ability to “demonstrate personal accountability, effective work habits, e.g., punctuality, working productively with others, and time and workload management” (p. 16). Given the different yet similar approaches, professionalism proves squirrelly to define.
13 The NACE task force wrestled with updating this definition while simultaneously considering the critique that professionalism is rooted in whiteness and can be used as an avenue to discriminate against employees of color (Gray, 2019). The critique rests on the idea that standards of professionalism serve as a coded language that favors the culture and values of white and Western employees. For example, braided hair, a natural hairstyle for Black or African-Americans, may not be considered professional, while other whiter, more Western, natural hair styles are considered professional. The task force determined the concept of professionalism is deeply embedded in the workplace; it’s a commonly used term and idea. Therefore, it is more useful to provide helpful guidance around this construct rather than adopt a new term that could later become problematic. Moreover, the task force decided to specify the term, removing some of the ambiguity that enables bias to creep into people’s evaluations of what is professional and what is not. The empirically-supported behaviors provided in Appendix A are the most specific the task force could reach in providing sample behaviors that are inclusive of cultures and values while reflecting the underlying concepts of professionalism to the extent they can be unbiased. To that end, the task force adopted the following definition: Knowing work environments differ greatly, understand and demonstrate effective work habits, and act in the interest of the larger community and workplace. Teamwork Given how much of today’s work is reliant on sharing information and contributing to workflows, teamwork is essential to nearly all jobs today, whether working remotely or in person. Indeed, it is listed near the top of every survey of important workplace competencies (e.g., NACE Recruiting Benchmarks, 2022; AAC&U, 2015). Casner-Lotto and colleagues (2006) defined teamwork as “Build[ing] collaborative relationships with colleagues and customers; be[ing] able to work with diverse teams, negotiate and manage conflicts” (p. 16). While similar overall, the revised NACE definition focuses less on customers and conflicts, and more on accomplishing goals through strong relationships because of the more specific nature of early career college graduates’ careers. As such, the task force adopted the following definition: Build and maintain collaborative relationships to work effectively toward common goals, while appreciating diverse viewpoints and shared responsibilities. Technology Use of technology is ubiquitous in the workplace, and inescapable in today’s remote work landscape. Casner-Lotto and colleagues (2006) defined technology application as “Select[ing] and us[ing] appropriate technology to accomplish a given task, apply[ing] computing skills to problem-solving” (p. 16). In comparison, ACT’s
14 conception of technology use focuses on: “Using technology knowledge and skills to effectively acquire and apply information” (ACT, n.d.). Both have similar definitions though and are also aligned with NACE’s definition. Originally titled “digital technology,” the most recent NACE task force revised and simplified the title to technology and removing specifications toward digital technology to be more inclusive of engineering and manufacturing type of careers. Similar to the ACT and OCTAE definitions, but again focused more on the types of technology use for early career college graduates, the NACE task force adopted the following revised definition: Understand and leverage technologies ethically to enhance efficiencies, complete tasks, and accomplish goals. The previous section has discussed the theoretical underpinnings of the definitions for career readiness and each competency, including their alignment with existing career readiness frameworks. The following section will detail the methods used and results obtained during the course of developing and validating the NACE Career Readiness Competencies.
15 Methods and Results NACE began the work of developing and validating a construct of Career Readiness in 2014 with a task force and has deployed subsequent task forces in the following years to refine and revise the competencies periodically. To provide an overview, initial development of the competencies was undertaken by the first task force, and then refined by second task force when it added global/intercultural fluency. Following that work, the Career Readiness Project launched with SkillSurvey to add observable behaviors to the conceptual definitions. The third task force used the work of the Career Readiness Project to make revisions, along with other sources of evidence, including public comments, empirical literature, and factor analyses. In all, the NACE Career Readiness competencies have been developed and validated iteratively, while relying on the expertise of leaders from career services and talent attraction. Presented chronologically, the following sections will detail the development and validation work of these task forces from 2015 to 2021. Original task force work in 2015 During 2015, NACE convened a task force that consisted of college career services and HR/recruiting professionals to define career readiness for the college educated. Informed by the body of research on 21st century skills and their professional experience, the task force identified seven competencies that define career readiness: critical thinking/problem-solving, oral/written communication, teamwork/ collaboration, application of digital technology, leadership, professionalism/work ethic, and career management. Following the task force’s work, NACE surveyed 606 of its employer members, querying them on the importance of these competencies in the workplace. See the table below for the results of this survey. Figure 1. Career Readiness Competencies Identified as “Essential” or “Absolutely Essential” COMPETENCY PERCENT OF RESPONDENTS Professionalism/Work Ethic 97.5% Critical Thinking/Problem Solving 96.3% Oral/Written Communications 91.6% Teamwork/Collaboration 90.0% Information Technology Application 72.0% Leadership 55.9% Career Management 45.0% Total Respondents 606
16 Second task force review in 2016 Based on feedback from NACE members and the public, the NACE president established a task force and charged it with revising the competencies to examine inclusion of global/intercultural fluency in 2016. A public comment period was provided to facilitate additional revisions to that competency. The task force determined an additional competency was essential: Global/Intercultural fluency was added to address important considerations of Diversity, Equity, and Inclusion. The task force then updated the competencies to include global/cultural fluency as the eighth competency, and the NACE board of directors voted to approve their recommendations. The task force also recommended NACE concretize the definitions of the competencies with observable behaviors that could be taught and assessed. Launch of current and ongoing career readiness project in 2018 Following up on the second task force’s recommendation, the NACE Center for Career Development and Talent Acquisition, working with SkillSurvey, began the first phase of an effort to give students, higher education professionals, and employers a consistent way to measure competencies as they relate to career readiness. This project was focused on adding observable behaviors to what were conceptual and aspirational definitions. To determine how to best measure career readiness with specific behaviors, the NACE Center and SkillSurvey collaborated to identify behavioral statements that can operationalize career readiness according to each of the eight competencies. More than 80 colleges and universities were involved in the first phase of the pilot, which ran from April through September 2018. Approximately 6,000 students, serving as interns or student workers, were part of the initial pool. Nearly 12,000 evaluators— made up of managers, co-workers, and mentors—provided their feedback on the students using the SkillSurvey instrument. Just over 86% of the students received feedback from more than one evaluator. Students were evaluated on a total of 28 behaviors that operationalized the eight competencies. These included such behaviors as “collaborate with others to achieve common goals” (mapped to the teamwork competency), “display proficiency with MS Office software” (digital technology), and “demonstrate dependability” (professionalism). Evaluators used a 7-point scale to rate each student intern’s behavior. In addition, they were asked to respond to two open-ended questions: Which work behaviors should the student continue demonstrating? Which work behaviors should the student start demonstrating?
17 Factor analysis #1 Based on this work, the NACE Research team conducted a factor analysis to identify a set of behaviors that can serve as reliable indicators of the underlying competencies. During the course of this analysis, the team discovered that the correlations between the competencies were extremely high, leading to a warning of a non-positive definite (NPD) matrix. In the case of a non-positive definite matrix, the results are not invalid, but they should also not be relied upon either (Wothke, 1993). To investigate, the NACE Research team examined the loadings and model fit of the behaviors for each competency separately, finding similar loadings, and none garnered the NPD warning. Only when combining all eight competencies did the NPD warning get triggered. In the end, the team interpreted the findings that the underlying structure was reasonable, but efforts should be made to reduce the overlap within the competencies and remove behavioral statements that did not perform well. See Appendix C for the results of the factor analysis, including factor loadings, inter-factor correlations, model fit, and residual correlations. Third task force review, 2019-20 A work group was formed in fall 2018 to do a simple review of the NACE Career Readiness Competencies. During that review, the Work Group recommended to the NACE Board that a much more in-depth review should be conducted and that such a review should be conducted on a more regular basis; perhaps every five years. The NACE Board of Directors concurred, and a third task force was formed to begin work in 2019. It was charged with revising the titles and definitions of the competencies as well as identifying behaviors that demonstrate each competency. After reviewing recent literature and data on career readiness and guidelines on competency development, the task force leadership team created a survey to capture general observations about task force members’ thoughts about the existing career readiness competencies based on the literature review. Results informed the beginning discussions as to how to proceed. The greatest amount of discussion revolved around Global Fluency and Career Management. Consideration was also given to the addition of new competencies such as adaptability and resiliency. Task force members agreed that competencies needed to be enhanced with the addition of practical examples of observable behaviors that could begin to be used as a means of assessing competence. Sub-committees were formed for each competency chosen to be included in the modified list. The committees’ charge was to consider the label, definition, and behaviors that would be associated with the competency. Sub-committees reported findings to the task force for review and alteration resulting in a revised set of career readiness competencies. The revisions were shared with the co-chairs of the Diversity, Equity, and Inclusion Committee to review for inclusive language.
18 The revisions were released to the NACE membership in late spring 2020 to seek public comment on the revisions, receiving 291 responses, generating 892 individual comments. NACE’s Research Department compiled and coded the data to provide the task force with a comprehensive report on themes. It is important to note that the George Floyd murder occurred during this time, and it became clear to all that these competencies have an important role to play in social justice, which should be reflected not only in the language but in the behaviors and actions therein. With the extensive member data and important societal events, the task force petitioned the Board to extend the competency review time. The Board concurred and extended the task force for a second year. Many task force members carried over while some members dropped off and others joined the task force. The second task force began its work by reading the NACE Research Department’s report and discussing the prevalent themes. The task force conducted additional research and made multiple revisions. In accordance with NACE’s Commitments to the Black Community and Anti-Racism, the task force endeavored to infuse anti-racist behaviors and those that are consistent with being equitable and inclusive into each competency. The task force Leadership Team consulted with the Knowledge-Based Director on the NACE Board for further assistance. The NACE Research Department also provided lists of corresponding behaviors from peer-reviewed research. Factor analysis #2 To support the task force’s work, the NACE Research Department conducted another factor analysis based on subsequent data collected by Skill Survey to determine a set of behaviors that serve as indicators of each competency. Again, the team discovered that the correlations between the competencies were extremely high, leading again to a warning of a non-positive definite (NPD) matrix. When checking the eigen values, they were smaller and closer to zero than for the first factor analysis, so the revisions made after the first factor analysis were helpful. Conducting a sensitivity analysis of sorts, the NACE Research Department modeled each competency and their associated behaviors separately; loadings and model fit were similarly strong. The NACE Research Department thus took the perspective that model fit and loadings were good, but the factors or competencies were still too highly correlated and producing the NPD warning (Wothke, 1993). With no other likely explanation, the NACE Research Department and task force had to accept that the competencies were highly correlated, as they are in real-life. Moreover, as of now, the behaviors are not being used within a specific instrument; they are being provided to the public as examples of empirically-supported behaviors for these competencies. As such, the NACE Research Department felt comfortable with the results, considering the NPD warning.
19 The NACE Research Department holds the opinion that the competencies will always be highly correlated due to their close relationship in real life. Indeed, to respond appropriately in a work setting, one will likely have to use multiple competencies at one time. For example, if an employee needs to navigate an uncomfortable social situation with a team member, the employee may have to use their professionalism, communication, critical thinking, and teamwork competencies all at the same time. Undeniably, these competencies are intertwined, which is why they are all important to the underlying construct of career readiness. Please see Appendix D for the specific methodology used in conducting the factor analysis along with the results including factor loadings, inter-factor correlations, fit statistics, and residual correlations. Content validity survey The task force agreed that one more round of revisions was needed to ensure empirically validated behaviors were included for each competency. It was also recognized that NACE, as an authoritative organization of professional practitioners, may include behaviors that achieved a substantial level of agreement according to the experts on the task force. As such, the NACE Research Department created a content validity survey to secure task force levels of agreement and feedback on the proposed titles, definitions, and behaviors. The results were shared with the task force for final review. A total of 22 task force members participated, each a content expert in their own right. The sample was 59% female and 73% white. The survey queried task force members on their levels of agreement with the proposed titles, definitions, and behaviors. Please see Appendix E for the full slate of content validity results. Given the sample size of the content validity survey and that the behaviors were being offered to the public as sample behaviors, the task force adopted a low threshold of 50% agreement on whether to include or reject the behavior from the final list. Additionally, the task force decided to keep all the items in the SkillSurvey instrument, except for two that were deemed “hard” skills. With the understanding that the list of behaviors would be refined in the coming years, the task force chose to begin with the widest set of behaviors possible to offer the field. Figures in Appendix E provide these results. The final set of revised titles, definitions, and sample behaviors can be read in Appendix A. In summary, the third task force used several sources of data from various constituencies, including: students, supervisors, public comments, empirical research, as well as NACE research, and last, the judgment and experience of the expert practitioners on the task force. The task force:
20 • Reviewed the literature on measuring these competencies and which behaviors are indicators of them. • Considered nearly 900 public comments sent in by nearly 300 members of the public. • Conducted a factor analysis. • Relied on the criterion related validity study conducted by SkillSurvey. • Conducted a content validity study with the members of our task force to estimate the extent to which the task force achieved agreement on the titles, definitions, and sample behaviors. The final proposal was presented to the Diversity, Equity, and Inclusion Committee co-chairs to review for inclusive language during January 2021. The NACE Board of Directors approved the final proposal, and the newly revised Career Readiness Core Competencies were released to the public in April 2021. Student perceptions of the revised competencies’ importance Shortly after this period of review, the NACE 2021 Student Survey was administered where students were queried on the importance of the newly revised competencies in beginning their careers. Compared to 2019, which was selected because it was the most recent pre-pandemic data, it appears the revisions to three competencies had an effect on students’ perceptions of their importance. For instance, the importance of career management increased from 80% of students endorsing it to 92% of students endorsing the importance of career and self-development. Interestingly, global/intercultural fluency never resonated strongly with students; students always ranked this competency among the least important. Revising the competency to focus more on equity and inclusion, however, seems to have connected with their outlook on the competency. The former version focused on respect for cultures and values, while the revised definition is based in cultural competence and anti-racism. It should also be noted that the country was actively undergoing a racial reckoning due to the murders of George Floyd, Ahmaud Arbery, Breonna Taylor, and too many others during this period of data collection; understandably, these events likely also played a role in students’ perceptions of this competency, just as they did in the public comment period during the summer of 2020. A decrease in the importance of professionalism is also notable, though the decrease in percent of students endorsing its importance was not very large. The decrease could be due to the removal of “work ethic” in the title or to a changing understanding of professionalism’s importance due to the pandemic’s effects on virtual vs in-person employment. As well, the underlying assumptions of professionalism are being challenged in the zeitgeist (as discussed above), and students’ lower importance ratings may also reflect that.
21 The methodology and results discussed in the previous section outline the work completed to date on developing and validating the NACE Career Readiness Competencies. Given the range of work, types of data, and diversity of thought leaders, the competencies have been developed and validated with solid evidence from a diverse range of sources. Please see the appendices mentioned for a more detailed treatment of the methods and results of each development and validation phase. The following section will discuss the path forward to continue developing and validating the construct. Figure 2. Student ratings of importance for NACE Career Readiness Competencies, 2019 & 2021 2019 (% IMPORTANT/EXTREMELY IMPORTANT) 2021 (% IMPORTANT/EXTREMELY IMPORTANT) 1. Professionalism/Work Ethic (95%) 1. Communication (97%) 2. Critical Thinking (95%) 2. Critical Thinking (94%) 3. Oral/Written Communication (94%) 3. Career & Self-development (92%) 4. Teamwork/Collaboration (92%) 4. Teamwork (91%) 5. Leadership (86%) 5. Professionalism (89%) 6. Career Management (80%) 6. Equity & Inclusion (82%) 7. Digital Technology (76%) 7. Leadership (85%) 8. Global/Intercultural Fluency (58%) 8. Technology (80%)
22 Discussion The purpose of this technical report is to present the evidence for the construct validity of the NACE Career Readiness Competencies. To that end, this report has detailed the theoretical underpinnings of the construct as well the methods used and results obtained during its development and validation. In summary, initial development of the competencies was undertaken by the first task force, and then refined by second task force when it added global/intercultural fluency. Following that work, the Career Readiness Project launched with SkillSurvey that moved the aspirational and conceptual definitions to more observable and measurable competencies. The third task force used the work of the Career Readiness Project in making its revisions, along with other sources of evidence, including public comments, empirical literature, and factor analyses. Taken together, the NACE Career Readiness competencies have been developed and validated iteratively, with input from various perspectives, and in concert with leaders from career services and talent attraction. Future research Future research on the NACE Career Readiness competencies will involve various approaches including ongoing behavior validation, examining differences in career readiness by in-person or remote experiential learning, examining the diversity, equity and inclusion implications, developing rubrics and assessments for career readiness, and linking career readiness with student outcomes. NACE will conduct analyses that examine differences in career readiness by a wide range of characteristics. NACE routinely asks students to what extent they think their experiential learning opportunity helped improve each competency. Thus, NACE can investigate if differences emerge between those who are in-person, remote, or hybrid, as well as a host of other differences including, but not limited to, gender, race/ ethnicity, age, and first-generation status. As well, through the data being collected through the Career Readiness Project in partnership with SkillSurvey, NACE and SkillSurvey will examine the differences between first-year students and seniors. One would expect that seniors demonstrate higher levels of career readiness than first year students. An analysis of these differences may demonstrate the discriminant validity of the competencies and its associated measures. More broadly, the partnership will seek to the further validate the construct by linking it with career outcomes, including for example salary, promotions, and career satisfaction.
23 References ACT. (2022) College and Career Readiness Holistic Framework. ACT. Retrieved from: www.act.org/content/act/en/college-and-career-readiness/holistic-framework.html on December 4, 2022. ACT. (n.d.). Holistic Framework eBook, Cross-cutting abilities. ACT. Retrieved from: https://pages2.act.org/rs/035-EZR-959/images/Cross%20Cutting%20eBook.pdf on December 5, 2022. Association for Training and Development. (2016). 2016 ATD State of the Industry Report. Casner-Lotto, J., Rosenblum, E., & Wright, M. (2009). The ill-prepared US workforce: Exploring the challenges of employer-provided workforce readiness training. New York, NY: Conference Board. Casner-Lotto, J., & Silvert, H. M. (2008). New graduates’ workforce readiness: The midmarket perspective. The Conference Board. Research Report, Report Number R-141308-RR. Retrieved from www.conference-board.org/topics/publication detail. Casner-Lotto, J., & Barrington, L. (2006). Are they really ready to work? Employers’ perspectives on the basic knowledge and applied skills of new entrants to the 21st century US workforce. Partnership for 21st Century Skills. Conley, D.T. (2011). Pathways to postsecondary and career readiness. Invited speaker at College and Career Readiness Regional Workshop, Wellington, NZ. Efetie, D., & Oetjens, M. (2021). VCU introduces new career readiness minor, aims to prepare students for workforce. The Commonwealth Times, retrieved from: https://commonwealthtimes.org/2021/12/01/vcu-introduces-new-career-readinessminor-aims-to-prepare-students-for-workforce/ on December 1, 2021. Educational Testing Service. (2022). Retrieved from: https://hiset.ets.org/states-policy on December 7, 2021. Frey, W. (2021). New 2020 census results show increased diversity countering decadelong declines in America’s white and youth populations. The Brookings Institution, retrieved from: www.brookings.edu/research/new-2020-census-results-showincreased-diversity-countering-decade-long-declines-in-americas-white-and-youthpopulations/ on December 1, 2021. Gallup & Strada (2017). 2017 college student survey: A nationally representative survey of currently enrolled students. Washington D.C.: Gallup.
24 Gray, A. (2019). The bias of “professionalism” standards. Stanford Social Innovation Review, https://doi.org/10.48558/TDWC-4756. Retrieved on August 1, 2020. Labi, A. (2014). Closing the skills gap: Companies and colleges collaborating for change. Economist Intelligence Unit. New York: New York. McGraw-Hill (2018). McGraw-Hill Future Workforce Survey. Retrieved from: www.mheducation.com/news-media/press-releases/2018-future-workforce-surveyresults.html on October 15, 2021. NACE Recruiting Benchmarks (2022). National Association of Colleges and Employers. Statista. (2020). Total training expenditures in the United States from 2012 to 2020. Retrieved from: www.statista.com/statistics/788521/training-expenditures-united-states/ on December 5, 2021. The Conference Board (2008). New Graduates’ Workforce Readiness: The Mid-Market Perspective. Research report R-1413-08-RR. Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 256-93). Newbury Park, CA: Sage.
RkJQdWJsaXNoZXIy MTM2NjgzMA==