Manufacturing Institute

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, Manufacturing Institute had issued functioning badges. Based on this information, we have classified this badge system as an implemented  (rather than a partial or suspended) badge system.

Summary

In collaboration with ThinkStache, the Manufacturing Institute designed the National Manufacturing Badge System, implementing their plans through the network Dream It. Do It. and the organizations SkillsUSA and Project Lead The Way. The Manufacturing System aims “to help ensure an educated and skilled 21st Advanced Manufacturing workforce, critical to the innovation capacity and business success of U.S. manufacturers in the global economy; to link and leverage formal and informal learning environments and opportunities benefiting students and workers; and to bring national recognition to world-class informal learning organizations that contribute to both student and business success” (DML Stage 1 Proposal). To this end, they sought to construct a badge system that would enhance students’ learning experiences and strengthen their capacity for self-presentation in the job application process.

As part of the project effort, the badge alone does not indicate an individual is prepared for a certain field. Rather, badges act as enhancements to a resume or application and communicate what a resume is unable to get across. Additionally, the badge development initiative envisioned that the “National Manufacturing Badge System will contribute to the larger badge ecosystem in several ways:  It will serve as a model for other industry sectors in linking and leveraging partnerships with formal education institutions and informal learning organizations; the badges will represent skills, competencies, capacities, qualities and achievements that are applicable across industry sectors, e.g., energy and construction; it will inspire practical use of the larger badge ecosystem by adding value that translates to high quality, middle class jobs” (DML Stage 1 Proposal). The educational context for the National Manufacturing Badge System is a career setting; they are working in community colleges and higher educational settings, and extending their program into upper grades of high school.

The initiative integrates badges into classroom practice. The Manufacturing System explained in their proposal that “[t]he Program of Work, properly executed, is co-curricular, not extra-curricular, and is integrated into daily lesson plans and activities of the classroom and lab. The Program of Work includes competitions, professional development, community service, in-school employment, social activities, public relations and chapter fund raising” (DML Stage 1 Proposal). The SkillsUSA focuses on carrying out extra-curricular activities through SkillsUSA Championships, which promote competition-based learning for career and technical students, while Project Lead The Way implements co-curricular badge system activities in science, technology, engineering, and math (STEM) within a classroom setting. Overall, the initiative built a badge system to promote technical and communication skills that would ultimately lead to careers in the manufacturing industry.

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. Below each heading, we indicate what stages this principle appeared in the project: as an (a) intended practice, (b) enacted practice, or (c) formalized continuing practice.

Design Principles for Recognizing Learning

The Manufacturing Institute aspired to recognize a diverse range of skills that included not only technical industry expertise but also interpersonal workplace skills. Moreover, the badge system aligned badges to a set of standards. The project reached out to the external community to increase the value and usefulness of the badge. While initially intending to incorporate a leveled badge structure, the badge effort decided to design one type of badge, ensuring that badge audiences can easily understand and interpret the learning it represents.

Recognize diverse learning > Badges for hard and soft skills

Intended, enacted, formalized

The badge development effort intended to recognize an array of both soft and hard skills. In their DML proposal, the project highlighted the skills of “critical thinking, creativity, innovation, real-world problem solving, teamwork, leadership, performance, ethics, and technical and industry knowledge” (DML Stage 1 Proposal). The Manufacturing Institute planned to award badges for both performance-based concrete and interpersonal, communication skills. They explained, “These ‘street smarts’ and ‘book smarts’ are directly aligned to the needs of manufacturers, who are looking for the next generation of skilled talent that can drive innovation, invent the next big products, and keep their companies competitive in the global marketplace” (DML Stage 1 Proposal). Jacey Wilkins, Director of Communications of the Manufacturing Institute, pointed out that these skills are critically important when it comes to career and hiring. For instance, soft skills are vital when performing under pressure and in teams is key in these career fields (DPD Initial Interview).

Align badges to standards > Use national or international standards > Standards alignment

Intended, enacted, formalized

The Manufacturing Institute planned to map badges to standards of an accredited institution or agency. The project articulated its aim to design Skill Connect Assessments that would reflect the competencies encompassed in “the Skill Connect Assessment in Automated Manufacturing Technology. . .[and in which] the student must interpret information from written, photographic, and animated depictions in order to interactively answer questions about technical knowledge and hands-on process protocols and procedures” (DML Stage 1 Proposal). The Skill Connect Assessments are industry and specialty specific. The Manufacturing Institute envisioned that learners would earn badges before they complete a Skill Connect Assessment,indicating that the learner is ready for the more formal assessment. In the implemented badge system, the initiative aligned badges to standards inherent in the project activities instead of the Skill Connect Assessments. Brent Weil, Senior Vice President and Treasurer of the Manufacturing Institute, asserted that badge earners meet a specific set of requirements for a task (e.g., scoring a certain percentage) before receiving a badge (DPD Follow-up Interview).

Seek external backing of credential > Externally valued > Outside community recognition

Intended, not enacted, not formalized.

The initiative envisioned the badge as a credential recognized outside of the issuing community or institution. They described that “[t]he badges will represent skills, competencies, capacities, qualities and achievements that are applicable across industry sectors, e.g., energy and construction; it will inspire practical use of the larger badge ecosystem by adding value that translates to high quality, middle class jobs” (DML Stage 1 Proposal). In an initial interview conducted by the DPD project, Wilkins elaborated that the badge credential would lead into a bigger formal, existing credential program. In this regard, the badges are about outside connectivity. If the badge is only useful within the badging community, it would not be useful for a learner who is trying to get a manufacturing badge.

As the badge system delved into the criteria for badges, their idea of what badges are going to represent has changed. Instead of just representing explicit skills, the effort shifted its focus to uses for directing more students into manufacturing careers. The project also planned to raise employers awareness of manufacturing preparation programs (DPD Initial Interview). Their DML proposal stated that “[t]he Institute has designed and deployed the national manufacturing education reform agenda, implementing the NAM-Endorsed Manufacturing Skills Certification System. . .of industry-recognized credentials integrated in high schools, community and 4-year colleges and universities, creates efficient, competency-based education pathways to careers ranging from skilled production to engineering and in all sectors in the manufacturing economy” (DML Stage 1 Proposal). In the DPD follow-up interview, Weil further added that the project has reached out to pilot companies and brought them into the practice to evaluate the badge development effort and give feedback on the criteria. From there, however, the project did not attain a broader reach, to push badge practices and “move needle nationally on companies” (DPD Follow-up Interview).

Use badges to map learning trajectory > Badges are leveled

Intended, not enacted, not formal.

The project intended to design different levels of badges to indicate learners’ progression towards mastery in a specific area. As skills increase, so too would the level of the badge. In the constructed badge system, since the primary audience of the badges is employers, the badges will not be leveled. The project articulated that having different versions of a badge will impede communication of the learning and skills that the badges represent. Badges must be easily understood by outside entities. Structure and naming of badges is reliant on assumptions of badge audience (DPD Initial Interview). In thinking about how potential employers will view the badges, it became apparent that leveled badges may confuse more than help the earner. As the project learned more about the potential of badges and thought about how they were going to reach students, they decided they wanted fewer badges based more on thresholds or benchmarks than levels of achievement (DPD Initial Interview). The DML proposal indicated the possibility that badges would be available for students and teachers. In the scaling back of the badges, this practice has changed, and badges are only available for students. While there is the potential for teacher badges, the badge system focuses on youth badges. Although there are certifications required to teach in the programs, badges are not part of those certifications (DPD Initial Interview). To keep badges clear and straightforward to employers, the project decided to streamline the process and design one kind of badge.

Design Principles for Assessing Learning in Digital Badge Systems

The badge system designed several layers of assessment to measure a combination of both concrete and soft skills. For instance, the badge system employs project-based assessments in which students draw from a range of skillsets and problem solving abilities. Besides this, experts and computer systems assess the learning represented by a digital badge, boosting its validity and credibility. Additionally, the assessments are aligned to the badge system’s standards, providing a framework from which to gauge learning.

Promote “hard” and “soft” skills > Combination of collaborative learning and discrete skills > Layered assessment

Intended, enacted, formalized

The project intended to design assessments for badges that recognize both performance-based and soft skills. The DML proposal delineates that SkillsUSA Championships “[offer] students the opportunity to demonstrate through competitions their leadership or hands-on occupational skills, to learn current industry expectations, and to receive recognition for achievement” (DML Stage 1 Proposal). In this way, badges promote skill and competency development through participation in projects.

In the badge system, assessment is layered to include project-based performances and end-of-course assessments so that ‘book smarts’ and ‘street smarts’ are assessed. The curriculum for the National Manufacturing Badge System is integrated into the classroom. Rather than just assess projects and cooperative learning, the National Manufacturing Badge System project realized that they needed a way to assess declarative and procedural knowledge, and have integrated layers of assessment to do so.

Enhance validity with expert judgment > Use a combination of human and computer experts > Validation by experts

Intended, enacted, formalized

Depending on the project, the badge system have experts or a computer system validate the learning recognized by a badge. To take a case in point, the SkillsUSA Championships involve competitions at all levels that are designed and managed by technical committees drawn from the ranks of business, industry and labor. Industry experts also judge these contests (DML Stage 1 Proposal).

Besides SkillsUSA, the DML Proposal highlighted that “[s]tudents who earn a Project Lead the Way Badge will have scored at least 85% on Pathway to Engineering end-of-course assessments” and engaged in an engineering- or industry-related project or activity (DML Stage 1 Proposal).  Because assessments are layered, some assessments may be scored by a computer system, but many projects will be scored by experts.

Use performance assessments in relevant contexts > Project-based assessment

Intended, enacted, formalized

The badge development effort planned to design performance-based assessments for badges. The badge effort asserted, “Much of the informal learning content can be validated only by performance. Modeling and simulation capabilities may be necessary to measure and account for new skills and knowledge” (DML Stage 1 Proposal). In the badge system, most assessments are project based, especially through Project Lead The Way. For instance, there are some assessments that a student can take from reading a textbook, but most assessments involve problem solving, providing opportunities where students work collaboratively to solve complex problems.

Align assessment activities to standards: create measurable learning objectives > Alignment of Skill Connect Assessments

Intended, enacted, formalized

The initiative aimed to align assessment activities to standards. The Manufacturing Institute delineated that “the Technical Standards for the Automated Manufacturing Technology contest identifies Math Skills, Science Skills, and Language Arts Skills that are embedded in the contest. Each hands-on contest also includes some element of professional (soft skill) assessment and a written test of relevant technical knowledge, both of which are added in the student’s final score” (DML Stage 1 Proposal). To this end, the project designed assessments that would adhere to a set of standards to measure the quality of learning. The Manufacturing Institute added that “[i]n the case of the Skill Connect Assessments, the student must interpret information from written, photographic, and animated depictions in order to interactively answer questions about technical knowledge and hands-on process protocols and procedures” (DML Stage 1 Proposal). In the constructed badge system, while there are standards to which the curriculum is aligned, each program and region has their own internal processes for determining what it takes to earn a badge. There are some national projects, but many projects are judged by regional or programmatic standards.

Design Principles for Motivating Learning

This initiative’s primary goal is to have a student hired for a manufacturing job as a result of his or her badges. The intention of the National Manufacturing Badge System is to present students’ skills, knowledge, and experiences in a way that is appealing and understandable to potential employers. This program also seeks to reach out to students for whom the traditional school system and testing modes are not well-suited; badges may be ideal for these types of students. In addition to providing motivation for graduation and finding employment, badges could potentially link students to scholarships.

Utilize different types of assessment > Computer > Increase evidence of employability

Intended, enacted, formalized

The badge initiative aspired to design badges that motivate earners by providing increased evidence of employability. In their DML proposal, the badge system explained that “[t]he opportunities include: connecting the Open Badge Infrastructure directly to The Manufacturing Institute’s web-based ‘Engagement to Education to Employment’ Pipeline, which is being rolled out as the U.S. Manufacturing Pipeline. Thus, badges earned and collected could be “transported” directly to a student’s or worker’s “e-portfolio” to present a comprehensive picture of skills, competencies, qualities, and achievements to both educational institutions and manufacturing employers…” (DML Stage 1 Proposal). In this respect, badges will motivate earners by providing a clear pathway toward certification and employment. Wilkins further described:

“We really see these badges as feeders into the certification programs, which are really important for employment. It’s great to have your SkillsUSA badge for welding: it shows that you know how to weld, it shows that you know how to work in teams, that you’re a problem solver and a leader. But most employers are going to need you to be AWS-certified. We can then guide those individuals whom we reach through badges into these formal education programs” (DPD Initial Interview).

As the badge system employs multiple kinds of assessments, the project strives to boost and sustain learner engagement.

Set goals > Personalized recommendation of goals > Personalization

Intended, enacted, formalized

The badge system envisioned boosting learner engagement by providing a personalized recommendation of goals. The project effort pointed out that “Thinkstache makes recommendations to students about post-secondary open content and courses that they may be interested in based on their background” (DML Stage 2 Proposal). As the badge system suggests a particular set of goals and badges to achieve, the project’s hope is to encourage students to persist and continue developing their skills.

Engage with the community > Involvement in digital community > Participatory communities

Intended, enacted, formalized

The effort planned to leverage badges to motivate earners to engage in a participatory community. The DML proposal detailed that “[t]he National Manufacturing Badge System will support and encourage participation in impactful, results-oriented youth- and worker-development organizations and their informal learning opportunities, producing a more highly educated and skilled 21st century workforce” (DML Stage 1 Proposal). In the badge system, the project has not specifically directed its focus toward building an online community, and many students engage in a face-to-face community.

Recognizing identities > Targets a specific group > Engaging at-risk students

Intended, enacted, formalized

The project intended for badges to motivate earners by providing evidence of skills and experience for at-risk students. Wilkins described that “[i]f you look at a model like SkillsUSA, students achieve in so many different ways, and the way that our education system picks winners and losers is test scores. Not all students learn that way, and not all students can display what they know that way. What we’ve witnessed at SkillsUSA is that these students, who might be at risk of dropping out, or might not be doing that well in school, they find their path and find their passion and find a focus in some of these technical skills” (DPD Initial Interview). The Manufacturing Institute strove to engage at-risk students in learning and cultivating their abilities.

Build outside value for badges > Evidence for outside opportunities > Showing real life connections

Intended, enacted, formal.

The Manufacturing Institute envisioned that badges could motivate earners by providing a clear pathway to graduation and employment as well as by connecting them to scholarships and other opportunities. Wilkins asserted:

“Finding the badge as a real connection to real opportunities is so critical to these kids, who don’t see the relevance in their schoolwork or are so discouraged because they don’t do well on tests. The badge is helping connect them to real opportunities, to connect them to scholarships, to show them this clear pathway with, ‘Hey, if I stay in school and I continue to be interested in math and science (or whatever it is), there’s a real clear pathway.’ The badges are kind of the mile markers along the way” (DPD Initial Interview).

In the implemented badge system, this is the continued hope of the badge project.

Design Principles for Studying Learning

The badge development initiative planned to conduct research on its badge system. To this end, they have collected data through email and gathered information based on the users that have participated in the system. Ultimately. they have been able to study fundamental aspects of and gain knowledge about the badge system.

Study badge impact > Research OF badges > Broad communication of research

Intended, enacted, formalized

The Manufacturing Institute showcases a history of conducting research and producing reports, including widely cited reports such as the Structural Cost Study, the Facts About Modern Manufacturing, the Annual Public Perception Survey, and the Skills Gap” (DML Stage 1 Proposal). The DML proposal pointed out:

“The Institute produces comprehensive, data-based research to enable industry, government, media, and the public to understand the full scope of issues manufacturers face today. Telling the dynamic story of domestic manufacturing is imperative to advocate its strength and value to the economy. The Institute produces a consistently updated library of the latest data around the opportunities and challenges for U.S. manufacturing, including the facts about the economy, the manufacturing image, structural costs associated with market competitiveness, solutions for innovation and market development, and innovative thinking on human resources best-practices, education reform, and workforce development” (DML Stage 1 Proposal).

As of the DPD initial interview, the badge system has been working on how to use badging data to do research. The project articulated that they were specifically interested in the first person who gets hired due to the badges they have (DPD Initial Interview). As of the DPD Follow-up Interview, the Manufacturing Institute badge initiative has worked with organizations on how best to capture email addresses. Weil explained that they monitor the users receive any email and whether sign up for badge validation occurred (DPD Follow-up Interview).

References

Manufacturing Institute. (n.d.). DML Stage 1 Proposal. Retrieved from http://dml4.

dmlcompetition.net/dml4.dmlcompetition.net/Competition/4/badges-projects.php%3Fid=2252.html

ThinkStache. (n.d.). DML Stage 2 Proposal. Retrieved from http://dml4.dmlcompetition.

net/dml4.dmlcompetition.net/Competition/4/badges-projects.php%3Fid=3247.html

Weil, B. (2014, December 19). DPD Follow-up Interview.

Wilkins, J. (2012, October 29). DPD Initial Interview.

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance