Who Built America Badges for History Education

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, Who Built America Badges for History Education (WBA) had issued badges and developed a functioning badge system. Based on the information, we have classified the WBA badge system as an implemented   (rather than a partial or suspended) badge system..

Summary

Who Built America? Textbooks by the American Social History Project at CUNY

Who Built America Textbooks, by the American Social History Project

Who Built America Badges for History Education (WBA) is a project developed by the American Social History Project (ASHP) in partnership with Electric Funstuff and Education Development Center (EDC). The project moves the face-to-face synchronous professional development for grade 7-12 history teachers that the ASHP has been conducting for over 20 years to an online asynchronous space where teachers work at their own pace and without a cohort. This has been a somewhat challenging process, but the use of leveled digital badges and formative assessments have aided in the process. WBA master history teacher badgeThere is substantial effort to build community within WBA, and teachers can build relationships beyond a small cohort. This professional development helps teachers grow in both content and pedagogy; through a series of tasks and engagement in both the community and with ASHP experts, teachers can earn the badge and title of Master History teacher. The title and badge of Master History Teacher is representative of a teacher’s work, expertise, and experience of growth as a professional, and is meant to be recognized by education professionals outside of the WBA community such as principals and district officials. Teachers are required to participate in a professional development activity, and are encouraged to share resources, insights, and constructive criticisms. Teachers can then deem certain posts as helpful. One exciting evolution of practice is the project’s original intent to award a collaboration badge. Due to technical limitations, the project needed to revisit how they might assess collaboration. They decided that rather than award a collaboration badge, they would award activities that required collaboration to complete. In this way, they avoid assessing collaboration and potentially stopping useful discourse, and encourage deeper collaborations and connections between teachers.

Timeline

In March 2012, the Badges for Lifelong Learning competition announced the final awards, and Who Built America (WBA) began designing their digital badge system. The American Social History Project teamed up with Jim Diamond at Education Development Center to move their synchronous, face-to-face professional development for history teachers to an asynchronous online setting that uses digital badges to recognize teacher learning. In July 2012, Jim gave an initial report of where the project was headed, making clear that they were still in the early stages of development. By January 2013, the project had developed both their system within the platform and the curricula, but by March 2013 it was clear that the project would not be ready to launch until November. October 31, 2013, WBA had a “soft launch” with a focus group of teachers who had met and worked face to face. The following week the project launched with new teachers. WBA is in the process of working with their first group of fully-online teachers at the time this is being written.

Evolution of Practice and Relation to Principles

The DPD team analyzed each of the 30 projects’ initial proposals to extract their intended practices for implementing a digital badge system. Then the team conducted interviews with each of the projects after they had had time to develop their systems; from these interviews, the team documented the shifts in practices the projects had made, and categorized their enacted practices into specific design principles in the badge practice categories of recognizing, assessing, motivating, and studying learning. These specific design principles were then categorized into general design principles within each of the four badge practice categories. What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. The paragraphs below each heading detail the project’s (a) intended practice, (b) enacted practice, and (c) how that practice relates to the specific and general principles.

Design Principles for Recognizing Learning in Digital Badge Systems

Ultimately, digital badges recognize some kind of learning. WBA built their system to recognize both the “hard skills” of content and pedagogy, and the “soft skills” of community building.

Have experts issue badges > Credentialed via accredited entity and community > Validation by experts.  

Not intended, enacted, formal (Credentialed via community)

Intended, not enacted, not formal (Credentialed via accredited entity)

WBA intended for the Master History Teacher credential (badge) to be validated by experts both external and internal to the community. They felt that “full certification as a Master History Teacher necessitates a multi-step review process validated by history educators and master teachers. There is no self-accreditation in this system.” In the beginning of the project, WBA had also intended to allow peers to issue smaller badges.

This practice was further refined to state that submitting an artifact does not automatically result in receiving a badge. The learner must engage in discussion with the WBA experts concerning the artifact and the process they engaged in before any badge is awarded. The peer-awarded badges were ultimately not included due to constraints of the platform.

While ultimately the peer community did not award badges, the experts at ASHP do award badges and engage in discussion with teachers about their lessons, implementations, annotations on student work, and modifications of the lessons. By having history content and teaching experts engage in these discussions and issue badges, the credential may have more weight to both the internal learning community and administrators and district officials who view them.

Use badges to map learning trajectory > Level badges > Badges are leveled

Intended, enacted, formal

WBA intended for there to be an explicit balance between low-value and high-value/scarce badges. They wanted teachers to develop a “rhythm of badges” through which the teacher would earn smaller and bigger badges awarded automatically and by experts. They felt that “the process of moving through the inquiry units creates a self-paced rhythm with checkpoints for assessment, both automatic and human mediated.” A self-paced rhythm was also to be developed in earning community badges, with the higher level badges requiring more time and investment in the learning process. The largest badge, the Master History teacher badge, was intended to be awarded twice annually, “giving the accomplishment a ceremonial feel as befits a significant achievement.”

As the project developed, the levels took shape as three types: (a) “Builder” Badges, which are awarded for performance and consist of three tiers (Apprentice, Journeyman, and Master); (b) “Specialist” Badges, which allow a teacher to become a specialist in specific content; and (3) “Community” Badges, which recognize posts to the discussion forum, interaction between peers, and general involvement in the community. The concern about the three types of badges and the tiers within the performance badges was how the structure would be communicated to learners. During the first release of the curricula and badges WBA will pay close attention to how learners understand this structure and process.

The three types of badges in the WBA system

The three types of badges in the WBA system

By adding levels to a badging system, WBA allows teachers to find benchmarks in their learning and mark smaller accomplishments with a badge before they tackle the ultimate Master History teacher badge. This gives both the teachers and the experts an opportunity to see the learning trajectory and mark achievement goals. 

Determine appropriate lifespan of badges > Never expires > Credential is permanent

Intended, enacted, formal

WBA intended for the Master History Teacher badge to exist permanently on the website. Even after this ultimate badge was earned, teachers could still earn Community and Specialist badges.

As the project developed, WBA realized a need to convert badges to PDFs that can be printed and given to administrators. These PDFs contain information regarding how many hours were spent in professional development, a syllabus of objectives and activities, and other quantifiable evidence that is easily digested.

There is a concern that administrators such as principals may not understand that the digital badge is a representation of the earning teacher’s experience and expertise, thus a paper version has been created to add to the permanence of the credential.

Use badges as a means of external communication of learning > Credential as external communication

Intended, enacted, formal

WBA intended for the meta-data contained in each badge to ensure that the credential would be recognized outside of the immediate learning community. They intended to make these credentials even more valued by making the rubrics with which the teachers were assessed publicly available.

Related to the issues that arose around making the credential permanent, WBA realized that professional educators outside of the WBA community may not understand all of the data and evidence a digital badge contains, so they made a paper version of the badge. The collaborator badge was removed because the constraints of the badging platform made it difficult to provide convincing evidence that warranted such a badge.

The concern that educators outside the WBA community will not recognize the work, accomplishments, and expertise the digital badge represents is still a concern for the the project and the teachers. WBA is actively trying to find ways to make these badges valued by the external community.

Align credential to standards > Use community and national/international standards > Align credential to standards

Intended, enacted, formal

WBA intended to align the curricula the teachers taught to both the Common Core State Standards and to other standards for teaching history. This meant that the work teachers did was also aligned to these standards. This intention did not change as the project developed.

Ultimately, both the credentials and the curricula for students are aligned to the Common Core State Standards, and several other standard sets. The language from the National Council of History Teachers 3Cs standards have also made their way into the curricula. WBA is interested in the Thinking Like a Historian project in Madison Wisconsin, though that project does not include badges.

Seek external backing of credential  > Externally endorsed > Badge endorsements by nationally recognized association

Intended, not enacted, not formal

WBA intended for their badges to be endorsed by the National Council for the Social Studies, National History Education Clearinghouse, and Stanford History Education Group to help ensure its recognition by people outside the WBA community.

While obtaining endorsements is still part of the goal, this effort has been put aside as the project moves toward its first launch. WBA would like to at least get outside organizations to link to or inside the badge to give it more weight and generally help promote the professional development. 

Design Principles for Assessing Learning in Digital Badge Systems

If one is going to recognize learning, they must consider how they are going to assess that learning. WBA’s assessments involve a lot of formative feedback from experts, and some engagement in a peer learning community.

Enhance validity with expert judgement > Use human experts > Artifacts assessed by experts

Intended, enacted, formal

WBA intended for teachers’ designs and implementations to be assessed by ASHP teaching and content experts. Performance badges earned from these assessments were meant to recognize achievement in the “‘hard skills’ related to content mastery and inquiry- and literacy-oriented pedagogy. This track progressed in six stages: (1) Learn, (2) Plan for Class, (3) Teach, (4) Reflect on Lesson, (5) Design and Teach, and (6) Reflect and Revise. Teachers must have completed two cycles of stages 1–4 before proceeding to stages 5-6.

Content and pedagogical experts at the American Social History Project will be the main assessors of artifacts, however it may be possible for teachers who have completed the professional development and become master teachers to become assessors as well. The Performance Track stages have been simplified to (1) Know Your Stuff (study and take quiz on content and Common Core-aligned skills), (2) Get Ready to Teach (planning and modifying unit for class), (3) How Did it Go? (lesson reflection with experts and student work analysis), and (4) For Next Time (revising lessons based on reflection).

Teachers who earn Master History Teacher badges can mentor other teachers and help in assessing them. WBA feels that having the ASHP experts conduct the bulk of the assessments will work for now, but if the project grows substantially, the small number of content and pedagogical experts at ASHP may not be able to do all of the assessments on their own.

Use formative assessment > Use a combination of peer and expert feedback > Build community

Intended, enacted, formal

WBA intended to have “teachers join inquiry groups, undertake collaborative assignments, and post comments and resources that are rated as valuable by peers, as well as by ASHP and master teachers. Community badges [were meant to] represent the ‘soft skills’ associated with communication, collaboration, and peer engagement.” WBA also intended for teachers to evaluate each other’s “Exit Tickets” of their lessons, reflections and annotated student work. EDC had planned to assist ASHP in developing frameworks for conducting peer evaluations. For the ”Specialist” teachers’ Exit Ticket tasks would be assessed by AHSP experts and Master History Teachers. For the “Community” track, “teachers upload or make contributions that are assessed (‘liked’) by other peers using the criteria of ‘is this valuable to the WBA community or wider profession?’”

However, due to platform constraints, WBA decided not issue a peer-awarded badge or implement a “liking” system. Instead, teachers are required to participate in the community forums and offer resources and constructive criticism. They are also asked to respond to theory-related questions as opposed to known-answer questions asked by the ASHP experts. There are four community badges (I <3 Community; Community Member; Constructive Critic; I <3 Sharing). While teachers can still deem another teacher’s resource, insight, or constructive criticism helpful, no formal assessment of collaboration will be enacted at this point.

By shifting their practice from peer-awarded badges to expert-awarded badges for participating in a community forum, WBA may actually end up fostering more formative feedback among peers than they would have if peers had awarded more summative atta-boy badges and avoid an implicit expectation of reciprocation. WBA has implemented strategies for revision and feedback that is part of the four stages of the “Performance” track. In this process, ASHP makes comments, asks for further clarification and reflection, and then approves the work. Their criteria for their “Builder” badge is still being determined. Expert feedback is also available via live private chat boxes, though this may not be manageable as the project grows. 

Use Rubrics > Rubrics developed for assessment for specific artifacts > Assess Teacher artifacts

Intended, not enacted, not formal

WBA intended to develop specialized rubrics for the assessment of teacher artifacts in terms of the aligned standards  discussed above. As with many of the badge projects that attempted to create and apply rubrics, WBA found that such rubrics actually constrained rather than developed learning. As such, WbA ultimately did not develop this practice into their badge system.

Promote “hard” and “soft” skillsCombination of collaborative learning and discrete skills > Collaborative Learning 

Intended, enacted, formal

As stated above, the project intended to award collaborative learning efforts with a community badge but could not due to platform constraints. The project ended up deciding to design learning activities that required participation in critical discussion forums. This practice encouraged collaboration while avoiding the issues found in awarding badges in the platform.

Design Principles for Motivating Learning Within Digital Badge Systems

When a system recognizes and assesses learning, those actions impact the motivation of the learner. WBA has taken this into account as they developed ways to motivate learners without undermining the learning.

Recognize identities > Roles within a system > Role recognition

Intended, enacted, formal

WBA intended for each “track of badges [to correspond] to formal and informal identities within teacher communities. Teachers have opportunities to level up as a collaborator, ‘History Geek,’ ‘Common Core Writing Specialist,’ etc.”

While the different tracks do suggest different roles, the project is unsure how these roles will be taken up because they have just launched their badge system with users.

However, creating these different tracks indicates that WBA values the different roles a teacher can play as the develop professionally, so these roles may be crucial in both building a community of learners and encouraging growth. 

Provide incentives > Peer mentorship > Privilege of being a peer mentor

Intended, enacted, formal

WBA intended for Master History Teachers to become mentors for other teachers, earning “blazons” for mentorship and review of teachers and their work. Specialists could also review other teachers’ work. This is hypothesized to increase motivation for collaboration among teachers since they must mentor their peers.

The project still plans to for a system through which Master History Teachers can earn the privilege to mentor teachers and review their work. This will not only motivate collaboration, but may also help the ASHP experts as the project grows. This may motivate engagement by inviting awardees to join accreditation team and earn recognition for mentoring.

Use different types of assessment > Peer > Peer assessment

Intended, not enacted, not formal

Through the mentorship system and community badges, WBA intended to encourage peer assessment. Both Master History teachers and Specialists would have the opportunity to evaluate peer work. This was hypothesized to increase motivation for collaboration among teachers since they must asses and mentor their peers.

Due to platform constraints, community badges are only awarded for quantifiable traits like certain number of posts to the forum. There was an intention for community badges to be awarded by the computer system as a result of users marking posts as “useful” and for sharing resources. In a sense badges are still awarded by peers, but a computer system automates this process.

Master History teachers earn the privilege to mentor teachers and evaluate their work, but WBA is still looking for ways for peers to award each other badges.

Give badges for small accomplishments to hook in learners >Hook in learners

Intended, enacted, formal

Initially WBA did not state in their proposal explicitly that they would give small badges to get learners into the system. However, they decided that participants could earn a badge for simply signing up for the program.

Giving out these small badges seems to be a common practice across projects. These are not the kind of badges earners would necessarily push out to Facebook or their backpack, but they engage earners and get them started earning badges in the system.

Provide incentives > Prizes > Prize

Intended, enacted, formal

WBA intended for teachers who earned all five community badges to receive a “set of Who Built America? documentaries for their school library.” This was thought to motivate engagement by offering valued educational resources to awardees who earn entire set of badges.

While the community badges have changed, teachers who participate in WBA have access to resources, and Master History Teachers can post resources for other teachers to use.

WBA is still engaging in practices in which badge earners receive educational resources.

Competition >Use of point system > Points for community participation

Not intended, enacted, formal

WBA collaboration badgeWhile the use of a point system was not in place in the initial proposals, when the I <3 Collaboration badge was replaced with the I <3 Community badge, a point system was put in place whereby teachers could earn the new badge by earning points for participating in the community forums.

These points will not be visible to users, so that after they pass a threshold for participation, they will be rewarded with earning the I <3 Community badge. The community badges are earned by completing quantifiable tasks like number of posts to a discussion forum. 

Design Principles for Studying Learning Within Digital Badge Systems

The initial proposal did not outline practices for studying learning. However, as the project progressed, some opportunities for studying learning surfaced.

Study badge impact > Research OF badges > Teacher surveys

Intended, not enacted, not formal

Early in the project’s development, WBA intended to award badges for completing a survey.

As the project progressed, it was decided that badges would not be awarded for completing a survey. However, the project does intend to administer surveys to teachers to gather data.

Improve badge impact > Research FOR badges > Incentivizing participation

Intended ,enacted, formal

The program intends to do research around the motivational components of their badge system. WBA is actively defining how they are incentivizing participation and investigating how they should research this. It seems likely that this research will eventually perform a formative role, helping WBA improve the badge system, though it is yet to be determined whether WBA will use the information embedded in the issued badges to perform the research.

Improve badge impact > Research FOR badges >Providing feedback

Itended, enacted, formal

WBA is actively finding ways to provide feedback that is useful and used, and how they might gauge teachers’ responses to the feedback.

Challenges that this Project Faced

Who Built America Badges for History Education faced three major challenges as they implemented their digital badge system: The American Social History Project (ASHP) has been dedicated to history and history education since 1981, and have offered professional development for history teachers for about 20 years. Needless to say, moving the professional development online while adhering to the structures and methods the ASHP wished to use was not a simple affair. Additionally, the the use of digital badges has been a point of contention for some teachers, as they feel that this is yet another evaluation among many new evaluations of teachers being put in place. Finally, the platform in which WBA was initially working did not accommodate all of the badging practices the project wanted to provide for their earners. Late in the project’s development, they made the tough decision to change to a new platform developer and started again from the ground up; they are hopeful that this move may help facilitate the kind of engagement they were looking for in the beginning of the project.

Challenge Across Recognizing and Assessing Learning: Moving to Asynchronous Online Instruction

The American Social History Project (ASHP) has been offering professional development for history teachers for many years. They hold seminars that help teachers grow in their content expertise as well as in their classroom practice. WBA is an effort to move that professional development online. However, because the ASHP has been doing professional development for so long, they have developed a specific way of conducting their professional development and feel that content should be delivered in classrooms using specific practices. Moving this structure away from face-to-face cohorts to an asynchronous online setting using digital badges has been somewhat of a challenge, as the way ASHP wants to assess teacher learning directly impacts how they recognize that learning. In order to preserve what ASHP felt were essential elements of the learning program, they were forced to implement a complicated set of criteria including conversations with accredited experts. They aim to ensure the validity of the WBA credentials. The first step to earning the title of Master History Teacher is to earn a Lesson Builder badge, which entailed downloading one of the five lessons designed by ASHP, adapting it for one’s particular classroom, and implementing it. Teachers then submit work done by the students with their annotations, and discuss changes they will make the next time they teach the unit with an ASHP expert. The way these lessons are designed reflect the way ASHP thinks the content should be delivered, and since an expert discusses the delivery and modifications to the unit with the teacher, the ASHP maintains a fair amount of control over how content is designed both in that unit and in future units. This assessment has a formative function in that the teachers receive feedback on their design modifications and implementation, but the summative function of a “correct” implementation indicates that in order to recognize that the teacher has learned the skill of designing curricula with a badge, they must first master the ASHP design model.

Challenge for Motivating Learning: Teacher Resistance to Assessment

The challenge of assessing and recognizing mastery of the design model may be impacting the second challenge WBA faced, which was that a focus group of teachers initially rejected the idea of digital badges, indicating that they felt this was yet another evaluation to which they did not want to be subjected. WBA is being implemented in New York, which has just implemented a new teacher evaluation and accountability system that has teachers concerned. While WBA intends to help teachers meet their goals in their evaluations by enhancing their classroom practice, it seems that digital badges were initially seen as somewhat judgemental. Because of the constraints of the badging platform, which will be discussed below, all of the badges are very quantifiable; the badges recognize traits like how many hours a teacher spent in professional development, how many posts they made in the discussion forum, and the quality of the lessons they adapted and designed.

Challenge for Assessing Learning Through Peer Interaction

While a community exists, there is little lesson-building-collaboration that is fostered between teachers. This is a result of platform constraints rather than intentional design. Initially, WBA intended to include an “I <3 Community” badge, which would foster and reward collaboration. However, the constraints of the platform and platform developer precluded that badge from being developed. Instead, the project id fostering community through discussion forums in which teachers can ask questions of each other, as well as respond to questions and prompts with a “theoretical bent” (as opposed to responding to known answer questions). This helps the teachers engage in conversations and build relationships beyond their school walls, taking them out of the isolation of their own classroom and providing them with a support network as they grow as teachers.

Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance