The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014.  This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, LevelUp had not issued badges or developed a badge system beyond its pilot. Based on this information, we have classified this project as a suspended (rather than an implemented or partial) badge system.


The badging initiative LevelUp is the result of a partnership between Adams County District 50, EffectiveSC, and Intific. LevelUp is a web-based platform for students to map the progression of their learning experiences on their own, personal competency map, with the capacity to take effect across systems and institutions, including “K-12 schools, colleges, extra-curricular activities, or job training” (HASTAC Q&A).

In 2009, Adams County District 50, located in Colorado, implemented its competency-based education system, which involves a learner-centered focus in the organization of assessment, standards, and instruction that emphasizes students’ demonstration of proficiency and competency in one level before advancement to the next. The LevelUp badging initiative conducted a pilot in the district and leveraged its competency-based model.

LevelUp tested the badging system in the spring of 2013. The initiative stated that it observed a classroom use the system substantially in the fall of 2013 (HASTAC Q&A). The project does not have formal practices, as it discontinued after the pilot.

Alignment to pre-existing curriculum

The project aligned badges to an educational game that initially had a math focus. LevelUp planned to bring the educational game into the curriculum and record how the learning that occurred through playing aligned to the curricular goals already in place. The technology partner, EffectiveSC planned to provide the ability to programmatically import achievements into the system, to allow schools to “push relevant milestones and badges back to educational management systems (like LevelUp)” (DML Stage 1 Proposal)

This project enacted this practice and then realized it had detrimental motivational effects, so they discontinued it (DPD Initial Interview).

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. Below each heading, we indicate what stages this principle appeared in the project: as an (a) intended practice, (b) enacted practice, or (c) formalized continuing practice.

Design Principles for Recognizing Learning

The LevelUp badging system aligned to the Common Core State Standards, providing a common framework for understanding and interpreting the claims made by badges. In addition, the badges are mapped to Marzano’s scoring guide, which represents a progression of students’ learning. Experts award the badges, boosting the credibility of a badge, and the project tied formal academic credit to badges that students earn.

Align badges to standards > National or international standards > Common Core mastery

Intended, enacted. not formal

LevelUp intended each badge to represent mastery of a single Common Core State Standard. For example, they proposed that, “the Outpost badges will be awarded for students’ success at applying algebra and geometry standards from the 8th and 9th grade math common core” (DML Stage 1 Proposal).

As initially enacted, LevelUp teachers approved practice evidence learners submit for a specific CCSSS.

Use badges to map learning trajectory > Level badges > Badges indicate competency level aligned to Marzano’s scoring guide

Intended, enacted. not formal

The project intended to assign the badges to one of Marzano’s scores, providing an indication of progression on that common core standard. The LevelUp initiative asserted, “To track the progress or growth of those standards, we support Dr. Robert Marzano’s scoring guide. He identifies this progression along a 4-point scale that aligns with the way humans learn, and is shown to be more accurate and transferable than percentage scores” (DML Stage 2 Proposal).

As implemented, there were bronze, silver, and platinum badges to indicate levels of mastery within each Common Core standard.

Have experts issue badges > Credentialed via external accredited entity > Accredited teachers award badges.

Intended, enacted. not formal

LevelUp intended for accredited teachers to award badges to students. In addition, the project planned for the badges to receive external accreditation to enhance the credibility of a badge. In the badge system, teachers issue the badges after reviewing the official evidence. Badges come with a Duke University certification, strengthening the level of trust in a badge that a teacher would have (DPD Initial Interview).

Award formal academic credit for badges > Badges for credit

Intended, enacted. not formal

An important part of LevelUp’s system involved students earning badges for the competencies they needed to develop at school, for which they received formal academic credit. They intended for the badge system to meet the needs of the schools for reporting of evidence:

“For those schools that accept the badges as official credit, like Adams County District 50, the reporting will allow the teacher to evaluate the content, assess what the student truly learned, and award credit that eventually applies to graduation. LevelUp will use crowdsourcing to help validate the content” (DML Stage 2 Proposal).

Design Principles for Assessing Learning in Digital Badge Systems

The project integrated aspects of formative assessment, as evidence is classified into two types within the badge system: practice evidence and official evidence. Students and peers would submit practice evidence of their learning, while teachers would review students’ work and performance and approve the official evidence of students’ competence that leads to the issuance of a badge.

Use formative functions of assessment > Use a combination of peer and expert feedback > Peers review practice evidence, and teachers approve badges

Intended, enacted. not formal

LevelUp intended learners to act as mentors, with the capability to create badges and activities. Peers are supposed to assign badges to practice evidence, followed up by teacher review where a teacher flags badges and accepts or overrides it as “official evidence” (DPD Initial Interview).

As implemented, only teachers approved official evidence for badges (DPD Initial Interview).

Involve students at a granular level > Learning pathways and badge design > Mentors earn ability to create badges and learning activities

Intended, not enacted, not formal

Lance Christmann, Executive Director at EffectiveSC and then-Principal Experience Strategist at EffectiveUI, asserted that this was a long-term hope and aspiration of the badging system, but the project was not able to secure the funding to build this functionality (DPD Follow-up Interview).

Enhance validity with expert judgment > Use human experts > Teachers as validator

Intended, enacted. not formal

Students can log practice evidence into the system, give students rubric, resources, and tool based on their self-assessment and other assessments about them to move forward, enables to have right support structure in place to progress LevelUp intended the system to allow creation of badges outside the original set, including “learner competencies (i.e, leadership, collaboration) and identities (“Outpost Captain”) not recognized by ‘standards,’ while still mapping to the state standards” (DML Stage 1 Proposal).

As enacted, the teacher was the validator, but not the sole bearer of knowledge. The teacher awarded badges once they sat down with the student and made sure the student didn’t “game” the system (DPD Initial Interview).

Align assessment activities to standards: create measurable learning objectives > Common Core State Standards > Map competencies to standards

Intended, enacted. not formal

LevelUp intended the “catalog” of badges to “tie directly into Common Core standards to allow teachers and students to search for content based on those standards, but it will also allow users to search for other badge data like learning modality” (DPD Stage 1 Proposal).

As implemented, LevelUp allowed assessing activity with other standards in mind as well, so the CCSS were one type of possible standards to align badges to (DPD Initial Interview).

Use rubrics > Rubrics developed for assessment for specific artifacts & General rubrics > Rubrics used to assess learning

Intended, enacted, not formal (Rubrics for specific artifacts)

Intended, not enacted, not formal (General Rubrics)

The project intended to leverage rubrics to assess learning. They planned for teachers to assess competency based on adaptation of general competency rubric frameworks, like the Adams 50 competency based education system and integrate Marzano’s scale. LevelUp proposed,

To track the progress or growth of those standards, we support Dr. Robert Marzano’s scoring guide. He identifies this progression along a 4-point scale that aligns with the way humans learn, and is shown to be more accurate and transferable than percentage scores…Sometimes there will be content that doesn’t explicitly teach something in a way that fits this 4-point scale, but rather supports a standard. … This type of content can be tagged with a standard, but will be identified as “support” rather than given a 1-4 score. (DML Stage 2 Proposal)

As carried out in the badge system, rubrics are competency based and generated ad hoc by individual teachers. However, the project is looking to standardize the process and pull the rubrics into a system. Rubrics exist for competencies in grades K-12

Design Principles for Motivating Learning

The badge system offered privileges, such as peer mentorship and prizes, strengthening the motivational effects of the badging system. LevelUp proposed two types of rewards distinct from the badges: awards and learning goals. The project aimed to encourage motivation in learning by nurturing students’ self-growth, stimulating competition, and fostering collaboration.

Provide incentives > Peer mentorship > Peers earn the privilege of mentorship

Intended, enacted. not formal

The project intended peers to go through a number of “gates” before becoming leaders and mentors within their class or group, and their credibility as peer assessors is enhanced, even though they do not gain the ability to make the final determination about awarding a badge or approving “official evidence” (DPD Initial Interview).

Set goals > User-determined learning trajectory > Promoting learning goals

Intended, enacted. not formal

The leveled structure of the intended badges, aligned to development in competency allows students to see how they are progressing. The “rank” of badges lets them set goals and move forward.

The project proposed the development of “Achievement Awards,” or “awards in which the individual learner or group of learners only have themselves to compete with. The award exists for the learner(s) to accomplish whenever they satisfy the criteria, and they are not competing with anyone else for that award” (Christmann, 2012, p.5).

In addition, LevelUp wanted students to be able to track their own progress, but also to set goals in terms of creating challenges for themselves.

Provide incentives > Prizes > Achievement awards

Intended. not enacted, not formal

There was an intent to occasionally offer awards in the form of prizes: “While there is implicit reward within badges, LevelUp allows for explicit rewards to be defined as well. There are two forms: Awards and Goals” (Christmann, 2012, p.5).

Stimulate competition > Scarcity of badges > Competition among learners

Intended, not enacted. not formal

LevelUp planned to introduce “Competitive Awards…in which a select few or only one will win. The amount and rank of winners will be defined in the criteria” (Christmann, 2012, p.5).

EffectiveSC Executive Director, Lance Christmann, described, “Partially because within the mastery-based environment that we’re working on, kids all progress at their own pace. They didn’t want to, at this time, turn it something that requires kids who are at totally different places in their personalized learning to be forced into the same competition with kids a little ahead of them or somebody below” (DPD Follow-up Interview).

Promote collaboration > Badge for group accomplishment > Group recognition

Intended, not enacted. not formal

LevelUp aimed to provide badges to groups of learners in addition to individuals. As it operates, the badging system awards badges to only individuals (DPD Follow-up Interview).

Engage with the community > Involvement in digital community > Developing learner-mentor relationships

Intended, not enacted. not formal

LevelUp intended that “In the absence of quality teachers or involved parents, the platform will allow access to mentors and quality content providers from around the world” (DML Stage 2 Proposal).

Instead, in its initial enactment, the system focused on mentorship relationships internal to the classrooms, wanting “to take advantage of things like stackoverflow where certain types of mentors naturally arise in the system, as mentors create activities, the more teachers take activities as student-mentor or parent-mentor, and actually give official evidence, further validate that you are a mentor, elevate those who are contributing into mentor status” (DPD Initial Interview).

Build outside value for badges > Evidence for outside opportunities > Connecting learning between school and everyday experiences

Intended, enacted, not formal

LevelUp intended to “use badges as a transport vehicle between the greater world of learning and their inside competency-based system” (DPD Initial Interview).

Design Principles for Studying Learning

LevelUp intended to maintain a database of activity so that they would have the capability to analyze data for research practices if funding arose to cover the costs, but they did not intend or pursue any specific research designs.


Adams County School District 50. (n.d.). Adams County School District 50. Retrieved from


Christmann, L. (2013, February 14). DPD Initial Interview.

Christmann, L. (2014, April 30). DPD LevelUp Follow-up Interview.

LevelUp. (n.d.). LevelUp DML Stage 1 Proposal. Retrieved from



LevelUp. (n.d.). LevelUp DML Stage 2 Proposal. Retrieved from



Project Q&A with: LevelUp. (2013, July 1). HASTAC. Retrieved from



Recognizing Principles Assessing Principles Motivating Principles Studying Principles
Specific principles: Specific principles: Specific principles:

Loves open education, Open Badges, free culture, Progress of the Useful Arts and Sciences, people-powered politics, and local food production. Coordinator for the badges Design Principles Documentation Project at Indiana University.

Leave a Reply

Your email address will not be published. Required fields are marked *


Mozilla Open Badges
Digital Badges are web-enabled credentials of learning or accomplishment. -Erin Knight, director of the Badge Alliance
Badges contain detailed claims about learning, links to actual evidence of learning, and they're shareable over the web. -Dan Hickey, DPD Project Lead Investigator
To me, digital badges represent the bridge between formal learning & informal structures. -Alex Halavais, DML research competition winner
Open Badges can help people tell a verifiable story about their accomplishments. -Nate Otto, DPD Project coordinator
Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design. -Carla Casilli, Director of Design + Practice at the Badge Alliance