The Design Principles Documentation (DPD) Project followed the Digital Media and Learning (DML) Badges for Lifelong Learning awardees as they proposed and implemented their badging systems. The DPD team categorized badging projects' practices in terms of using badges to recognize, assess, motivate, and study learning.
The Seed Phase presents the recognition principles we found and links to the specific projects that enacted those principles. The Sprout Phase presents external resources linked to the principles. The Bloom Phase presents the big ideas we learned about recognition and badges.
The Digital Media and Learning 2012 Badges for Lifelong Learning Initiative funded 30 projects to develop digital badge systems. We captured the emerging knowledge among these projects, as they designed, implemented and maintained their badge programs. First, we identified the specific practices that the 30 projects intended for recognizing, assessing, motivating, and studying learning with digital badges. We then clustered and organized those practices into general design principles in each of these four categories.
This Example shows what we learned about recognizing learning and details the nine general design principles we found for recognizing learning with digital badges.
DML grantees tasked with implementing badges are using the technology in their efforts to recognize, assess, motivate and study lifelong learning. The knowledge created via trial and error as the projects implement digital badge systems, however, will evaporate as projects evolve and teams dissolve. This means that this knowledge must be gathered up while it is being created, and it must be organized to be learned from and shared with others. A related problem is that this knowledge is rather tied to the context of its use, which means that this knowledge must be presented in ways that will help it make sense to the various people who want to use it, showing how it relates to appropriate contexts.
Generally speaking, the DPD Project is informed by sociocultural theories of learning that emphasize the contextual nature of knowledge. This means that we are not looking for evidence that badges “work” in a general sense. Rather we are trying to find particular practices for using badges that are appropriate for particular contexts. The goals were well suited for the approach to theorizing associated with design-based research. While the DPD team was not advising projects, the projects were iteratively refining their practices. By documenting this process more systematically across projects, we captured emergent local theories about recognizing learning with digital badges in particular contexts.
From the badge practices of the DML projects, we derived a set of design principles for recognizing learning, organized by order of prevalence as they appeared across the projects:
Use badges to map learning trajectories (17). Most projects used badges to organize learning by determining levels of badges or offering meta-badges. This principle was present across 17 projects. Among them, projects such as the National Manufacturing Institute badges and American Graduate organized levels of badges, while projects such as 4H/USDA Robotics Digital Badges, BuzzMath, and SA&FS Learner Driven Badges Project provided routes or pathways to learning.
Align badges to standards (15). Many used national or international standards to increase external value. 15 projects demonstrated this principle in their badge systems. Disney-Pixar and Global-Kids Hive employed standards that were internal to the community. In contrast, some projects such as PASA Pathways for Lifelong Learning aligned their badging systems to national or international standards, while other employed standards internal to the community and national standards, including Leverage for Digital On-ramps, Pathways to Global Competence, and 4H.
Have experts issue badges (15). Experts increase credibility and influence the usefulness beyond the issuing community. The project SA&FS Learner Driven Badges are credentialed by an external accreditation body, whereas projects like Mouse Wins! and Design for America are credentialed through the community. Pathways for Lifelong Learning and Who Built America? Badges for Teaching Disciplinary Literacy in History are credentialed by both the community and an external accreditation entity.
Seek external backing (11). Increases usefulness as name recognition is important to schools and employers. This principle was observed across eleven projects. Projects such as 4H are externally endorsed, which can boost how it is perceived by other organizations or institutions. The projects CS2N and Sweetwater AQUAPONS are externally valued, which has an impact on its usefulness.
Recognize diverse learning (10). Broad recognition helps legitimize what would otherwise only be implicitly noticed. Sweetwater AQUAPONS, Pathways for Lifelong Learning, and Planet Stewards recognized diverse forms of learning in their badging projects. For instance, Sweetwater AQUAPONS identified seven competencies to recognize, including Experimentation & Inquiry, Interpersonal Communication, Understanding Values, Civic Engagement, and Personal Development. Further, they created badge types without pre-determined criteria from which students can define new skills and recognize various kinds of learning.
Use badges as a means of external communication of knowledge and/or skills (9). Take advantage of unprecedented opportunity to present data and links to evidence of learning. Badges for Vets demonstrated this principle as they communicated to employers the knowledge or skills of veterans through badges, signaling the training they underwent that prepares them for job or career advancement.
Determine the appropriate lifespan of the badge (7). Provide credentials that have a lasting impact for an appropriate duration and permanent evidence that will be accessible forever. While the duration or permanence of the credentials were not explicitly discussed, some badging systems made the case that learners should be able to have credentials that always exist to represent specific skills and knowledge, including the projects S2R Medals and BuzzMath.
Recognize educator learning (7). Badges have a unique potential in this regard, often alongside issuing them to learners. Projects such as Pathways to Lifelong Learning and S2R Medals also recognized the participation of educators alongside the issuing of student badges.
Award formal academic credit for badges (2). While rare, a very consequential function of digital badges. In the projects Youth Digital Filmmaker and Pathways for Lifelong Learning, the badges awarded to students can translate into formal academic credit.
See Andi Rehak’s original post on Recognizing Learning with Badges for some discussion and more detail on how we derived these principles. Additionally, there is actually really great discussion about recognizing learning continued on our post about Studying Learning.
In this Example, we aim to document and organize the emergent processes and practices of the DML projects into general design principles for recognizing learning in the context of digital badge systems. Our goals are to share the general design principles to individuals who are interested in implementing digital badge systems. In the process, we have recorded the intended practices and goals of the DML projects.
Our immediate audience includes researchers and anyone designing a digital badging system for supporting learning. New projects can learn a great deal from the shifts projects made from their initial proposals to their enacted designs. Implementing digital badges to support learning is not merely a technical challenge, and important design decisions made up front can have long-lasting effects. We aim to help future badge projects make well-informed decisions in their design process.
Looking across projects, it was clear there were other forces impacting recognition, but there were challenges within recognition as well. Recognition practices are constrained by and in turn constrain assessment and motivation practices. What one decides to assess has an impact on what one recognizes, which influences the motivation of learners.
Within the area of recognition, there are tensions between different types: formal, informal, and crowdsourced. Formal recognition involves traditionally established forms of credentialing, whereas informal recognition provides new ways of recognizing learning and specific skills. Additionally, crowdsourced recognition includes socially defined peer recognition and the value that peers ascribe to badges. The value and credibility of the claims are constructed by everyone, and in this process individuals are coming to a collective judgement of what is valued.
Factors among the projects also affect recognition practices. The obvious challenge is that the decisions about what learning to recognize with badges might already be made. If there is already an existing curriculum, then figuring out a way to recognize it with badges will be a challenge. If projects are building the curriculum and the badges together, that probably introduces a different set of challenges (more at the HASTAC blog post). Challenges also surface in the design of badging systems, as badging practices can be influenced by the software platform. For instance, the design of badge systems can face constraints that arise from the development of a software platform to support the overall learning experience.
Like digital badges, this is uncharted territory. The closest example we know of is the Design Principles Database project led by Yael Kali and Marcia Linn. Their project also captured design knowledge across multiple projects and helped share that knowledge. To do so, they distinguished between specific practices within projects and more general principles across projects. The Badges Design Principles Documentation project is organized around this distinction as well (more at the HASTAC blog post introducing the project).
This project is progressing alongside insightful analyses and deep thought on recognition practices such as in Sheryl Grant’s HASTAC post on Michael Olneck’s paper "Insurgent Credentials." In addition, Sheryl Grant’s slide presentation "Credentialing: Diploma, Certificate, and Badge" from ITHAKA Sustainable Scholarship described the coexistence of diplomas, certificates, and badges and how they can complement one another. Moreover, Cathy Davidson’s HASTAC blog post "Badges Now" provided an overview of the function and potential of badges, describing the value provided by badges as alternative forms of assessment and credentialing, and Erin Knight discussed the topic of badges and credentialing in her post "Certification’ Revisited," describing the rationale behind badges as well as raising open questions for further exploration. In his HASTAC blog post "Evaluating Microcredentialing Public Networked Environments," Alex Halavais proposed research questions on microcredentialing and presented existing research that can inform the way that microcredentials are viewed.
The recognition practices of digital badge systems also relate to alternative credentialing practices, assessment, motivation, and research practices in technology more broadly, outside of badges.
We welcome any feedback and ideas on the design principles AND the Working Example. For example, if you realize we are using different words for something familiar to you, let us know. And if there is anything we can do to make it easier to find the information you need, please let us know. In particular we would like feedback from people with perspectives that are different from ours. You can put comments at the bottom of each entry.
The badge Design Principles Documentation Project team is led by Daniel Hickey, an Associate Professor in the Learning Sciences program at Indiana University. Learning Sciences PhD student Andi Rehak and Christine Chow are leading the recognition strand of this project, while Hickey, Rebecca Itow, Kat Schenke, and Cathy Tran lead the other strands. Nate Otto is working on the DML project Working Examples and other aspects of the project. Tara Kelly, Garrett Poortinga, and Thomas Smith are helping out with various things.
First, we clustered the specific practices of the 30 DML projects into the nine general design principles (described in the Seed phase). These principles (and the practices behind them) are a great context for organizing published research and other external resources so they are useful for others. We are just getting started, and hope others will contribute as our understanding of the practices, principles, and resources evolves.
The most interesting pattern was the way that the various specific practices for recognizing learning in the projects ended up clustering into nine more general principles. We knew that it would be easier to identify relevant research resources for a smaller number of principles.
Then we found that there was almost NO peer reviewed literature about digital badges and none specifically about recognizing learning with badges. But we did see lots of blog posts. Because badges are fundamentally about recognizing learning, that is one of the first things that people wonder about.
Our initial concepts for this concern the external resources that we have located that are relevant to each of the standards. So far these are as follows,
Use Badges to Map Learning Trajectories
Community Research Partners (2008) published the report “Ohio Stackable Certificates: Models for Success,” describing a series of levels in which learners can advance through a certificate system. This is similar to the levels of digital badges that map one’s learning trajectory. If you are mapping learning trajectories in K-12 math and science domains, you might consider research on learning progressions. In recent years, assessment and measurement specialists have made tremendous progress in creating very detailed research-based “maps” of the idealized learning in specific domains. Here is a good overview from ASCD (2007). As of 2013, the digital badges community was just beginning to ponder the learning progressions research. In a report on digital badges for the National Science Foundation, Riconscente, Kamarainen, & Honey (2013) included some consideration of learning progressions. More detailed research on learning progressions science education include articles by Wilson (2009) and Duschl, Maeng, & Sezen (2011).
Align Badges to Standards
In the report brief “Strengthening Transitions by Encouraging Career Pathways” by the Community College Research Center, Hughes and Karp (2006) describe alternative pathways for learners to transition into college and careers. They discussed the emphasis that the federal government places on maintaining high standards and accountability, and they delineated the connection of curricular alignment to career pathways. This can provide a framework for viewing the alignment of badges to specific standards and the function of badges to bridge training and learning to individuals’ entry into the workforce.
In the paper “RFC: An Open, Distributed System for Badge Systems,” Knight and the Mozilla Foundation (2013) described the value of standards alignment but also the appropriateness of designing badges apart from standards to represent specific skills. Badge systems have the flexibility to align their practices with curricular standards or chart their own course and recognize learning and skills not captured by standards.
Have Experts Issue Badges
In the article “Professional Certification,” Ulmer (2010) discussed the role of experts in the certification of individuals’ grasp of the concepts in a field. This had to do with the way that industrial technologists get certified for widely acknowledged credentials for the six-sigma certification. They described how a discipline-specific certification almost always requires input from recognized experts in that discipline. This sheds light on the function experts can carry out in strengthening the validity and credibility of badges.
As they discussed in the report “The Certification Advantage,” Foster & Pritz (2006) described the part of experts in the formation of standards and the certification of individuals’ skills and competencies. They provide an overview of the rise of certificates issued by certifying bodies and the principles by which they are regarded as meaningful. This has implications for what badges translate into in the job sphere when overseen or validated by experts.
Seek External Backing (Externally Valued)
Adams & DeFleur (2006, pdf here) described employers’ preferences of traditional degrees over online and compared the external value and acceptability of traditional and online credentials in terms of employability. The paper presented employers’ views of the quality of one’s education based on the reputation of the institution, suggesting greater applicability of badges that are externally valued.
Seek External Backing (Externally Endorsed)
Rivera (2011) discussed the prestige factor of degrees in employer’s screening and hiring decisions, suggesting an increase in value and acceptance for externally endorsed badges.
In “Educational Impostors and Fake Degrees,” Atwell and Domina (2011, pdf) described a social phenomenon of false claims made by individuals about their educational credentials. There are issues of costs and authentication of college degrees, whereas digital badges provide metadata and evidence of one’s learning. It seems likely that including external backing in this metadata is one way of addressing this issue.
Recognize Diverse Learning
In the publication “Training Tomorrow’s Workforce,” Lerman (2009, pdf) discussed the wide array of skills individuals learn in the context of community colleges and apprenticeships. It described the fit between skills learned in the programs and skills required in the workplace, and it explained the goal of federal government to invest in job training programs to prepare and certify individuals for skilled occupations. Badges can capture the diverse range of skills that are typically sought after in the workplace and that are not usually communicated through traditional credentials.
Use Badges as an External Means of Communication of Learning
In this article “What Do Educational Credentials Signal and Why do Employers Value Credentials?,” Arkes (1999, pdf) shed light on common characteristics of the hiring process, describing the way in which employers regard credentials and the inferences employers make about a candidate’s attributes and abilities. The article showed the general attributes that employers can infer from a candidate’s degree. This suggests the value that badges can provide in communicating specific skills and abilities.
Determine Appropriate Lifespan for Badges
In “Education and Social Stratification Processes in Comparative Perspective,” Kerckhoff (2001, pdf) discussed the transition from full-time schooling to joining the labor force, describing patterns between job placement and education from a comparative perspective. The paper described how many individuals continue to obtain new credentials after entering the labor force and explored the ways that factors of stratification and vocational specificity can influence the flow of individuals in the workforce. This highlights how important it is to consider the lifespan of badges.
Recognize Educator Learning
In her article “Recognizing and Enhancing Teacher Effectiveness,” Darling-Hammond (2009, pdf) described the effects of assessing and recognizing educators for continued learning and professional development. This paper takes into account the impact that such recognition can have on teacher effectiveness as it applies to the profession and thereby on the quality of students’ learning experiences. This article suggests the value that badges can provide in recognizing educator learning alongside the issuing of student badges.
Award Formal Academic Credit for Badges
In a concept paper from the WASC Accrediting Commission on the New ‘Ecology’ for Higher Education, Ewell (2010, pdf) discussed the various forms of learning that accrediting bodies will need to take into consideration, examining how learning that is present across multiple avenues can contribute toward individuals’ credentials. The paper described the transformation of the higher education landscape and what that means for credentialing practices. This highlights the possibilities of translating digital badges into formal academic credit as a way to recognize various learning trajectories.
Generally Relevant Resources for Recognizing Learning
False Claims of Credentials: In “Educational Impostors and Fake Degrees,” Atwell (2011) described a social phenomenon of false claims made by individuals about their educational credentials. There are issues of costs and authentication of college degrees, whereas digital badges provide metadata on and evidence of one’s learning.
Potential of Credentials to Open Up Avenues for Socioeconomic Attainment: Bills (2003) presentd and examined a set of theories that explain the relationship between credentials and socioeconomic attainment, including human capital, screening and filtering, signaling, cultural capital, and credentialist theories.
Sheryl Grant and Kristan E. Shawgo summarized several sources of literature, including the following references, in their extensive annotated bibliography.
As described in their bibliography, Baker (2011) explained the impact that the increase in occupational credentialing has on social stratification. The article explored different theories on credentialing from a sociological perspective.
Value of Credentials: Adams & Demalter (2010) discussed the distribution of credentials in the information technology field, pointing out that individuals with a degree could work with colleagues without it. On the other hand, the article discussed the inferences that employers tend to make from an individual’s formal education and its potential advantages.
In the article “Merit Badges for the Job Market,” Young (2012) described the effects that digital badges can have for professional development and the likelihood of employer take-up, including perceptions and ease of use of digital badges.
American Public Media released the piece “An Alternative to the College Degree?” in which Scott (2012) detailed the function of a college degree and the information or story that alternative credentials can provide by measuring specific skills that are not able to be captured by the traditional degree.
Validation and Reliability of Credentials: In her article “Badge System Design: What We Talk About When We Talk About Validity,” Casilli (2012) addressed questions and assumptions of the validity of digital badge systems, including the authority of the badge issuer and the issues of what badges represent and indicate about learning. It explored question such as how can the learning that is badged be validated, the credibility with which the badge is perceived, and consistency of the measurement of learning as it adheres to a set standard.
Costs of Earning Badges and Commodification of Learning: In his post “Welcome to Badge World,” Reid (2011) raised some issues of concern with digital badges, such as the commodification of learning experiences (i.e., equating learning strictly with income) and the costs that would be associated with earning badges.
We will continue to refine the design principles as the DML projects keep moving ahead with their implementation of digital badge systems. The design principles will take into account the enacted practices of the projects and reflect the process and approach of badge systems.
The “things that we are creating” here are a set of design principles and related resources. We are trying to make sure that our audience (badge designers) adopts these categories for their own work so that they can communicate what they want to find out and what they learn more readily. We are trying to frame the ideas of the academic research in ways that practitioners and access and use them.
Just as the blog post at HASTAC is attracting new comments and suggesting new resources, we hope that others will find this a natural place to share the resources that they locate. We also hope to scale by publishing a formal literature review on recognizing learning with digital badges in widely-read peer-reviewed research review journals.
We captured the recognition practices across the DML projects and identified a set of general design principles for recognizing learning with digital badges. As shown in the projects, digital badges can map out individuals’ learning paths and recognize certain skills and accomplishments. Many projects awarded badges to recognize learning and accomplishment that encompass a broad array of skills and abilities. Additionally, the recognition of learning interacts with and affects the strands of assessment and motivation of learners.
Our primary conclusions concerned the way that project contexts constrain recognition and the way that different goals for recognition might conflict with each other.
Recognition is Constrained by Context
In nearly every project we saw ways that project contexts constrain the kind of learning that can be recognized with badges. If there is an existing curriculum, then that really limits the learning you can recognize. For example, the 4-H developed badges for an existing robotics curriculum, while the Girl Scouts developed badges for a curriculum for building cell phone apps. That pretty much dictated the learning that the badges could recognize. In contrast, AQUAPONS had developed their urban aquatics curriculum and their digital badging system at the same time. They said:
“We learned that the development of the curricular content of our badge system and the development of the web platform are inextricably linked to one another. It is a lock-step dance between designing the scope and sequence of the curriculum and designing the web tools with which our curriculum will be delivered. At the outset, we determined that we should design our curriculum first, and then build the software to support that curriculum. However, as the software development progressed, we had to re-work much of our curriculum to fit the format of the software. Likewise, the changing curriculum then affected the initial software designs. An awareness of the evolutionary nature of the iterative process would have helped us manage our overall timeline better” (AQUAPONS HASTAC Q&A)
Even if there is not an existing curriculum, the context in which badges are being developed inevitably constrains the learning that can be recognized. For example, the project NatureBadges encountered certain challenges in designing their curriculum and badge system at the same time. They explained:
“[B]adges seem well suited to workshops or other contexts where they represent certification or participation in a program with a clear beginning, middle and end. For inquiry-based learning in a free-choice, non-linear, user-driven environment, badge design gets more complicated. Also, badges lend themselves well to a system built on levels of achievement, so it is important that content be organized in this way. . . .We’re undecided on whether it would have been easier to build the badge system after all of our content was created, instead of simultaneous development. On one hand, building the badge system deeply informed the way we thought about our activities and the activities we chose to pursue in important ways. On the other, if our content was in place before starting, we would have had a clearer set of parameters to work within” (NatureBadges HASTAC Q&A).
Recognition Practices Sometimes Conflict
Most of the projects struggled to prioritize which learning was most important to recognize; in some of the projects serious tensions emerged over competing recognition practices. For example some projects tried to align their recognition practices to existing standards while also using recognition practices to organize the learning objective in their own ecosystem. The project Who Built America? created a badge system in which teachers integrate and design lessons that are mapped onto the Common Core State Standards, and the project organized badge groupings that recognized educators’ skills in “iterative lesson design, professional engagement, content expertise, Common Core proficiency, and effective technology use” (Who Built America? HASTAC Post). At the same time, educators are recognized for their engagement and participation in building the community that develops in the emerging ecosystem. In the process of designing the badge system, the project encountered the question of how to communicate the value of and rationale for the badges in a way that would make sense to educators. In approaching this question, they worked toward the objectives of fostering the growth of a community of educators and demonstrating the use and relevance of the opportunities for professional development.
Some projects struggled in deciding how clearly defined the learning trajectory should be. If it is too well defined, then learners won’t learn how to manage their own learning. But if the trajectory is too ill-defined, learners will get lost. S2R Medals reflected on their process of designing and implementing digital badges, explaining “Looking back, we feel our badge ecosystem may be too complicated. If we were to start again, we would possibly simplify and create fewer badges to make it easier for people to work through the program independently” (S2R HASTAC Q&A). This suggests that projects needs to strike a balance between the specific structure of the learning trajectory and the latitude with which learners pursue their own learning pathways.
One success has been figuring out a way to present outside resources in a way that others who are interested in using our principles can use and reference them. This is a hard thing for many newcomers to knowledge communities to sort out. The amount of potentially relevant information is overwhelming. Relatedly, we also came up with some standards for presenting such information. We decided to not include formal references, but rather link to both (a) the paywall site for the full reference and (b) to a copy or the article or a pre-print whenever possible.
Another success has been linking the more specific practices in the working examples from the DML projects with the more general principles in the four working examples from the DPD project like this one. While we are still working this out, this seems like a big step forward for the WEx community.
There are constraints on the recognition of learning in badge systems. Most DML projects intended to capture a broad range of learning and specific skills. However, they faced a sizable challenge in assessing the diverse forms of learning (e.g., blogging, community organization skills, etc.) they aimed to recognize. Additionally, because of the resources required to implement the assessment and thereby recognition practices, many projects re-focused the scope and shifted their thoughts on the exact learning to recognize and the granularity of that recognition. For example, the EarthWorks project initially designed a system in which learners could create learning opportunities linked to badges for the community. The project explained their shift in focus, describing the badge-earning process for users on the EarthWorks site:
“After exploring background information and challenges, [badge] earners will identify a challenge they wish to make into a project. Projects may be of various media types, including community organizing, writing a blog, producing a dance, publishing a presentation, or other creative outlet that can be documented digitally. The prospective earner submits a link to their creative project, which is reviewed by volunteer, American Indian mentors. Mentors evaluate the project using a rubric that is tied to learning objectives, and if approved, a segment of the badge is awarded [through] an administrative section of the website. . . . We had originally planned that earners would be able to create and contribute learning modules for others and that learner-created projects contributed would be crowd-sourced to a group of three learners or participants who volunteer as mentors. We have simplified that approach in our prototype by starting with all Native American mentors and letting earners choose from a wide range of challenges presented to them” (EarthWorks HASTAC Q&A).
The system and infrastructure constrain the recognition practices of badge systems. Many projects addressed these challenges by adapting their workflow or focusing on an initial aspect of the project more deeply. In addition, a dilemma exists between linking digital badges to an existing curriculum and concurrently designing a curriculum and a badge system (more can be found at the HASTAC blog post on recognizing learning), and tensions are present within the area of recognition itself, including formal and informal credentialing practices.
Generally speaking, our biggest mistake was setting up a database system to keep track of our stuff. First we used Media Wiki and then we used TikiWiki. But both were really unwieldy and impossible for people from the projects to navigate. We probably should have used Working Examples from the very start. Most of them would have been finished much sooner.
Our biggest mistake with recognition practices was probably that we did not review the relevant research literature earlier. It took a long time to sort out the principles and they are still not cleanly aligned to the categories in the research literature.
As part of the DPD project, we are sharing the design principles for recognizing learning as a set of general guidelines that can adapted for use in various badge systems. Individuals can apply the principles to their recognition practices in the context of learning with digital badges. Some principles may be relevant only to certain contexts, and not every project will use every principle, but new projects can employ the principles to see how they work in specific contexts, and existing projects can use them to talk to other projects and refine their practices.
We will continue to refine the design principles and add to the relevant research literature as projects evolve, people connect, and conversations develop around these resources. The next step is to follow the projects that persist after their funding from the DML initiative has ended, and refine the principles further considering the successes of the different projects.
The design principles are shared with the broader community of individuals interested in creating digital badge systems. We are making our findings available for new projects to apply to their efforts and existing projects to discuss with other projects and fine-tune their badge systems.
As the recognition of learning constrains the other strands of badge projects, developing a fuller understanding of recognition practices can contribute to the assessment, motivation, and evaluation of digital badge systems. We want to foster conversations around the principles, research literature, and practices of badge systems so that these systems can grow and make informed decisions about their designs.
We hope that this set of principles for recognizing learning in badge systems helps future projects with design and that the knowledge gained in their new contexts advances our shared understanding of these principles. Our ultimate goal is for the database of badge design principles to be open for the continual refinement and circulation by the broader community.