SA&FS Learner Driven Badges Project

The SA&FS program at UC Davis is a new interdisciplinary major promoting holistic critical thinking about food systems. Funded by a MacArthur DML grant, SA&FS designed and implemented a badge system to complement the degree awarded through the program.

This Example shows how SA&FS uses digital badges compliant with Mozilla’s Open Badges Infrastructure with their program. This analysis is provided by the badges Design Principles Documentation Project at Indiana University.

Comments (
0
)
SHOW COMMENTS
    Seed is your jumping off point, building the foundation for your project. It helps you define your idea, vision and audience.

    The Sustainable Agriculture & Food Systems is a new interdisciplinary major that integrates a portfolio system featuring digital badges. UC Davis’s Agricultural Sustainability Institute (ASI) and College of Agricultural and Environmental Sciences developed “a model of learning, participation, and assessment focused around high-level ‘core competencies’ that bridge classroom and real-world experiences, academic investigations and concrete skills. We believe this model has the ability to train leaders who will transform the food system, changing the way each of us eats and lives” (from the original proposal).

    A distinguishing feature of the Sustainable Agriculture & Food Systems badge system is that students have a role in defining the criteria for the badges they attempt to earn. While badges that represent named “core competencies” are identified and include some program-related requirements, the the criteria for all of the other badges in the system are not defined. Students each develop their own criteria, and in claiming a badge, open up both their achievements and their criteria to feedback by peers and mentors (proposal).

    This Example Seed describes how SA&FS originally intended to use digital badges to recognize and reward learning by students in the program. The purpose for implementing badges is to allow students to create a portfolio of work that demonstrates the skills and competencies developed across many educational experiences. The intended practices below reflect the DPD project’s analysis of the SA&FS program’s original thinking about implementing badges.

    Comments (
    0
    )
    SHOW COMMENTS

    The new major aims to create graduates who can lead in shaping the future of agriculture. “ASI carried out a Delphi survey of academics, practitioners and students that identified which skills, content and experiences were most essential to prepare students to lead in the field of Sustainable Agriculture and Food Systems”. The new program intends to create a system to ensure that program requirements are being met “while preserving the individualized, dynamic nature” of each student’s achievements in order to match the needs of the careers in the field and the range of learning opportunities available (Proposal).

    In addition to explicitly allowing students to “find and represent learning experiences outside the university, to help them build learner identities and communities, and to clarify learner goals and gaps”, badges can be “a system that responds not only to the diversity of learners and diversity of problems to be solved, but also to the process of learning in the real and changing world” (Proposal).

    Comments (
    0
    )
    SHOW COMMENTS

    “Our main goal is to support a learning process that is student-driven and outcomes-based, in a way that helps faculty and other mentors and advisers interact effectively with students and create a learning community that can foster and dynamic and networked experiences for everyone in the system “ (HASTAC Q&A).

    The SA&FS program wants to help students and faculty better understand the process of competency development and reflection as they integrate diverse learning experiences. The idea is to “create a system that helps individuals track progress over time and beyond the traditional grading and evaluation system contained within the course structure; this will provide students with a more comprehensive view of their growth and development and support them in the creation of an individual portfolio of work and collections of badges.” (HASTAC Q&A)

    Comments (
    0
    )
    SHOW COMMENTS

    “Our badge system is designed first with the learner in mind“ (HASTAC Q&A). Students who join the SA&FS program are often those who want to work in farming, not necessarily those who want to spend lots of time organizing a digital portfolio, so the program needs to design a tool that is both easy to use and motivates this audience to flesh out their achievements and advance their learning (DPD Interview).

    Comments (
    0
    )
    SHOW COMMENTS

    INTENDED PRACTICES FOR USING DIGITAL BADGES

    The Design Principles Documentation Project at Indiana University examined the original SA&SF proposal to identify four types of intended practices for using digital badges. 

    Intended Practices for Recognizing Learning

    Developing Badge System and Curriculum Together
    The interdisciplinary Sustainable Agriculture and Food Systems major is new to UC Davis, and the programs that created it were able to think about their new opportunities for recognizing learning as they decided what learning they hoped to emphasize. Designing a competency-based recognition system was appealing because SA&FS organizers felt the skills important to the future of agriculture and food are not exclusive to one course or experience.

    SA&FS created formal curriculum requirements and identified 7 high-level Competencies to be represented in a system of badges:

    • Systems Thinking
    • Experimentation and Inquiry
    • Interpersonal Communication
    • Understanding Values
    • Strategic Management
    • Civic Engagement
    • Personal Development

    The badge program is designed as complementary to the formal degree not as a replacement, a structure that allowed SA&FS to recognize diverse achievements recorded in a portfolio (DPD Interview).

    Self-Selected and Self-Created Badges
    Besides the 7 main competency badges, students have freedom not only to select, but also to create, badges to pursue. SA&FS generated a number of category-level badge definitions of “Skill Badge", 'Knowledge Badge", Experience Badge" and "Honor Badge" that represent the most important skills, content, and experiences for students in the major to develop or be exposed to, as determined from a survey of stakeholders in the field.

    These badge definitions (Skill, Knowledge, and Experience) serve as templates for students to create their own specific badge, selecting its name, criteria, and design. Students would then point to evidence of their achievement from their portfolio (DML Proposal).


    Intended Practices for Assessing Learning

    The Sustainable Agriculture & Food Systems program aims to recognize learning through student portfolios containing badges and evidence to support them. Within each course, internship, or educational experience, instructors are responsible for assessing work to their own standards. Students then submit work to their portfolios, identify examples corresponding to their desired badges, and this work is judged by their mentors as to whether or not it fulfills the criteria of each specific badge.

    Involve Students at a Granular Level in Badge (Assessment) Design
    SA&FS intends students to select portfolio pieces that match the criteria of a badge, whether that badge represents one of the core program competencies or is self-designed. Students are the first assessor of their work in deciding if they qualify for a badge. Program experts validate their decision afterward.

    Assessing to Levels of Achievement
    Assessments get more involved as students progress through the program and develop their badged competencies. As students develop their skills, knowledge, and competencies, they indicate their level of understanding from novice to expert, with 3 intermediate gradations. When they assert expert level, peers and mentors can provide summative assessment to validate that achievement.

    Badge level selector interface.
    For each badge, students indicate their current level of expertise. The badge updates to take new information into account (SA&FS Design Sketches)

    Additionally, the SA&FS program intends to attach preconditions to some badges to help guide students through levels of achievement.

    Promotion of Different Types of Knowledge and Assessment
    Through validated self-assessment of e-portfolios, SA&FS hopes to promote a holistic assessment program. Students are all assigned an advisor in the program who helps provide formative feedback (See the DPD Project's Assessment Working Example: Seed phase) that helps students develop how they present material to be judged against criteria for particular badges (summative assessment).

    E-Portfolios
    One of the key features of the badge system is that it is a dynamic, learner-driven experience, focusing on the development of a portfolio of content, skills and experiences. Items are organized for viewing around the badge system, as well as other methods of sorting. See the Nov 2011 SA&FS presentation for sketches of how portfolios are intended to be organized and viewed, including the sketch below:

    Provide Feedback
    Students collect feedback through the portfolio system: “As they develop, learners receive formative feedback from peers, faculty, and community partners with whom they have worked. When a learner has reached what he or she believes is ‘expert’ level in a particular ‘competency,’ members of the community can agree or disagree with the expert designation and provide targeted feedback. Viewers see how many reviews a learner has received, but not all of the review contents. By making the content – not just completion – of learning experiences evident, this integration of self, peer-, and practitioner-assessment can guide future learning as learners recognize gaps and strengths.” 


    (SA&FS Design Sketches)


    Intended Practices for Motivating Learning

    The element of user-driven learning and choice is woven throughout the SA&FS badge project. The system allows students to design/define new badges or select pre-designed badges, as opposed to only pursuing badges that an instructor or supervisor has chosen. The initiative presumes that students are more likely to be motivated to participate in a system that they helped create, leading to more user buy-in.

    Peer Assessment
    Students receive formative feedback as they develop, but when they get to the point of claiming an “expert” level badge in a competency area, members of the community can chime in to agree or disagree with the assertion. Aggregate figures of endorsement are publicly available, adding to badge credibility. Students may be motivated to aim for this level of recognition but also may only wish to submit to summative judgement when they truly feel qualified, hopefully motivating

    Student-Selected Badges
    SA&FS hopes that the ability to select and create badges will motivate students to think holistically about the education they gain and how their portfolio could create a cohesive picture of a skilled graduate.

    Self-Assessment
    Choosing when to self-assess at an “expert” level will motivate students to set and accomplish goals, because of the higher stakes of the community’s judgement that is allowed by that designation.

    Choice of Publicly Displayed Information
    The program intends to allow students to show or hide the badges they earn and those they are working toward, which should encourage them to think about how their accomplishments fit together.

    Evidence for Collaborators
    SA&FS intends to create a resource that can help open opportunities for students in the future by showing potential collaborators that they would be valuable partners. Organizers hope students recognize this possibility and are motivated to use the portfolio tool to showcase their skills.


    Intended Practices for Studying Learning

    Like most of the DML2012  projects, the SA&SF proposal did not include a specific plan for systematically studying learning or evaluating their program.

    Comments (
    0
    )
    SHOW COMMENTS
    Sprout is the story of your project, its process and evolution. It illustrates how the work's getting done.

    ENACTED PRACTICES FOR THE SA&FS BADGE SYSTEM

    The Design Principles Documentation Project interviewed Joanna Normoyle and other project members in November 2012 and March 2013 to learn how the intended practices in the proposal (detailed in the Seed phase) were being enacted.  These enacted practices reveal how aspects of the project context supported some intentions while requiring others to be changed.
     

    Enacted Practices for Recognizing Learning

    Developing Badge System and Curriculum Together
    The badge and portfolio tool will roll out to students with the Fall 2013 term and feature the SA&FS-developed competency badges as well as the capability for students to create their own badges in Skill, Knowledge, and Experience categories (DPD Interview).

    Badge Issuer
    UC Davis will have no trouble issuing badges to its students, though some questions exist around the fact that the platform requires a Davis log-in to use the portfolio tool, which could limit the growth of the project and integration with community partners (DPD Interview).

    Self-Selected Badges
    Students have a private work-in-progress section to collect the badges they wish to earn as well as the section of their portfolio showing finished accomplishments (DPD Interview).

    E-Portfiolio Balance
    As development continues and focuses on the user experience, SA&FS is working to balance being transparent and causing information overload by displaying more than the most relevant information about each achievement (DPD Interview). Developers are working with student testers and advisors to determine what should be viewable to provide a rich picture of students’ accomplishments and skills (HASTAC Q&A).
     

    Enacted Practices for Assessing Learning

    Involve Students at Granular Level in Badge Design
    As planned, upon rollout, learners may categorize, name, and define criteria for their own badges, customizing them to their needs and experience (DPD Interview).

    Assessing to Levels of Achievement
    More information about the leveled assessment functions will be available after students have been earning and peer-assessing badges for several terms.

    Promotion of Different Types of Knowledge and Assessment
    Students’ knowledge, skills, and experience come from a variety of classroom and outside work, such as internships. Classroom work assessment is well-developed. SA&FS will see how the opportunity to award badges affects assessments of student performance and learning through internships. The spreadsheet of badge types SA&FS developed shows various badge types issuable for this outside work, like a badge endorsing a student’s skill, as assessed by an outside specialist, or an internship badge that is attached to a particular competency, skill, or issue area within the e-portfolio system.

    E-Portfolios
    E-portfolios have been developed to reside within the UC Davis course management platform, which is closely tied to students’ assessed coursework. Advisors provide formative assessment as portfolios develop (DPD Interview).

    Providing Feedback
    Students create the badges they think they qualify for and send in a request. Peers and experts then validate or deny that request. Faculty can provide comments and start a dialogue with the student, fostering a mentor/mentee relationship (DPD Interview).
     

    Enacted Practices for Motivating Learning

    The badge system rolls out for students in Fall 2013. The intent for the portfolio and achievement system is to motivate students to both think holistically and specialize in their learning, making connections between experiences and competencies. More information will be available as students use the system over the coming terms.

    Peer Assessment
    After badges accumulate in student portfolios, SA&FS will be able to investigate the motivational effect peer assessment has on learning within the major, particularly as students discuss the progress of their portfolios with advisors (DPD Interview).

    Choice of Publicly Displayed Information
    Program staff and faculty advisors will see students’ badges in progress as well as completed badges, though public viewers will only see published content. Students will be able to solicit feedback from these experts on this information as they work (DPD Interview).

    Evidence for Collaborators
    Badges will be a sign to those who might collaborate with a student about their fitness for the work. The opening of future opportunities with the audience of the badge system will motivate them to carefully develop their portfolios.
     

    Enacted Practices for Studying Learning

    Like most of the DML 2012 Projects, most of the effort in this one year project was devoted to getting the badge system up and running.  The project had neither time nor resources to devote to systematically studying learning within the badge system, evaluating the program, or systematically studying the usefulness of particular practices for implementing badges.

    Comments (
    3
    )
    SHOW COMMENTS

    As noted in the Seed phase, this presentation of sketches shows the portfolio tool SA&FS imagined and explains its value for different use cases. 

    Comments (
    0
    )
    SHOW COMMENTS

    The goals for the project have not changed, though the schedule for accomplishing them has developed across initial and continuing phases. “We are currently asking ourselves what we’d do differently, and most of it has to do with starting from a user experience perspective, and including the faculty user experience in that thinking. We made some assumptions that faculty would be willing to deal with a less polished interface and a clunkier, back-end experience. Given the demands on their time, that doesn’t turn out to be a fair assumption.” (HASTAC Q&A)

    Comments (
    0
    )
    SHOW COMMENTS

    The project aims to perform extensive testing of the user experience and roll out the program to students full-scale for the fall of 2013. At all continuing stages of development, SA&FS will keep an eye toward “how the system might really be built upon and/or adopted by others” (HASTAC Q&A). Usage research will determine how well the system motivates learners to develop along prescribed and self-generated learning pathways, give and receive peer feedback, and share badges with others (HASTAC Q&A).

    Comments (
    0
    )
    SHOW COMMENTS

    In the original project proposal, organizers were already aiming to develop a system that could expand beyond the SA&FS program: “The badge system has a waiting network of students, educators, researchers, and partners. Due to the interdisciplinary and experiential focus of the SA&FS major, this network extends throughout many departments at UC Davis, as well as to community organizations, institutes, practitioners, and educators across California and around the world.” They also noted possibilities for secondary education and planned their badge system “with these multiple sites and audiences in mind” (DML Proposal).

    “We envision this system to be useful, at a minimum, for all majors at UC Davis, as well as majors similar to SA&FS at higher ed institutions across the country.” (HASTAC Q&A).
    “We're exploring the potential to develop learning pathways from K-12 into post-secondary institutions throughout the land grant university system” (HASTAC Q&A).

    Comments (
    0
    )
    SHOW COMMENTS
    Bloom is a celebration of your end-product, a reflection back on your process and a look forward into the future.

    Give us some time. We're not in the bloom phase yet.