Exemplary Initiatives

Sign up here and receive our online newsletter via email!

Preview a previous issue:

November 2005

Welcome to Exemplary Initiatives online!

Exemplars is a community of users dedicated to helping schools become successful in standards-based performance assessment and instruction. Our monthly newsletter features short pieces from Exemplars users, as well as brief reflections on current education issues and trends that impact standards, assessment and instruction.

If you would like to contribute, or have comments you would like to share, please get in touch with us at info@exemplars.com. We look forward to hearing from you.

In This Issue:

The Next Big Thing: Formative Assessments

At a meeting of educational publishers a few years ago, several of the speakers predicted that formative assessment would be "the next big thing." A recent perusal of ads in educational journals and test publisher web sites indicated that although there is a wide variation in what people believe constitutes formative assessment, their predictions were right. Although, I question much that is marketed as "formative assessment" - Does it pass the test?

Formative assessment has been around for years. Why the interest now? First, No Child Left Behind has placed a premium on outcomes. Schools are punished if their students do not reach specified levels of achievement on standardized tests (summative assessments). Improving student performance on these tests has become THE priority in many schools, and formative assessment is considered a way to measure and enrich performance.

Formative assessments provide vital feedback to students and information to teachers that they use to improve teaching and learning throughout the year. Formative assessment is contrasted with summative assessment, which is typically done at the end of the year, and whose focus is accountability, and assuring that students have achieved a certain level of performance. Summative assessments are higher stakes, the credentialing of students, for example, or determining rewards or sanctions for teachers or schools. Formative assessment involves more teacher guidance throughout the year, and nurtures students' ability to articulate their understanding of the curriculum.

The conditions for successful formative assessment include:

  • The student and teacher share a concept of what constitutes quality work.
  • Both student and teacher can compare the student's performance to their standards.
  • Teaching and learning activities are adjusted so that the gap between the standard and performance is closed. Both teacher and student can take appropriate action to close the gap.

Teachers have been doing this forever, but not necessarily well. In reviewing the research Black and Wiliam found that many teachers lack the knowledge to implement formative assessment successfully in their classrooms. The assessment and teaching strategies most closely tied to successful formative assessment are:

Effective questioning - Asking meaningful questions, increasing wait time for student answers and having rich follow-up activities that extend student thinking. "Put simply, the only point of asking questions is to raise issues about which a teacher needs information or about which the students need to think." (2) p.13

Appropriate Feedback - In reviewing the research Black and Wiliam found that giving grades does not improve performance. Using tasks and oral questioning that encouraged students to show understanding, providing comments on what was done well and what needs improvement, along with guidance on how to make improvements should be focused on instead of grades.

Peer and Self-Assessment - Peer assessment and self-assessment "secure aims that cannot be achieved in any other way." (2) p.15 Achieving success requires that students have a clear understanding of the standards and be taught the skills of peer and self-assessment.

While this issue of Exemplary Initiatives does not explicitly focus on formative assessment, a number of articles discuss different facets that are relevant to it. Tracy Lavallee identifies ways of teaching students to become more proficient self- and peer assessors using Exemplars rubrics for young children as well as benchmark papers. Lori Jane Dowell Hantelmann describes how she used what she learned in her students' math journals to change her teaching and how her students learned to reflect on their problem-solving strategies. Cecile Carlton talks about how teachers in Nashua, NH used Exemplars rubrics to think about different ways of assessing student work and were able to use the rubrics to convey their expectations to their students. Finally, Becca Lindhal of the Archdiocese of Des Moines, IA, describes the value of teachers working together to assess students' solutions to Exemplars problems.

We are currently creating a permanent section at www.exemplars.com devoted to formative assessment resources.

(1) Wiliam, P. B. a. D. (1998). "Assessment and Classroom Learning." Assessment in Education: principles, policy and practice 5(1): 7-74.

(2) Paul Black, C. H., Clare Lee, Bethan Marshall, and Dylan Wiliam (2004). "Working Inside the Black Box: Assessment for Learning in the Classroom." Phi Delta Kappan: 9-21. Back to top

Using Exemplars for Formative Assessment "Pays Off"

In this day and age of No Child Left Behind, assessment practices themselves are under assessment. Heidi Hayes Jacobs, president of Curriculum Designers, Inc. and educational consultant to thousands of schools nationally and internationally, advocates for the need to redefine assessment more comprehensively. Richard J. Stiggins works with teachers and learning communities and has found that assessment cannot function solely as an accountability measure through standardized test results and that teachers need to 'understand that assessment can work in positive ways to benefit learning, the time is right to add to our definition of good teaching the skillful use of assessment - doing it right and using it well.' (Classroom Assessment for Student Learning - 2004). Assessment is the third part of our curriculum office's triangle for improved student learning which intertwines curriculum, instruction and assessment (the Nashua Curriculum Instruction and Assessment 'CIA' model). By introducing Exemplars we have come to realize that we were providing the foundation for our teachers to begin moving toward formative assessment practices.

In 1995 the Nashua School District embarked on a full scale alignment of its Mathematics Curriculum K-12 with New Hampshire's Mathematics State Frameworks and the National Council of Teachers of Mathematics Curriculum and Evaluation Standards for School Mathematics (1989). The state of New Hampshire began its state testing under the New Hampshire Educational Improvement and Assessment Program (NHEIAP) for students in grades three, six and 10. The goal of NHEIAP was to provide data to districts as they reviewed how well their programs of instruction were working to improve student learning.

Our district's Curriculum Area Research and Development (CARD) team for mathematics recognized that the teachers in grades three, six and 10 could not shoulder the responsibility for students' achievement. As a district, the CARD team identified student proficiency outcomes (spo's) for each grade level. It was during the 1995-1996 school year that the district set its sights on identifying materials that would aid teachers in improving how students solved problems, communicated that information, and how they demonstrated the connections within mathematics and the real world. Simultaneously, summer institutes were provided for elementary teachers where they explored how to introduce meaningful context through problems that could be interpreted mathematically, that used multiple strategies and helped to build understanding. They were expanding their tool box of teaching strategies in mathematics through problem solving. We came upon Exemplars materials with its prepared problem-solving tasks, and rubrics, which are designed to help teachers identify proficiency levels along with sample examples of students work. In January 1996, Ross Brewer, of Exemplars, provided professional development for our K-6 Mathematics Facilitators who learned how important it is to know why students are being assessed and what students need to know to be better at problem solving. Teachers learned that students should know the standard by which their work is measured. We were on our way to looking at a better way to assess students' learning and how to have students begin to take responsibility for their learning by enabling them to self-assess. From there we incorporated annual professional development for all teachers to become knowledgeable with this resource.

In addition to the Exemplars material, we were researching materials for our elementary mathematics program. When Everyday Mathematics (EM) was identified and implemented in 2001 - we continued to use Exemplars in conjunction with the newly adopted materials. During the summer, teacher leaders identified Exemplars problems that aligned with our EM expectations. We were delighted to see EM and Exemplars come together to complement each other and help us meet our program goals.

With Exemplars we were preparing students to improve their problem-solving skills and writing skills. In addition, with the introduction of Exemplars rubrics, teachers were better informed of additional ways to assess student learning and were able to convey those expectations to students. Rubrics were reviewed with students and samples of what constitutes good work were available. Our end of year district assessment incorporates an open-ended question some are direct Exemplars questions and others have been tweaked to meet the identified learning goals. Grade level teachers are responsible for scoring results as a team, providing sample papers of students' work and recording the data to compare against district results. The review of these results becomes the basis for introducing the Exemplars work with new hires and as a refresher for veteran staff as we focus on curricular areas in need of improvement. District results have been tracked since 1998, and each year we have data to inform us about students' progress at each grade level. From the state NHEIAP results, use of Exemplars has paid off. Our students in grades three and six have consistently reported higher scores compared to the state results. This has held true from 1998 through 2003. In October 2005 the state embarks upon the Tri-state New England Compact Assessment Program (NECAP). We are confident that our students will continue to perform well from the experiences they have with Exemplars. Finally, Exemplars has provided a conduit, which directly impacts our efforts with the implementation of the Nashua 'CIA' model. Back to top

Using Journals in Mathematics

My teaching philosophy has changed significantly since I began teaching in 1990. My students sit at tables and mathematical concepts are taught in a problem-solving context instead of being presented in isolation through discrete workbook pages. One change in my teaching is due to the classroom research into mathematics journals I completed recently for my Masters of Education. My students became active participants in their own learning as they confidently solved realistic problems and explained their ideas in mathematics journals, and I was excited to be part of this rewarding learning environment.

In past years, when attempting to have children write in a mathematics journal, I would read: "This was easy. I like math." My students were not able to successfully reflect or share what they understood about problem solving or mathematics through their writing. Frustrated, I began to read about using math journals in the classroom.

What is a math journal?

My students write in a notebook to answer open-ended questions using numbers, symbols, pictures and words, and their writing can best be described as written conversations. A math journal is a place where every student has the opportunity to verbalize their math knowledge to their teacher, internally to themselves, and to their classmates. Students' writing becomes a source for social interaction as they read journal entries to partners and the whole class, talk about their learning and listen to others share different levels of mathematical reasoning.

What did I learn about using math journals?

My classroom research on mathematics journals led me to recognize four important steps needed to help students write reflectively about mathematics.

  1. Teachers need to model the writing process and the language of mathematics. First I modeled my own problem-solving process by thinking out loud as I solved problems and as I recorded my reflections on chart paper. Students soon began to contribute their own ideas about problem solving but continued to model the writing process by recording their comments on chart paper. Students copied these sentences into their math journals. Modeling the writing process took longer than I expected as students needed to become familiar with reflective writing and the language of mathematics, the words and symbols unique to mathematics. Once students became familiar with the vocabulary necessary to communicate in mathematics they began to independently express their own thoughts on paper.
  2. Teachers need to ask open-ended questions to guide students in their writing. I learned how to ask open-ended questions to help students think about their own understanding of problem solving and to guide their writing. I began my research by using a list of questions I found in Writing to Learn Mathematics, by J. Countryman (1992). Students answered my questions verbally at first and became comfortable sharing their thoughts and ideas with others. It was through their participation in our verbal discussions that students learned how to reflect upon their own knowledge of mathematics and to record their ideas on paper. I soon adapted the questions I found to better meet the needs of my students and to match the problems we were solving. Here are examples of questions I used in my research:
    1. Why was this problem easy?
    2. Would this problem be easier today than yesterday? Why?
    3. What did you do to solve this problem?
    4. Are numbers important in solving this problem? Why?
    5. Did graphs help you to solve the problem? Why?
  3. Students need to revisit similar tasks to increase their confidence as problem solvers and their knowledge of problem solving. As a teacher of young children, I quickly realized that involving students in rewriting similar problem-solving tasks to the problems they just solved was important in developing their confidence as problem solvers and in understanding the process. Children could not always solve the task independently the first time and were enthusiastic to help rewrite the task and solve it again. I noticed they were more successful in solving the second task.
  4. For example, we solved the task "Space Creatures" (Exemplars, 1996, March): On a new planet the astronauts discovered unusual creatures. The features they counted were 15 eyes, 3 horns, 12 legs and 7 arms. More of the creatures had scales than fur on their bodies. Draw your creatures and make a graph for each creature's features.
  5. The next week, we revisited a similar problem by writing our own version of "Space Creatures" called Super Robots: Students were visiting a robot factory. They saw and counted 13 eyes, 10 legs, 8 knobs, 9 arms and 4 antennas. More bodies were triangles than rectangles. Draw a graph first for each of your robots and then draw a picture of your robots to match your graphs.
  6. Teachers need to support students in recognizing their individual problem-solving styles. Students discovered they were different in the strategies they used to solve problems. I guided students to solve "Space Creatures" by drawing pictures of the creatures before making their graphs. For Super Robots, I asked students to make their graphs before drawing their pictures.
  7. Some of the students became frustrated when I asked them to begin their problem-solving task with a graph. I was curious to know why, so my first journal question asked students: "Is it easier to draw a picture first or draw a graph first?" Twelve children chose to draw their picture first and 11 children chose to create their graph first. Surprised with the split in their choices, I asked students to explain: "Why do you think it was easier to draw the graph or picture first?" The children wrote:

The picture was easier because you could count the objects better than the graph. (mathematics journal, Oct. 23, June)

It's easier to draw the graph first. Yes, because I knew what my Robot would look like. The graph helped me to count the features. (mathematics journal, Oct. 23, Cassy)

Their responses led me to realize the importance of drawing pictures in the problem-solving process for some children, but not all children. I needed to listen carefully to students so I would know how to best support them in recognizing their individual needs as problem solvers.

For four months I had supported students in becoming problem solvers and reflective writers in mathematics. In the end I questioned the students to see how they had changed since September. Students had become confident and independent writers, and they understood what it meant to be a problem solver. Two students wrote:

I can think of more answers. (mathematics journal, Dec. 3, Mitchel)

I've learned how to do harder problems. (mathematics journal, Dec. 3, Sam)

I now believe writing belongs in mathematics and is as important in developing students' mathematical knowledge as numbers and computation. It was my student-to-teacher interactions and my open-ended questions that guided students to write reflectively. I will always have a classroom of diverse learners and I now feel confident I can meet individual needs of students and lead them in their learning. Mathematics journals will guide my teaching.

References

Brewer, R. (1994-98). Exemplars. Underhill, Vermont: Exemplars.

Countryman, J. (1992). Writing to Learn Mathematics. Portsmouth, NH: Heinemann Education Books, Inc. Back to top

Reflections on Using Exemplars

In 2001-2002, Ross Brewer of Exemplars, Inc., came to central Iowa to do a presentation on Exemplars math and science assessments at our local area education agency. In our diocese at that time, we were still at the beginning of making the major shift to standards-driven education. We had written standards K-12, for all content areas, but we were still finishing up writing benchmarks and developmental levels.

What we saw and experienced with Ross that day moved us to closely examine Exemplars for our required "second" math assessments. We use ITBS/ITED [Iowa Test of Basic Skills/Iowa Test of Educational Development] tests currently as our primary assessments for reading, math and science. But since students deserve more than one way to show what they know and can do with our math standards, we needed to find a second assessment that took a different approach and format from the norm-referenced tests. We liked what we saw in the Exemplars math assessments. My group and I took our information back to our diocesan assessment team, and between that group and our large group of diocesan administrators, we settled on using Exemplars as an assessment instrument in all our diocesan schools.

First, our diocesan assessment team took a "Train the Trainers"-type professional development session from an Exemplars consultant. In this first training, we all learned some theory, we learned about administering Exemplars, and we learned about scoring using the Exemplars rubric. Our trainer modeled very well for us that day; we felt we had a pretty good grasp of Exemplars math assessments in a general sense. Each team member then went into a different diocesan school and practiced administering a sample Exemplars assessment. We came back together and talked over everything, trying to iron out the wrinkles before we were to begin full implementation in all schools. Our assessment team then trained the diocesan administrators.

I now train all new teachers every fall in Exemplars. I train them the same way we were trained: theory behind and usefulness of Exemplars assessments; fair and consistent administration; working on an actual Exemplars task; and reflection upon how students handle this type of math assessment and how they can improve their math learning over time. I train all new K-12 teachers with the "Chick-A-Dee-Dee" Exemplars assessment, as it is a K-12 problem. Training is typically three hours.

Thoughts and Comments on Exemplars

Because we have used Exemplars since 2002-2003, we have had ample time to reflect on our use of Exemplars. Following are some of our thoughts:

  • We like the aspect of checking students' understanding of a math concept, checking their strategies and reasoning, and checking their math communication on paper. This represents such a big change from math problem solving of the past, where a numerical solution was just about all a teacher looked for.
  • Our teachers really like the fact that for so many Exemplars tasks, there is always more than one right way to find a solution. This definitely taps into the creative/intellectual aspects of students of varying abilities.
  • Our diocesan grades four and eight scoring days are some of the best professional learning we do. Teachers, with their scorers' hats on, learn about students' math thinking. At the end of the day, we turn back into teachers and discuss what the data is telling us and how we can perhaps make instructional decisions from the data.
  • One of our greatest challenges has been getting students to articulate their math thinking on paper, not just restating how they, for example, punched in a sequence of numbers on a calculator. We continue to work on this in our classrooms. Using the Exemplars rubrics to help students become better self-assessors has facilitated this work
  • Getting students to answer in words, pictures and numbers is coming along well, although slowly. This is a big change for both teachers and students.
  • While we say that giving Exemplars to K-2 students is optional, we find most schools do give them in order to give students a friendly, teacher-led introduction to this format over the space of three years.
  • We have talked about moving toward using Exemplars math tasks as our primary math test instead of ITBS/ITED. With enough evidence of technical adequacy and trend data for the satisfaction of the Iowa Department of Education, our use of Exemplars could satisfy compliance issues as well as move us toward best practice math assessment.

Exemplars math assessments have worked well for us. We continue to look at our processes, the tasks themselves, and how students respond to them and learn by them. The continual improvement process and growth over time is slow but steady. Exemplars are worth taking a good look at. Back to top

Self-Assessment Improves Student Performance

My students use Exemplars student rubrics and benchmark papers first working as a whole class, then transferring that knowledge to small group and independent work. This gradual method allows students to gain the confidence needed to eventually self-assess. Below are several methods to help introduce rubrics to students.

  1. Use the "Kid Friendly" Exemplars rubric orally with students as a whole group to look at examples of student work from Exemplars. Ask each question from the rubric and have the students vote thumbs up or thumbs down if they agree or disagree after looking over the work. Look at different levels and use this same strategy then ask how the student could improve the work/solution if it is a thumbs down. Conference with students when they feel comfortable with this rubric and pose these questions to them about their own work. Allow them to go back and revise work if necessary.
  2. Introduce a section of the Exemplars rubric or "Kid Friendly" rubric one section at a time. Score examples of student work using only the criteria from that section so that students become very familiar with the language of the rubric section-by-section. Have students score their own work using only one section at a time, or have them work in pairs and score each other's work.
  3. Look at sample student work from Exemplars. Use this to introduce the rubric (whichever rubric you are using) and score as a whole group, section-by-section. Have students work in small groups or pairs using the rubric to score samples of student work and then report out to the group how they scored and see if there is agreement. Discuss scores when there is not agreement.
  4. Have students score samples of Novice, Apprentice and Practitioner work and then share ways to improve those students' score in order to move them to the next level. Have students list ways they improved upon the samples of student work for when they look at their own work.
  5. Look at samples of both Practitioner and Expert work. As a whole group list the qualities and characteristics that make it a Practitioner or Expert. Compare these to criteria on the rubric. Compare to their own work.
  6. Have students solve a problem, complete a science investigation or writing assignment and have a partner assess it using the rubric. Have the groups then discuss with each other their scoring and give their partners time to revise and then re-score each other's work.
  7. Have students self-assess using the rubric and then revise if necessary. Students can also meet with the teacher after self-assessing to see if the teacher agrees with the score and then the student can go back and revise. Do a final scoring together.
  8. Once students feel comfortable scoring work and discussing their own work with others have them volunteer to share their work with the whole class for scoring (on an overhead). Students can then ask the class for suggestions on revising their work.

* Note: When using student work from Exemplars with your students remember to cover up the annotations and scores.

** Note: The rubric you use for any of these activities depends upon the age level of your students. Exemplars student rubrics can be found here. Back to top