Reading the Reading Research: Afflerbach on Assessment

Barb Novak contributed this post. 

Click here for more of Barb's thinking.

Click here for more Literacy Booth thoughts about research.

Note. I'm sharing this article simply to give access to how I process when I read from academic journals - how I organize my thinking before, during, and after reading. This post is not intended to be an endorsement of the research or its conclusions.

"Reading Assessment: Looking Ahead"

Peter Afflerbach in The Reading Teacher (2016)

Wisconsin educators can use Badgerlink to access full-text of Reading Teacher articles. .

Click here to access Afflerbach’s article.

Why this article?

I’m feeling now, more than ever, the need to think carefully about assessment in order to actively seek change where it is needed. From his book (Understanding and Using Reading Assessment) to that time he recorded a webinar just for Wisconsin, I trust Peter Afflerbach’s thinking about and work around assessment.

This is not research; “The Inside Track” is a column in each issue of The Reading Teacher. The author of the column is selected (issue-by-issue) by the journal’s editors.

Research Questions:

Not applicable - this is a column written by a literacy expert rather than research.

Methods:

Not applicable - this is a column written by a literacy expert rather than research.

Main Points:

Afflerbach uses the article to advocate for “developing comprehensive formative assessments, assessing the wide array of factors that contribute to students’ reading development, and fostering student independence by helping students learn to use reading assessment on their own” (p. 413).

• An assessment system (and each assessment within it) must be based on “a detailed model of the thing we are to assess” (p. 413). What is reading? How do we define it?
• Afflerbach believes all assessment should follow this credo: “Assessment should produce information that is useful in helping students become better readers, and assessment should do no harm” (p. 413 - 414).
• Widely used assessments measure what was considered by the National Reading Panel Report, which is not a match for what is required of a reader on the National Assessment of Educational Progress (NAEP) or by the Common Core State Standards.
• Assessment should include “affective aspects of reading” (p. 416).
• Students should be the end users of data from reading assessments. Afflerbach states, “When we view reading assessment as an important teacher task, we may gather information that informs our instructional decision making. When we view reading assessment as something to teach, we can help students move toward the goal of independence through self-assessment. We want to use assessment that helps shift students from an outward orientation, where there is dependence on the teacher for assessment feedback, to one that looks inward” (p. 417).
• What impact could we have if we took all the money we spend on assessment and, instead, spend it on professional learning for educators about instruction and assessment?

Limitations:

• I wanted/needed more about “comprehensive formative assessments”. Common formative assessments, in my opinion, are interim assessments and are too formal to be considered formative. I was hoping comprehensive formative assessments could be an idea to replace common formative assessment.
• The article is theory; there is not a research component. It could inform questions for future reference.

Discussion:

• I’m interested in how Afflerbach’s ideas about engaging students in instruction and practice to become reflective readers could apply within universal instruction and intervention. What would it look like to make this a systemic and systematic practice? How could the results of implementing this practice be measured?
• Reliance on numeric data and the high-stakes nature of numeric data are obstacles to implementing Afflerbach’s ideas - especially his ideas about assessing the affective aspects of reading.

• This article would be useful foundational piece for reading and discussion by a school/district team reviewing its literacy assessment system.
• Excerpts from the article could be used to consider how a school/district engages students in reading assessment and teaches self-efficacy.

Full Citation:

Afflerbach, P. (2016). Reading assessment: Looking ahead. The Reading Teacher 69(4), 413 - 419.

Mindset On My Mind

Carol Dweck's Mindset: The New Psychology of Success found its way to my pile of favorite professional books shortly after I read Peter Johnston's Choice Words. That was in 2006. Almost 10 years later, this is an idea that still seems to surround me.

Mindset is all over Pinterest and Facebook in fantastic infographics like this one:

Image from Jackie Gerstein

Mindset is all over the educational news:

• "100% is Overrated" from The Atlantic challenges us to stop referring to others as "smart", writing, "When people perform well (academically or otherwise) at early ages and are labeled smart or gifted, they become less likely to challenge themselves. They come less likely to make mistakes, because they stay in their comfortable zone and stop growing. And their fixed mindset persists through adulthood."  
• Dawn Wilson's blog post for Learning Forward reminds us, "Choose your words carefully. Once shared, they are irretrievable."
• A survey by the National Center for Learning Disabilities found that "strong support from parents, a strong connection to friends and community, and a strong sense of self-confidence" - not an IEP or a 504 plan - are the most important factors for predicting a success. Sounds like a growth mindset to me.
• In a recent article for EdWeek, Carol Dweck encourages each of us to "legitimize the fixed mindset". Basically, she suggests we view ourselves as a mix of fixed and growth mindset and that we recognize the triggers that make us move into a fixed mindset.
• This book, Awakened: Change Your Mindset to Change Your Teaching, is on my "To Read" list.

My most recent idea about mindset is its connection to formative practices (aka formative assessment). Formative practices cannot be truly successful within a school if students, families, and educators don't share a growth mindset. We use formative practices to gather information about where each student is in his/her journey to understanding. We use the information to inform our teaching. Sounds like a growth mindset for students and educators. 

I'm thinking about starting some new professional learning about formative practices with growth mindset. What do you think?

(I'm also thinking about blogging about each chapter of Mindset - one chapter per month. Would anyone want to read and write along with me?)

Using Assessment for Classroom Decision Making

Jaimie Howe contributed today's post. More of Jaimie's thinking can be found here.

Over the last 4-5 years the use of assessment in our district has really evolved. Below is a look at the progression of how assessment has looked in my building specifically.

In my building, we have always tried to use assessment to make decisions , but in the past it was much more informal and observational – mostly just used to form reading groups; partly due to limited assessment options.  Our knowledge and thinking started to shift back in 2011 as we began to gain knowledge of building systems and the RtI Framework.  Now with more systems in place and more knowledge of assessment, we are starting to use it more effectively with many other areas of instruction and for many other purposes.

There have been several things that have been implemented that were crucial to our shift in thinking and ability to start to use assessment more effectively.

• Standards Based Report Card: We implemented a Standards Based Report Card in 2011 for ELA. This caused us to see the need for assessments to align and be more systematic rather than only informal or observational.  

• More Assessment Options: We also began using the DRA2 in 2011, which gave teachers the opportunity to look at each individual child more closely and select specific learning targets for students.  It also allowed teachers to consider grouping students by skill/strategy versus reading level based off of the DRA2 Class Profile.

• District Assessment Frameworks: Each grade level has an Assessment Framework indicating which assessments they need to give and when.  It is organized by type of assessment (screener, diagnostic, formative, etc.).  This creates so much more consistency and common knowledge across buildings in our district and specific grade levels.

• Master Schedule and Intervention Block: Implementation of a master schedule and an intervention block in 2012 made it so much easier for us to have a consistent collaboration schedule.  This was imperative for us in moving forward.  It gave us the opportunities we needed to look at our assessments more closely and make decisions as a team. Below are the consistent collaboration times that we use each year; however, much more collaboration happens as needed.

• Grade Level Teams meet 2x per week during MAPE (45 minutes: 1 day – ELA, 1 day – Math)
• Grade Level and Cross Grade Level Teams meet with Literacy Coach 1x per month. (½ hour - before school)
• Unit Teams (K-2 and 3-5) meet with Math coach 1x per month. (½ hour - before school)
• Individual Grade Levels with resource teachers (SPED, psych., lit. coach, etc) ½ day every 6-8 weeks (4x per year) – includes analyzing assessments and instructional planning
• Opportunities for ½ day release time for instructional planning to create common assessments and/or collaborate around assessment.

This consistent collaboration schedule has been imperative in the work that we do. Without collaboration, it would be nearly impossible to use assessments to guide our instruction. We are trying to develop a system where assessment is a precursor to instruction; telling us what to teach, not just how students did.  This takes time and a lot of hard work, but it’s worth it!

Wisconsin's Specific Learning Disability (SLD) Eligibility Criteria

A new episode of Literacy Live is posted. It's all about finding data sources within your balanced assessment system that could be helpful when an IEP team is making an eligibility decision for specific learning disabilities.

The episode can be viewed here:
https://sites.google.com/a/dpi.wi.gov/lit-live-assessment/home/literacy-data-and-sld-eligibility

Teaching to the Test

Andrea Reichenberger contributed today's post. More of Andrea's thinking can be found here.

Standardized testing is a topic that weighs heavily on all of our minds as coaches, teachers, administrators, parents, and students this time of year.  Ultimately, the testing is required whether we agree with it or not, but it does trigger interesting discussions about the idea of “teaching to the test.” When I hear this phrase, I often react negatively and have to refrain from screaming, “ABSOLUTELY NOT!” But the more rational side of me forces me to ask, what does “teaching to the test”  mean, or rather, what should it mean in 2015? How can we change the narrative?  I recently “attended” Jim Burke’s online webinar sponsored by Corwin Press entitled: When Is It Best to Teach to the Test? One of the ideas I loved about this webinar (and I loved several) was Mr. Burke’s creation of a new definition for “teaching to the test.”

(Forgive me Mr. Burke, as I have stolen the important points straight from your slides.)

Burke reiterates that we need to prepare our students for assessments such as the Badger Exam, ASPIRE/ACT, AP Exams, etc without interrupting our daily instruction. We do this by focusing on the skills our students need as a result of the shifts in the standards that these assessment align to including:  close reading, using text based evidence to support their thinking, writing from sources, using academic language, increasing text complexity, focusing on the  importance of argument and a greater emphasis on nonfiction and literacy in other disciplines. Since this is the case, and since we are shifting our definition, Burke reminds us that we should ALL be collectively “teaching to the test” and therefore need to be intentional when choosing and/or creating:

• Texts
• Topics
• Tasks
• Tests
• Technology
• Techniques

Mr. Burke summarizes it best when he states, “Students learn best when assessment and preparations for tests are integrated into our daily instruction as a means of enriching and extending that instruction.” I agree. Thank you for changing the narrative for all of us. Spread the good word, my friends!

Please see page 4 of the link from Burke’s English Companion handouts for using Webb’s Depth of Knowledge Model. If you have an hour to spare, you can access the archived webinar here.

Preparing for the Badger: Finding the Positive

Jaimie Howe contributed today's post. More of Jaimie's thinking can be found here.

I’m sure we all see and hear about the controversy associated with state testing and specifically this year in Wisconsin with the new Badger Exam.  The controversy that accompanies this assessment and the decisions that are made regarding the Badger exam our pretty much out of our control as individual districts, schools, and educators; however, how we choose to handle, proceed, and/or deal with the fact that in the end we will all be giving the assessment in some way shape or form, is ultimately our choice.  Instead of continuing to dwell on the fact that changes seem to be made daily, technology isn’t always working, or the possibility that it may not even be around next year; consider the material that is available to us and how those materials may support our teachers and students in so many ways; more than just preparation for a test.

Back in January, I was given the charge from my principal to dig into ways in which we could prepare the students in our building for the upcoming Badger Exam.  I browsed, researched, printed, and read an abundance of materials available to us; in search of a way to build not only our students capacity, but our teacher’s capacity and knowledge in regards to the Badger Exam and ultimately, the Common Core State Standards.  As our teachers plugged away at the materials through January and February, I couldn’t help but worry that we were spending all of this time “teaching to the test,” and wondering if all of this preparation was truly worthwhile. As I continued to have conversations with the 3-5 teachers in my building about their experiences with the materials, I started to worry less.  I heard a lot of, “Wow, the content is so HARD,” but not hard in a bad way; hard in a rigorous way.  Teachers have come to me and expressed that using these materials has really opened their eyes to what really is expected of our students with these standards and the rigor involved. This is really the first look, since the implementation of Common Core, that teachers have had, that truly allows them to be able to see and understand what is expected.  I also feel that the way in which the teachers in my building chose to use the materials, has been paramount.  At first, it was frustrating because the technology wasn’t working on ipads, so the students couldn’t access the tests since we do not have a computer lab in our building.  However, not using the technology turned out to be very valuable. The teachers ended up truly using the materials to teach and not just practice. They modeled, worked as a class, worked in groups, and really dug in. I had numerous comments regarding how engaged the students were with the materials and the topics.  Teachers have also commented that the resources have helped them in their own instruction.  They have reflected on their own teaching and indicated ways in which they are going to change their instruction to better meet the demands of the Common Core.

As a result of  this, the teachers in my building seem to be less stressed than other buildings about the unknown of the upcoming Badger Exam.  They've worked around the complications, which are certainly inevitable with anything new; and have turned the complications into a positive change in their teaching.  

In current times of education, I see too often, time spent dwelling on all that is negative and we fail to realize all of the positive that is out there.  Because the teachers in my building chose to find the positive in the materials and use them to better their instruction vs. “teach to the test,” the controversy regarding the Badger Exam has affected them very little. They don’t feel they've wasted instructional time; rather, they’re thankful for the opportunity that they had to use the materials to improve their instruction.

If you haven’t had the opportunity to see some of the Badger Exam materials that are out there, check out some of the links below.

Online practice tests:

http://sbac.portal.airast.org/practice-test/

https://wdpibadger-tss.caltesting.org/inbrowser/

Print Resources:

http://sbac.portal.airast.org/practice-test/resources/#activities

Additional Activities:

http://sbac.portal.airast.org/ca/field-test-ca/classroom-activities/

http://readingandwritingproject.org/resources/assessments/reading-writing-assessments

Leading a Balanced Literacy Assessment System

Barb Novak contributed today's post. Barb is a literacy consultant at the Wisconsin Department of Public Instruction and serves as the moderator of this blog.

A big part of my job - a favorite part - is talking to Wisconsin's students, parents, educators, and leaders. What seems to be on everyone's mind lately? 

Assessment.

What are we assessing? How long does it take? How much money does it cost? To what extent is students' privacy protected? What assessments are required? Who analyzes the data? Does the data actually make a difference? How can I access the data? What assessments exist for special populations? Why are certain data sources valued more than others? Who makes decisions about assessments? 

Laura Adams, fellow literacy consultant, and I answer all of these questions and more in the 2014 - 15 season of Literacy Live!, a pre-recorded webinar series for Wisconsin's literacy leaders.

"Leading a Balanced Assessment System" will include about 10 sessions. Each session includes a recording (generally 25 - 30 minutes) about a specific topic related to literacy assessment. Supporting resources accompany each session. New sessions are posted about every three weeks.  The episodes can be viewed at any time. While the sessions can be viewed independently, we do suggest watching with colleagues to take advantage of opportunities for discussion and application.

Introductory episodes include:

• Required Literacy Assessment in Wisconsin: An overview of all assessments required in Wisconsin
• Reflecting On Your Balanced Assessment System: A process for recording details about the assessments in your system that may lead to identifying redundancies or improved communication

Contact me via email (barb.novak@dpi.wi.gov) or phone (608-266-5181) with questions.

Assessment: How Much Is Necessary?

Jaimie Howe contributed today's thinking. To learn more, read Jaimie's post about education initiatives.

Three years ago our district did a huge overhaul of our elementary English language arts assessment system.   We wanted to align with the balanced assessment suggestions of the RtI model. We made sure we had a common screening and diagnostic assessment, as well as common formative assessment options documented for the entire district.  The diagnostic assessment we chose was the DRA2. Every K-5 teacher was trained and then required to assess all of their students in the fall, winter, and spring.  Although there was push back to begin with, the DRA came to be a “must have” to most teachers.  Having the DRA2 data strengthened our collaboration and instruction tremendously.

Fast forward to the present.  We now have PALS in grades K-2.  This test is considered a screener; yet, is also diagnostic and provides great information and reports.  With the gradual implementation of PALS to first kindergarten, then first grade, and now second grade, we have continued to keep the DRA2 as a requirement.  I have been struggling with the idea of whether or not the DRA2 is still necessary for ALL students, especially in first and second grade where an oral reading passage is a required component of PALS.  Do we need another one from the DRA2, too?  I still believe the DRA2 is extremely valuable, but teachers just do not have the time to assess all students on PALS AND all students on the DRA2.  Although the DRA2 gives us such great information, can we get most of that same information from PALS, our state required assessment?

I keep wondering if it will be more beneficial to take away the DRA2 or make it optional. If we required PALS in the winter (instead of DRA2), we would have three common data points with the same assessment - comparing apples to apples. Teachers would have more time for instruction because they aren’t spending so much time assessing.   Teachers could use the DRA2 for those students that were ID’d on PALS for more information if needed; however, I do feel that PALS offers enough information for us to pinpoint student needs and tailor our instruction accordingly.

Although there are so many great resources and assessments on the market, we can’t do or use them all.  It all comes back to quality and not quantity.

Three Resources About Conferring

Confession. I worry about conferring.

I worry that this valuable classroom practice will be lost in a fast-paced educational environment that values quantitative data (and its technical adequacy). 

Conferring will not result in a numerical score. It probably isn't reliable or valid. But, within about 60-seconds a conference can tell a teacher a whole lot about what a student knows and doesn't know quite yet while building a relationship with the student. 

Conferring is a valuable tool for formative assessment. 

The following resources provide some further information.

Resource 1: Conferring General Information
This resource provides curated information about conferring, including how it fits within a gradual release of responsibility instructional framework. It is part of a larger resource from the Department of Public Instruction about about instruction in English language arts.

http://bit.ly/ELAinstr

(Select "Independent Task" and "Conferring")

RESOURCE 2: Conferring How-To

The above document is part of a Classroom Strategies and Practices bank hosted at the Wisconsin RtI Center. It provides step-by-step directions and supporting record keeping forms.

The bank includes other assessment strategies and practices. (In addition, the bank includes instructional strategies and practices for each cluster within the CCSS ELA standards for reading literature and informational texts.

Resource 3: Conferring via GoogleHangouts
This great blog post (from Two Writing Teachers) describes one teacher's experimentation with GoogleHangouts as a place to host writing conferences.

Literacy (un)Conference 2014

The Wisconsin Department of Public Instruction is pleased to announce the 2014 Literacy (un)Conference. This year's online conference (our second ever!) is a free-of-charge opportunity to learn more about:

Balanced Literacy Assessment for Early Learning

The information above can be downloaded through GoogleDrive for wide distribution.

All conference sessions are pre-recorded for easy viewing within the busy schedules of educational leaders. We recommend viewing the sessions in professional learning communities and returning to the sessions for later work. (All sessions and supporting material will be archived.)

A Twitter chat to discuss applications of new learning is scheduled for Monday, May 19 at 7:00 PM CST. #wisuncon

Contact Barb Novak, Literacy Consultant, Wisconsin Department of Public Instruction, for more information.

barbara.novak@dpi.wi.gov (608) 266-5181

Note. The 2013 Literacy (un)Conference is archived. It includes sessions about:

• standards-based instruction and assessment
• planning for professional learning

Smarter Balanced: Resources for Professional Learning

Today's post is brought to you by Bobbi Campbell.

The following resources were developed by Madison Metropolitan School District in response to the 2015 implementation of the Smarter Balanced Assessment in grades 3 - 8.

Resource 1: 30-minute Overview (with speaker notes)
Can be accessed in GoogleDrive.

Resource 2: Facilitator Guide for Overview
Can be accessed in GoogleDrive.
The first facilitator guide is intended to be paired with the Power Point.  It provides information about the components of the test and includes examples.  

Resource 3: FAQ (for practitioners)
Can be accessed in GoogleDrive.
This is designed to be a quick snapshot of the Facts and Questions associated with Smarter Balanced Assessment. This version of the FAQ is designed for practitioners and referenced in the Overview presentation. 

Resource 4: FAQ (for administrators and instructional leaders)
Can be accessed in GoogleDrive.
This is designed to be a quick snapshot of the Facts and Questions associated with Smarter Balanced Assessment. This version of the FAQ is designed for administrators and/or instructional leaders and referenced in the Overview presentation. 

Common Assessments: Using the 4 Essential Questions

Today's post is brought to you by Jaimie Howe.

Our district has been focused on the full implementation of an RtI framework and system for the past several years.  The one area we always try to bring everything back to are the four essential questions associated with PLCs:

1. What do we want all students to learn?
2. How will we know they have learned it?
3. How will we respond when learning has not occurred?
4. How will we respond when learning has already occurred?

Coming back to these questions helps us put everything into perspective. As a literacy coach trying to support my teachers through this process and being the facilitator for many collaboration meetings,  I developed the following flowchart (based off of a similar chart that was developed by Solution Tree in accordance with The Professional Learning Communities at Work Coaching Academy (Solution Tree, 2011) I attended during the 2012-13 school year) to help teachers pull everything together in regards to PLCs, common assessments, SMART goals, CCSS standards, etc.

Common Assessment Flowchart is available AT THIS LINK

When you first look at the flowchart, it looks a bit confusing; however, if you truly follow the arrows and complete each step, it is a great organizational resource for teachers to use when planning and developing common assessments and SMART goals based on the CCSS.

Our Journey With Common Assessments

Today's post is brought to you by Julie Schwartzbauer.

“[Common formative assessments provide] regular and timely feedback regarding student attainment of the most critical standards . . . [and] also foster consistent expectations and priorities within a grade level, course, and department regarding standards, instruction, and assessment. . . . Most importantly, common formative assessment results enable educators to diagnose student learning needs accurately in time to make instructional modifications.” (Ainsworth, 2007, pp. 95–96)

Our grade level PLCs have been hard at work in Appleton learning about common assessments.  Many articles and professional learning opportunities have been put in place to help build teacher expertise in creating common assessments.  We also formed a Common Assessment Committee.  The committee had three or more grade level representatives from across the district, as well as specialists and literacy coaches.  The purpose of the committee was to design sample common assessments for teachers.  The teachers were then encouraged to use the samples within their PLCs, as a starting point for developing common assessments that were directly linked to their instruction and specific to the needs of the students in their classrooms.

The following is a list of steps we used in designing a common assessment:

1. Form the group (grade level teachers, specialists)

2. Meet as a team (PLC)

3. Decide which standard(s) to measure

4. Create an assessment

• Consistent with grade level Common Core State Standards (CCSS)
• Linked directly to instruction
• Tied to the needs of the students in the teacher’s classroom

5. Develop evaluation criteria (to assess the degree of understanding or proficiency)

As a starting point, we focussed on creating samples for the standards that appeared on our report card.

Our teachers are still in the process of understanding the true purpose of common assessments, but we definitely have jumped into the work required to build expertise.