Dangerously Irrelevant

Education Trust: Four ways of thinking about achievement gaps

The Education Trust has come out with a nifty little report, Gauging the Gaps: A Deeper Look at Student Achievement, that examines different ways of thinking about achievement gaps. Although EdTrust uses NAEP data to illustrate its points, this could be done with any achievement data.

EdTrust outlines four different perspectives when considering achievement gaps. Each has its own merits, but together they have more power than simply using one or two of them in isolation. At the end of the report, EdTrust includes several NAEP data tables that show the progress of each state along each of these perspectives.

Perspective 1: Simple gap narrowing

“Have absolute gaps in mean performance between groups decreased over time? Nationwide, low-income students and students of color perform, on average, below their peers. So it is imperative to evaluate whether we’re helping these young people catch up.” (p. 2)

Perspective 2: Progress for all

“Have all groups of students gained over time? Our country needs to improve achievement for all students and accelerate gains for those who lag behind. Reading performance for low-income fourth grade students nationwide inched up by four points from 2003 to 2007. This represents movement in the right direction but at far too slow a pace. Some states, however, improved much more rapidly than the nation as a whole.” (p. 3)

Perspective 3: Gap size

“What is the current size of the gap between groups? In addition to examining how far a state has come in closing the gaps and looking at whether all students are gaining, it’s important to know the extent of the gaps that remain. The current-year size of a state’s gap suggests how far we have to go until race and income no longer play a significant role in student achievement. Nationally and in every state, low-income students trail their higher income peers in reading performance. Yet a closer scrutiny of state data shows that some are closer to achieving equitable results than others.” (p. 4)

Perspective 4: Group comparisons across jurisdictions

“How does each group of students currently perform compared with their counterparts in other jurisdictions? Although many assume that certain groups of children perform about the same no matter where they attend school, comparisons of group performance across jurisdictions can reveal striking differences. In fact, dramatic variations in the achievement of similar groups of children occur across states or from one district to the next.” (p. 4)

What do you think?

How does your school organization think about achievement gaps? Does it predominantly use only one or two of these perspectives? What could be done to increase your organization’s capacity to use more or all of these perspectives? Is all of this even worthwhile?

Data-driven decision-making resources from CASTLE

I have done a lot of work over the years on various data-driven decision-making projects. I've taught courses, given workshops, delivered multi-day institutes for state departments of education and corporate partners, and written book chapters and white papers.

I collected some of the highlights of that work on my new DDDM page here at Dangerously Irrelevant. Resources include some of the products from my work with Microsoft, the American Psychological Association, the Chicago Public Schools, the Minnesota Department of Education, and others. 

I hope these are useful to you. Happy reading!

It's not 'the tests.' It's us.

I often hear educators say...

We could be teaching differently if it weren't for ‘the tests.’

Or…

We could do a better job of meeting our students’ needs if it weren't for ‘the tests.’

I emphatically dispute these assertions. We must take ownership of our own culpability.

Our prevalent instructional model that emphasizes low-level, decontextualized, factual recall was dominant long before ‘the tests.’ Our challenges of providing higher-order thinking experiences, opportunities for authentic collaboration, and real-world connectedness existed long before the No Child Left Behind Act. Our inability to effectively facilitate empowered technology usage, true cultural/global awareness, and other necessary skills for a digital, global, information age is a byproduct of long-held, deeply-rooted cultural and pedagogical norms, not recently-acquired beliefs and behaviors.

Is anyone willing to argue that achievement gaps were smaller before evil NCLB came along and messed us all up? Does anyone think that we were doing a fine job of meeting the needs of underserved populations before ‘the tests?’ Have we all forgotten that school has been boring for generations?

It's not ‘the tests.’ It's our unwillingness and/or inability to do something different, something better.

It's not ‘the tests.’ It's us.

 

UPDATE: There are some phenomenal comments below. I hope you'll take a few moments to read them. Be sure to also read Greg Thompson's reaction to this post.

The President is calling

The President is calling:

I'm calling on our nation's governors and state education chiefs to develop standards and assessments that don't simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking and entrepreneurship and creativity.

President Barack Obama, March 10, 2009

Alia iacta est. How will we answer the call?

Alfie Kohn on the next U.S. Secretary of Education

Alfie Kohn’s new article in The Nation comes out in print next week. You can read it early online and get his perspectives on the next U.S. Secretary of Education. Here’s a quote:

Almost never questioned ... are the core elements of traditional schooling, such as lectures, worksheets, quizzes, grades, homework, punitive discipline and competition. That would require real reform, which of course is off the table.

I’m sure that many of you are unsurprised that Kohn is one of my favorite education writers. My all-time favorite education book is Beyond Discipline. I like how Kohn speaks truth to power and is willing to hold topics up to the light that are given little thought by most educators because they’re so deeply ingrained in the system.

Update: if the link at The Nation doesn't work for you, this one should.

Slide - No generation in history...

[Download this file: png ppt pptx]

[from David Warlick, Happy Birthday Jude]

SETDA's Class of 2020 Action Plan for Education

The State Educational Technology Directors Association (SETDA) has been on an unbelievable tear this year. Back in February it released its annual Trends Report on NCLB Title II, Part D (Enhancing Education Through Technology, or E2T2). Previous national reports are available at the SETDA web site. You also can access state-level reports at the Metiri Group’s web site.

Now SETDA’s Class of 2020 Action Plan for Education project is releasing its reports. The first three already are available:

• Powerful broadband access for all: Breaking through the barriers
• STEM education: Achievement and innovation
• Technology-based assessments improve teaching and learning

Two more reports are coming out this month and next:

• Empowering teachers: A professional and collaborative approach
• Robust, rigorous, and right-on-time learning through online courseware

Be sure to tap into the incredible wealth of good information on the Reports, Research & Tools page of the SETDA web site [warning: it’s easy to get lost in here for hours…]. There are numerous high-quality resources available for K-12 educational technology advocates and change agents, including the 2007 report, Maximizing the impact: The pivotal role of technology in a 21st century education system.

Keep up the great work, SETDA!

Not so irrelevant 013

My latest roundup of links and tools…

When did the IT staff get promoted above the superintendent?

Will Richardson notes:

[A] school superintendent I spoke with … lamented the fact that his IT staff wouldn’t give him access to YouTube and even Wikipedia.

See also my older post: Principal blogging not allowed.

Math and motocross

Check out this sweet series of motocross math videos at HotChalk. The brains behind the math? Former guest blogger Jason Dyer!

“I didn’t know Sasquatch was real.”

Fun with the Pacific Tree Octopus!

Maybe we should do this for teachers and administrators too

"Seventy-one-year-old Peggy McIntyre needs to learn as much as she can about Windows before 8 a.m. Or else."

Post-Gutenberg economics

It’s now a publish-then-filter world. Clay Shirky notes that “we’re clocking a singularity a week at this point.”

We need to educate our educators

Seth Godin says:

It’s easy to be against something you’re afraid of. And it’s easy to be afraid of something that you don’t understand.

Open your brain, open your model of education

The Education Innovation blog has an interesting post on closed v. open models of education. [Note to self: this might be the world’s longest URL]

Some good thinking going on here

Thanks to Mike Sansone, I recently discovered the Union Square Ventures blog. In Power to the People, they state:

[W]e believe that we are only at the beginning of the web’s impact on the fundamental structure of education. We expect much of that change to be away from the existing educational institutions and towards empowering individuals and newly-formed groups.

In Why the Flow of Innovation Has Reversed, they note:

[T]he vector of innovation has changed. It used to be that innovation started with NASA, flowed to the military, then to the enterprise, and finally to the consumer. Today, it is the reverse. All of the most interesting stuff is being built first for consumers and is tricking back to the enterprise. . . . [O]ne reason this is happening is that the success of a web service is more often determined by its social engineering than its electrical engineering.

Students aren’t the only ones missing the big picture

The Florida Department of Education is concerned that students are missing the big picture when it comes to science. A task force stated that “teachers should provide a broader focus on scientific concepts and process in a 'big picture' sense.” Hmmm… I wonder if that means the Department is going to narrow down the list of required science standards and also pare down the size of approved textbooks. I’m guessing not. Download the full report if you dare.

Disempowered today = disempowered tomorrow

I left this comment at Jim Gates’ Tipline blog:

Students who aren't fluid technology users today will be the low-wage workers and disempowered citizens of tomorrow.

I want it right THERE

Finally, if you’re anal-retentive about your Windows taskbar like I am, check out Taskbar Shuffle.

Help wanted - Resources for high-poverty rural schools

This semester my preservice administrator students are creating a wiki that hopefully will become a helpful resource for high-poverty rural school districts. In particular, they're trying to locate resources that are helpful for educators working to increase the academic achievement of economically-disadvantaged rural students. If you know of any good resources in this area, please leave them here as a comment. Thanks!

Not so irrelevant 012

Three great questions

I especially like the last of these three questions from Rodney Trice. We should be asking teachers and principals that question more often (and just that directly).

• How do you intend to bring the global community into your classroom?
• How will you prepare students for a future that is relatively unknown?
• How you will eliminate the racial predictability of achievement outcomes in your classroom?

This just in: Teenagers play video games!

All kidding aside, the latest report from the amazing Pew Internet & American Life Project confirms that kids - even girls! – are up to their eyeballs in video games.

We’ll stick to the tried and (not) true

Nope, sorry. iPods are not allowed. Back to the old way. Too bad it doesn’t work as well. Gotta do it anyway. Oh, and I love how the music players are categorically, by definition, a ‘distraction’ (if not in actuality). Who needs reality when we have these little educational policy fantasy worlds that we can create for ourselves?

Throw da bums out!

After attempts to bring in turnaround experts didn’t work, the state of Maryland is increasingly leaning toward completely restructuring schools that are academically unsuccessful. State schools Superintendent Nancy Grasmick says:

We are very comfortable being more aggressive about this. We have seen much better results [when the staff is replaced].

Blog like a farmer

I ran across an old post by Mike Sansone, one of my Iowa blogging buddies. I really like his metaphor that blogging should be like farming.

Scorecards

I bet parents and community members would really like to see scorecards like this one (maybe with different data) for their local schools. I know some schools and districts already do this. Hopefully they use line graphs rather than tables of numbers. Could you tell the essential story of a school district with 10 key, well-done graphs? I bet you could!

No writing in journalism class?

Check out this excellent article about the NYU journalism student who got in trouble for blogging about her class. [hat tip to Tim Stahmer]

I got no money, honey

Did you catch Edutopia’s advice on how to innovate without extra money or support?

Spend hours on content you can find with Google in 3 seconds!

One of my favorite things about Wes Fryer is his ability to highlight the ridiculous. I also enjoy his irreverance (“Behold! I hold aloft the holy words!”), particularly when I have the same experience at my kids’ school.

Speaking of Google…

Finally, I’m digging Google Chrome. it’s now my default browser and I’m using Firefox less and less (and I love Firefox). Chrome is much faster. I also like that each tab is a separate process; I have yet to have a browser hang…

Recommended reading - Data-driven decision-making

I often get asked by administrators for some recommended reading. Here are some of my favorite books on data-driven decision-making. If the Amazon widget doesn't load in a few seconds, here's a static picture of the list.

[Transparency disclosure: If you buy a book using this list, CASTLE gets 4% of the proceeds. Your cost doesn’t go up any. Amazon just pays us a little for the referral through its Associates Program.]

The bottom billion

[cross-posted at LeaderTalk]

I just finished Dr. Paul Collier's award-winning book, The Bottom Billion. An economics professor at Oxford University, Collier notes that traditionally we have thought of the world's population as being made up of '€˜a rich world of one billion people facing a poor world of five billion people'€™ (p. 3).

Instead, we need to recognize that the developing world should be divided into two groups, the middle four billion and the bottom billion. The countries in the middle group have economies that are growing, life expectancies that are rising, infant mortality rates that are dropping, and malnutrition rates that are declining. They still have a long way to go compared to the developed world, but they are making progress. India and China are both part of the middle four billion, despite once being poorer than many of the countries in the bottom billion.

In contrast, the bottom billion people continue to live in '€˜the fourteenth century, [doomed by] civil war, plague, and ignorance'€™ (p. 3). Their economies are not growing and, '€˜given the power of compound growth rates, [the] differences between the bottom billion and the rest of the developing world will rapidly cumulate into two different worlds'€™ (p. 10). The bottom billion not only are '€˜falling behind ... [they are] falling apart'€™ (p. 3).

Below is a graph of the growth rates of the bottom five billion over the past few decades [note that the actual growth rates were not this linear; Collier reported his figures by decades so I had to approximate]. Growth rates were as follows:

When these growth rates are plotted cumulatively, the chart looks something like this:

As you can see, the bottom billion are barely better off than they were 35 years ago. About 70 percent of the bottom billion lives in Africa. The bottom billion includes countries such as Somalia, Afghanistan, Haiti, Yemen, and North Korea.

I have greatly appreciated how this book has reoriented my thinking about global economic development challenges. Being a data guy, I have jumped into some of Collier'€™s numbers in order to help me envision what the challenges are that face the bottom billion. As we look toward the plight of our own disadvantaged student populations, we would be well-advised to look for data that help us reorient our thinking so that we may appropriately address our learning challenges. What data do you have that might shake up the thinking of your educators, students, and families?

[I learned a lot from this book. I give it 4 highlighters.]

What's the best way to ensure mastery of low-level content?

[cross-posted at the TechLearning blog]

State and federal accountability schemes require that students master low-level academic content. Our decisions regarding how we structure our instruction to facilitate student mastery of that content strike to the very heart of what we believe about teaching and learning. To facilitate conversations about this issue, I made a short video:

What do you believe is the best way to structure instruction to ensure student content mastery?

Music credit: Safe Passage, Freeplay Music

Why don't we do more pre-testing?

[cross-posted at LeaderTalk]

When I moved to Iowa from Minnesota, the Iowa Motor Vehicle Division (MVD) didn’t test me before it issued me a driver’s license. It took into account my long history of driving and my clean record and determined that I did not need to take either a written or driving test. I did a quick vision check, took one of those goofy photos, and I was all set.

Imagine, however, if the MVD, before it would issue me a license, wanted me to sit through a series of classes intended to ‘teach’ me how to operate a car and drive safely. I would have been completely annoyed. ‘Test me now!’ I would have exclaimed vociferously. ‘I already know how to do this! Stop wasting my time!’ By now you’re probably nodding your head in agreement, knowing that you’d do the same thing in my situation. Although you’d rather not have to do the written and driving tests again, you’d definitely rather be tested than sit through hours of instruction on material you already know.

Unfortunately this is exactly what happens to our nation’s schoolchildren on a daily basis. Millions of students regularly experience curricula and lessons that address content and concepts with which they’re already familiar. It’s not just the ‘talented and gifted’ kids; there are plenty of students who know the material in a particular learning unit before they even start. They’re just never given the chance to demonstrate their knowledge ahead of time. Nor do they have the opportunity to request to be pre-tested.

What a colossal waste of time this is. Rather than the joy of wrestling with and thinking about new material, students suffer through yet another hour ‘learning’ old information. Rather than working with children who are eager, interested learners, teachers suffer through yet another group of disengaged students.

I wonder why we don’t care more about this? It’s one thing to cover the required curriculum. It’s quite another to have students cover the curriculum despite the fact that they already know it. As a former eighth grade teacher, I know how difficult it is to differentiate instruction. But it’s relatively easy to do some simple pre-testing and at least make an attempt at altering a ‘one size fits all’ lesson plan. If more teachers did this on a regular basis, they might be surprised at how much instructional time they gained back during the year. And of course they’d also have better baseline data with which to assess student learning growth for each curricular unit. And did I mention the message of respect for students that accompanies the practice of pre-testing?

Why don’t we do more pre-testing? Why is it so hard to get teachers to buy into this?

The diminishing returns of too much information

I really liked this image of the inverted U from Peter Morville’s Ambient Findability so I decided to make my own version:

I’m a strong proponent of data-driven practice, particularly for classroom- and student-level progress monitoring of essential learning outcomes. But, as Morville notes, there are diminishing (and even negligible) returns if one goes too far. We have to know when to say when.

What would you ask Mike Schmoker?

[cross-posted at LeaderTalk]

In May I have the glorious opportunity to interview Mike Schmoker, guru of data-driven education and author of Results, The Results Fieldbook, Results Now, and The Crayola Curriculum. And, yes, I’m going to try and record it as a podcast.

I know that many of you are familiar with Mike’s work. If you were me, what interview question(s) would you ask him?

[cross-posted at LeaderTalk]

In May I have the glorious opportunity to interview Mike Schmoker, guru of data-driven education and author of Results, The Results Fieldbook, Results Now, and The Crayola Curriculum. And, yes, I’m going to try and record it as a podcast.

I know that many of you are familiar with Mike’s work. If you were me, what interview question(s) would you ask him?

Help wanted: Strategies for maximizing performance on computer-delivered assessments

I received this message recently from someone who works in a school district department of research, evaluation, and assessment:

Do any of you know of research on effective strategies for maximizing performance on computer-delivered assessments? I think we have some common-sense ideas, but I am wondering if there is anything out there that may be more substantive or research-based. 

 

Do any of you have documents that discuss tips and strategies (for computer-delivered assessment) that you share with your teachers? 

Any suggestions for her (and each other)?

Data done right

[cross-posted at eduwonkette; see also her response]

When eduwonkette asked me to guest blog about data-driven decision-making in schools, I eagerly agreed. Why? Because in my work with numerous school organizations in multiple states, I have seen the power of data firsthand. When done right, data-driven education can have powerful impacts on the learning outcomes of students.

Unfortunately, most school districts still are struggling with their data-driven practice. Much of this is because they continue to think about using data from a compliance mindset rather than using data for meaningful school improvement. An uninformed model of data-driven decision-making looks something like this:

This is the NCLB model. Schools are expected to collect data once a year, slice and dice them in various ways, set some goals based on the analyses, do some things differently, and then wait another whole year to see if their efforts were successful. Somehow, this model is supposed to get schools to 100% proficiency on key learning outcomes. This is dumb. It's like trying to lose weight but only weighing yourself once a year to see if you're making progress. Compounding the problem is the fact that student learning data often are collected near the end of the year and given back to educators months later, which of course is helpful to no one.

A better model looks something like this:

The key difference in this model is an emphasis on ongoing progress monitoring and continuous, useful data flow to teachers. Under this approach, schools have good baseline data available to them, which means that the data are useful for diagnostic purposes in the classroom and thus relevant to instruction. The data also are timely, meaning that teachers rarely have to wait more than a few days to get results. In an effective data-driven school, educators also are very clear about what essential instructional outcomes they are trying to achieve (this is actually much rarer than one would suppose) and set both short- and long-term measurable instructional goals from their data.

Armed with clarity of purpose and clarity of goals, effective data-driven educators then monitor student progress during the year on those essential outcomes by checking in periodically with short, strategic formative assessments. They get together with role-alike peers on a regular basis to go over the data from those formative assessments, and they work as a team, not as isolated individuals, to formulate instructional interventions for the students who are still struggling to achieve mastery on those essential outcomes. After a short period of time, typically three to six weeks, they check in again with new assessments to see if their interventions have worked and to see which students still need help. The more this part of the model occurs during the year, the more chances teachers have to make changes for the benefit of students.

It is this middle part of the model that often is missing in school organizations. When it is in place and functioning well, schools are much more likely to achieve their short- and long-term instructional goals and students are much more likely to achieve proficiency on accountability-oriented standardized tests. Teachers in schools that have this part of the model mastered rarely, if ever, complain about assessment because the data they are getting are helpful to their classroom practice.

NCLB did us no favors. It could've stressed powerful formative assessment, which is the driving engine for student learning and growth on whatever outcomes one chooses. Instead, it went another direction and we lost an opportunity to truly understand the power of data-driven practice. There are hundreds, and probably thousands, of schools across the country that have figured out the middle part of the model despite NCLB. It is these schools that are profiled in books such as Whatever It Takes and It’s Being Done (both recommended reads) and by organizations such as The Education Trust.

When done right, data-driven decision-making is about helping educators make informed decisions to benefit students. It is about helping schools know whether what they are doing is working or not. I have seen effective data-driven practice take root and it is empowering for both teachers and students. We shouldn’t unilaterally reject the idea of data-driven education just because we hate NCLB. If we do, we lose out on the potential of informed practice.

Thanks for the guest spot, eduwonkette!

A focus on student learning

From Karin Chenoweth at the Britannica blog:

Everyone in a school knows that some teachers are effective and some aren’t, but in most schools there is no organized way to ensure that students who get weak or bad teachers still learn what they need to learn. That’s not his fault, or the fault of any individual teachers who work hard; it is the fault of the way schools have been organized for generations.

and

[T]he essential issue at the heart of the controversy is whether schools should be asked to organize themselves to have student learning be at the center of all of their activities. I say they should. But the fact is that relatively few schools have done it.

The good old days before NCLB?

I ran across an interesting thread on NCLB, courtesy of Joanne Jacobs. First, Karin Chenoweth notes that the good old days before NCLB weren’t so good. TMAO concurred, stating that ‘more kids learned less in the past than now.’ Cardinal Fang, a commenter on Joanne’s post, disagreed: ‘in middle class [schools] … there is more emphasis on testing, arguably making those formerly good schools more boring, duller places by focusing on tests which most of the students would have passed anyway.’

What do you think?

1 year ago: Not enough time to be data-driven

School data tutorials

[cross-posted at LeaderTalk]

Even when principals and teachers have access to data, they often aren't sure what to do with it. That’s why CASTLE (okay, it was me!) created School Data Tutorials, a web site intended to help K-12 educators work with raw student and school data.

The tutorials on the web site highlight many of the Excel skills that are helpful when working with building- and district-level data. The tutorials are targeted at data managers, principals, guidance counselors, teachers, and other school personnel who have the responsibility for collecting, analyzing, and reporting K-12 performance data (which is just about everyone these days!). You will see that the tutorials are much like the ones created by Atomic Learning (they trained us!) but are focused on data-driven decision-making needs of educators rather than being generic.

Below are the four sets of tutorials we recommend for every teacher and administrator. If every educator knew how to do these four things, schools’ capacity to do some basic monitoring of student progress would be greatly enhanced. They take just over half an hour to watch once, and of course they can be viewed as many times as necessary to accomplish mastery.

Many more tutorials are available on the site, including instructions on how to make your own data collection templates. Let me know if you feel empowered after watching these!

Freeze panes

• Freezing rows to remain visible (1:11)
• Freezing columns to remain visible (0:40)
• Freezing both rows and columns to remain visible (0:49)

Sort

• Sorting data using a single criterion (3:04)
• Sorting data using multiple criteria (2:29)
• Sorting data with multiple header rows (1:27)

Filter

• Filtering data with AutoFilter (1:13)
• Analyzing data with AutoFilter (1:45)
• Using Custom AutoFilter to filter numerical values less or greater than a specified value (1:41)
• Turning filters off (0:34)
• Printing datasets from AutoFilter (1:39)
• Copying just the dataset obtained from AutoFilter (2:23)
• Using the Esc (Escape) key to clear selections and copy another dataset (1:16)

Conditional formatting

• Changing a numerical cell's color with single conditional formatting conditions (2:17)
• Changing a numerical cell's color with multiple conditional formatting conditions (1:50)
• Changing a text cell's color with single conditional formatting conditions (0:55)
• Changing a text cell's color with multiple conditional formatting conditions (2:53)
• Changing a cell's color with calculated value conditions (3:58)

Note that you may need to turn off your browser's pop-up blocker or install the latest Flash plug-in to view these tutorials. Happy viewing!

ROTW: Assessing students with disabilities

The latest installment of the Report of the Week (ROTW) comes from Education Sector:

• Labeled: The Students Behind NCLB’s ‘Disabilities’ Designation

Here’s a quote from the report:

[B]ecause the majority of special education students have disabilities that do not preclude them from reaching grade-level standards there doesn't seem to be a need to rollback NCLB's accountability measures for students with disabilities. To the contrary, doing so could hurt such students by reducing schools' responsibility for ensuring that they are taught to high standards.

Happy reading!

Interview with Phala Daniel

I have had the pleasure of working with the Chicago Public Schools (CPS) on several occasions over the past few years, primarily in conjunction with its Principal Technology Leadership Institute (PTLI), which is a collaborative venture with the International Society for Technology in Education (ISTE). I really enjoy the CPS folks. They face numerous challenges but are some of the most dedicated educators I know.

Some of the principals participating in the data-driven decision-making strand of the PTLI (which I helped design) had some questions that they wanted me to answer about data-driven leadership, so today I had a very pleasant conversation with Phala Daniel, who works in CPS' eLearning office along with Gerry Beimler (who is one of our CASTLE graduate certificate students). I recorded our 37-minute discussion and present it for your listening pleasure:

• May 25, 2007 CPS DDDM Podcast (36:36, 33.35 MB .mp3)

FYI, here's one example of a protocol for a data-driven team meeting (as mentioned in the conversation). Hope you enjoy the podcast. Let me know what you think!

Gapminder

I have seen the future of data presentation and it is Gapminder. Visit Gapminder and click on

• the Play button (bottom left)
• the geographic regions (top right)
• country names (right)
• the magnifying glass (bottom right of chart)
• the Map tab (top left)

Also use the three sliders at the bottom (under Income per capita in international dollars and above and below Population). Notice also that merely hovering your mouse over different parts of the graphs highlights various information aspects. Finally, you can change both the x and y axes by clicking on the dropdown box triangles for Income per capita in international dollars (right of label) and Life expectancy, years (top of label).

I had seen the video of Gapminder founder Hans Rosling at TED Talks, but it's fun to get to play with the software a little bit. How great would it be to have this tool for K-12 data-driven decision-making?

Wow.

Decision-making department

Does your school organization make decisions because they sound good or because internal analysis shows they're the right decisions to make? In other words, what department do your decisions fall under? Marketing or R & D? 

Put your hands up and move away from the textbook

Listen to this post!

Dan Meyer is a dynamic young math teacher for the San Lorenzo Valley Unified School District. He totally gets it when it comes to assessment. Here are four great posts, the first three of which I wish I could get every teacher and administrator in Minnesota to read:

• How math must assess (be sure to read Dan's 'mini-thesis')
• From the mailbag: Kay Endriss
• "NCLB is forcing me to teach to the test."
• The presentation (includes a Quicktime movie)

Thanks, Chris, for introducing me to Dan's blog. You called him a 'must-read.' I think you're right.

Not enough time to be data-driven

I do a lot of work with schools on data-driven accountability issues. Before you immediately decide that I’m just another data huckster, I’ll point out now that my work with schools focuses on good ongoing, formative assessment for student progress monitoring purposes rather than on the stupid yearly summative autopsy data that most schools are spending WAY too much time on. Intelligent use of progress monitoring data related to key academic and other educational goals has been shown time and time again, in both high-quality research studies and in tens of thousands of schools and classrooms across the country, to have significant impacts on student learning outcomes. The most common complaint that I hear from teachers, however, is that they’re already pressed - they don’t have time to add another thing to their plate. Balderdash. Here are a few things that teachers can get rid of that will free up some valuable time.

Teachers work extremely hard. They’re some of the most caring, dedicated people I know. But, like most of us, they often don’t use the time that they have very effectively (or others don’t use their time very intelligently). If we truly care about student learning, we should be taking a critical look at teachers’ precious time and try to eliminate many of our low-yield practices.

• How about coloring? As Mike Schmoker points out so eloquently, students in elementary schools spend a LOT of time coloring. Couldn’t teachers do a little less of this and spend some of that time doing quick, timely, useful assessments of student learning and/or individualized instruction for struggling students?
• Cursive writing, anyone? Does anyone think this will be a needed skill in the digital future? How much time and energy do teachers and students still spend on this?
• One-shot, single day staff development sessions are almost always completely useless if our desired goal is to facilitate changes in teacher practice. What is remarkable is the persistence of this practice despite everything that we know about adult learners and effective professional development. Couldn’t we use this time for teachers to collaboratively examine student learning progress instead?
• As Alfie Kohn reminds us, and despite decades of researchers trying, there is no evidence to support that homework in the elementary grades has any positive benefit whatsoever on student achievement, study skills, greater responsibility by students, etc. Teachers and students spend a lot of time processing homework - time that could be better spent elsewhere.
• Watching entire movies in class instead of short video clips. Enough said.

You get my drift. The list probably could on for a while - each of you can think of other things that teachers could eliminate or do differently to free up valuable time for high-leverage instructional strategies (add them below as a comment!). So what could teachers do instead with the time they gain? Here are a few things:

• Collaboratively design short assessments that would allow them to monitor student progress on key learning goals
• Collectively examine the data that they receive from these assessments and formulate instructional modifications for learners that are still struggling
• Participate in ongoing, long-term learning groups that help them gain new skills (or new technologies!) for addressing the needs of struggling learners

In other words, they could do things that we know to have better instructional payoff than some of what teachers are doing now. Plus, there’s the very sobering list from Mike Schmoker’s newest book, Results Now, that reminds us that teachers often don’t do things they already know have high payoffs instructionally (from a study that did thousands of classroom observations):

• Classrooms in which there was evidence of a clear learning objective: 4 percent
• Classrooms in which high-yield strategies were being used: 0.2 percent
• Classrooms in which there was evidence of higher-order thinking: 3 percent
• Classrooms in which students were either writing or using rubrics: 0 percent
• Classrooms in which fewer than one-half of students were paying attention: 85 percent
• Classrooms in which students were using worksheets (a bad sign): 52 percent
• Classrooms in which noninstructional activities were occurring: 35 percent

As Schmoker notes, “such statistics point to how even fairly obvious actions could have an immediate and enormous impact on students and their levels of learning” (p. 18).

The phrase “work smarter not harder” is trite and often is used in a condescending manner. Many times, unfortunately, it also happens to be true.

This post is also available at the TechLearning blog.

We need DDDM evaluation panel participants

See below - a message I sent out over a few listservs - thought I'd post it here too. Please forward on to others and consider participating yourself (if appropriate). Do some good - earn $50!

--------------------

Dear colleagues,

We are looking for teachers, principals, assessment directors, and district technology coordinators who are interested in formative assessment and are willing to participate in an evaluation panel. The purpose of the panel is to review and contribute to the design of a formative assessment tool. This project is funded by the U.S. Department of Education - its purpose is to design an affordable tool that would assist teachers with data collection and analysis on teacher-created and other formative assessments to help improve student achievement.

Each session will be no more than 1 hour long and will be conducted via webcast. Educators will receive a $50 stipend for participating.

Teachers from all subject areas are welcome. Sessions will be conducted in early October and early November. Please view the schedule at the end of this message for exact dates and times.

Educators can participate from home or from school. To participate in the webcast, each participant will need a phone to call into the discussion (toll-free) and a computer to see the presentation. Most individuals should be able to easily get into the web conference, but you may wish to check these system requirements:

• A stable 56k, cable modem, ISDN, DSL, or better Internet connection
• For Windows users: Internet Explorer 5.0 or newer, Netscape 6.0 or newer, Mozilla Firefox 1.0 or newer (JavaScript and Java enabled)
• For Mac OS X users: Safari 1.3 or newer, Firefox 1.5 or newer on PowerPC G3/G4/G5, Firefox 1.5.0.2 or newer on Intel (JavaScript and Java enabled)

If you are interested in participating, please reply to this e-mail as soon as possible by clicking the reply button and typing in your answer to each of the questions below - please address your e-mail to [email protected]. A confirmation message with the toll-free phone number and the web link you need to access the webcast will be sent to you.

• Your name?
• Your position? (teacher, principal, assessment director, district technology coordinator)
• If a teacher or principal, are you in an elementary, middle or high school?
• If you are a teacher, what subject do you teach?
• In what state are you located?
• Session date/time that is your 1st preference? (see below)
• Session date/time that is your 2nd preference? (see below)

Thank you in advance for your assistance with this important project that aims to create helpful tools to meet educators’ data-driven accountability needs. Please share this message freely with other educators.

Dr. Scott McLeod, Assistant Professor
University of Minnesota

----------------------

Session Dates / Times (all times Eastern)

Elementary School Teachers:

Oct 16 - 7:30 to 8:30 PM EST 

Oct 17 - 8:30 to 9:30 AM EST 

Oct 23 - 7:30 to 8:30 AM EST

Oct 25 - 9:30 to 10:30 AM EST

Nov 6 - 7:30 to 8:30 AM EST

Nov 8 - 9:30 to 10:30 AM EST

Nov 9 - 8:30 to 9:30 AM EST

Nov 10 - 10:30 to 11:30 AM EST

Nov 13 – 7:30 – 8:30 PM EST

Middle School/ Jr. High Teachers:

Oct 16 - 8:30 to 9:30 AM EST

Oct 16 - 7:30 to 8:30 PM EST 

Oct 17 - 10:30 to 11:30 AM EST

Oct 23 - 9:30 to 10:30 AM EST

Nov 8 - 7:30 to 8:30 AM EST

Nov 9 - 10:30 to 11:30 AM EST

Nov 10 - 8:30 to 9:30 AM EST

Nov 13 – 7:30 – 8:30 PM EST

High School Teachers:

Oct 16 - 3:30 to 4:30 PM EST

Oct 17 - 6:30 to 7:30 PM EST

Oct 23 – 8:30 – 9:30 PM EST

Nov 13 - 3:30 to 4:30 PM EST

Nov 14 - 5:30 to 6:30 PM EST

Nov 15 - 4:30 to 5:30 PM EST

Nov 16 - 8:30 to 9:30 PM EST

Principals:

Oct 16 - 1:00 to 2:00 PM EST

Oct 24 – 6:30 – 7:30 PM EST

Nov 13 - 10:00 to 11:00 AM EST

Nov 14 - 11:00 AM to 12:00 PM EST

Nov 14 - 7:30 to 8:30 PM EST

Nov 15 - 12:00 to 1:00 PM EST

Nov 16 - 1:00 to 2:00 PM EST

Assessment Directors:

Oct 17 – 8:30 – 9:30 PM EST

Oct 24 - 2:00 to 3:00 PM EST

Nov 6 - 1:00 to 2:00 PM EST

Nov 7 - 4:00 to 5:00 PM EST

Nov 8 - 3:00 to 4:00 PM EST

Nov 9 - 2:00 to 3:00 PM EST

Nov 15 - 7:30 to 8:30 PM EST

Technology Coordinators:

Oct 17 – 8:30 – 9:30 PM EST

Oct 25 - 1:00 to 2:00 PM EST

Nov 6 - 3:00 to 4:00 PM EST

Nov 7 - 2:00 to 3:00 PM EST

Nov 8 - 1:00 to 2:00 PM EST

Nov 9 - 4:00 to 5:00 PM EST

Nov 15 - 7:30 to 8:30 PM EST

Note: If none of the above dates/times work for you but you’re interested in participating, please e-mail [email protected] with preferred dates and times. We may be able to add more sessions.

DDDM and Moneyball

I've been wanting to write this piece for a long time, but never figured out the right outlet.  This blog, however, is a great space for me to try it out (ah, the beauty of blogging!).  Plus, I think I did reasonably well with my Freakonomics analysis on Monday, so I figure it's safe to work my thoughts through another popular non-fiction book.

Moneyball, by Michael Lewis, is a book about baseball.  So, I don't know how many readers of this blog will have read the book.  But, for me, the book is about so much more than baseball and the lessons of the book are incredibly applicable to education.  Lewis initially set out to write a book about how the Oakland A's, a major league baseball team on the low end of the financial resources continuum, managed to compete at such a high level seemingly every year.  There are tremendous disparities in spending across Major League Baseball, and the game has been plagued by the perception that the teams that spend the most money win the most.  The New York Yankees (my favorite team!) were the face of this perception.  The A's seemed to defy this perception.

What Lewis ultimately discovered was that the A's, led by their unique general manager Billy Beane, had adopted an organizational commitment to scouting and assessing players using statistical analysis of the loads of data generated by the game of baseball.  These forms of analysis, labeled generally as sabermetrics, had been around for many years, but they were largely written off as the province of geeks and statheads who just happened to like baseball.  Beane, however, came to believe that a sabermetric approach to scouting and valuing players would allow them to make the most cost-effective decisions possible. They were able to figure out which individual statistics were the greatest predictors of team success.  Then they sought players who thrived in those key areas despite being deemed by others as flawed in other areas of the game thereby devaluing their salaries.  So, for example, sabermetricians were able to demonstrate that on base percentage (OBP - the likelihood that a player will get on base in any given plate appearance) was the single greatest predictor of runs scored.  That seems logical since a player must get on base to score a run.  But, traditionally, players were valued based more on their batting average (the likelihood that a player will get a hit in any given plate appearance) than OBP.  Therefore, there were some players who didn't have great batting averages (and as a result didn't earn very high salaries), but had relatively higher on base percentages (mostly because they earned lots of base-on-balls - aka walks).  Those players became Oakland A's.

Most baseball purists and old school baseball people fervently opposed this sabermetric orientation.  They argued that you couldn't judge a player by crunching numbers.  You had to watch the players play, get to know them as people, etc.; in other words, make value determinations by scouting the old fashioned way.  The numbers were cold and unreliable, they'd say.

Is this starting to sound familiar?  Purists, traditionalists arguing that we should not rely on numerical data to make decisions?  Numerical data are cold and unreliable, and they can't tell you what you need to know about people?  These are the same arguments you hear from those opposed to what has been labeled "data-driven decision-making" (DDDM) in education. 

I could stop here and argue that the Oakland A's commitment to sabermetrics and cost-effective decision-making has been highly successful (just look at how well they're doing in this year's playoffs!), so everyone should buy in to DDDM in education.  But, that's not the real point I want to make.

For me, there is another perception problem here.  The popular sports media has, for the most part, portrayed this so-called Moneyball philosophy inaccurately.  The popular sports media would have us believe that sabermetric analysis is an opposing paradigm to traditional baseball scouting methods.  But, the fact is that sabermetric analysis has been used by the A's (and now many other teams as well) as a complement to more traditional methods of scouting and player valuation.  It is not as if the A's have fired all of their scouts and hired all statisticians; their scouting department includes a few number crunchers in addition to all of the scouts who do what they've always done.

Similarly, in education, "data-driven decision making" is the label given to the movement to making decisions based on the scores of numerical data that are now available to educators as technological means (computers, databases, etc.) have intersected with a climate of standards and assessment.  But, to suggest that DDDM is a new movement or idea in education implies that before now, decisions were made in a vacuum; decisions were made in the absence of data.  That's not the case, though.  Decisions, particularly those about individual students, were made based on professional judgments (teacher perceptions, observations, etc.).   Like sabermetrics in baseball, (statistically oriented) DDDM is a complementary approach to professional judgment in education.  They are epistemologically different approaches, but they are not mutually exclusive. 

Finally, the Oakland A's needed to add sabermetric analysis to their organization because they were playing on an uneven playing field with respect to financial resources.  As a result, they have been able to compete successfully against the big spenders.  Education is a notoriously uneven playing field with respect to financial resources.  I hope schools and districts struggling with relatively low per-pupil expenditures see DDDM as a way to make more cost-effective decisions.

A Freakonomic Analysis

If you haven't read the popular non-fiction book Freakonomics, I highly recommend it.  Or, if it's more your speed, you can visit the website associated with the book.  The authors of the book, Steven D. Levitt and Stephen J. Dubner, even have their own blog!

Levitt and Dubner are running a series of articles in the New York Times Magazine that read much like the chapters in their book.  The most recent article (free registration required, sorry) ran last week and sparked a number of thoughts about the issues around which this blog revolves.  So, I thought I'd use this post to share those thoughts.

The article is about one doctor's efforts to improve hand-hygiene compliance at Cedars-Sinai Medical Center.  Dr. Leon Bender, the hospital's Chief of Staff, along with other hospital staff, devised a number of incentive schemes to get doctors to be more diligent about washing their hands.  They tried the usual campaign of memos, e-mails, etc.  They formed a "Hand Hygiene Safety Posse" that walked around the wards handing out Starbucks gift cards to doctors caught in the act of washing their hands.  These efforts increased compliance to about 80%, but they needed 90% compliance to satisfy their accrediting agency (really?  only 90% compliance is sufficient?). 

So, one day, during a meeting of Dr. Bender's Chief of Staff Advisory Committee, the hospital's epidemiologist took a culture of the hands of the committee members.  The members pressed their palms into agar plates, and the plates were cultured and photographed.  When the results came back, the images were apparently "...disgusting and striking, with gobs of colonies of bacteria.”  The hospital administration decided to use the power of those images by making one particularly disgusting image the screen saver on all hospital computers.  Suddenly, hand-hygiene compliance shot up to nearly 100%.

Dr. Bender's explanation is what struck me the most.  He said, "With people who have been in practice 25 or 30 or 40 years, it’s hard to change their behavior.  But when you present them with good data, they change their behavior very rapidly.”

There are obvious implications of that quote for DDDM, but I'll pass on that discussion until Thursday's post.  For today, I'm more interested in the implications for school leaders trying to facilitate technology integration (a term of art, BTW, that will be the subject of tomorrow's post).  One of the greatest challenges facing technologically-inclined school leaders is getting veteran teachers (even those who've been teaching the same way for 10 or so years) to change their behaviors.  After Dr. Bender's explanation, the NYT article goes on to say that "some forms of data, of course, are more compelling than others, and in this case an image was worth 1,000 statistical tables."

So, the question for me is, "what kinds of 'data' (broadly conceived) can we present to teachers that would compel them to change their behaviors and to integrate technology more into the teaching and learning process?"  I like to show my students a PowerPoint slide that contains two photos side-by-side.  One image is of a classroom from the 1880's; the other is of a classroom from the 1980's. 

   

(photos from Raymond Bial's One Room School and PBS's evolving classroom series)

Not surprisingly, the classrooms look nearly identical.  That photographic juxtaposition is usually pretty compelling.  I also think David Warlick's letter from principals is a compelling artifact.

What other data can you think of?  Are there other "frightening" images or artifacts we can use to "scare" teachers (and administrators, for that matter) into changing their behaviors?

Minnesota DDDM readiness study

Since David posted about one of his data-driven decision-making (DDDM) projects yesterday, I thought I'd chime in with some info about a massive DDDM initiative underway here in Minnesota...

The Minnesota Statewide DDDM Readiness Study is an attempt to answer two basic questions:

• Are Minnesota teachers and administrators ready to engage in data-driven education?
• What needs do Minnesota educators have as they work to integrate data into their daily practice?

To try and answer these questions, paper surveys were sent out to every public school principal, superintendent, and technology coordinator in the state of Minnesota, including charter schools. In addition, surveys were sent out to five randomly-selected teachers in every public school building. Participants had the option to complete the survey online instead of on paper. A total of 13,850 surveys went out for this initiative; we've heard back from over 4,200 educators across the state. The next phase of the project hopefully will be to complete some follow-up interviews.

We're currently doing the first round of data analysis. If you're interested in the survey instruments themselves, they're available here:

• Teacher survey
• Principal survey
• Superintendent survey
• Technology coordinator survey

We believe this is the first statewide DDDM needs assessment in the country and we are delighted that the Minnesota Department of Education saw fit to fund this project. If your state has interest in using the surveys we created, we have the ability to host those online for you. The general URL for the project is www.minnesotadatasurveys.org.

Data-Driven Decision Making and Change

I have learned much about data-driven decision making (DDDM) from Dr. Scott McLeod.  He is an acknowledged authority on DDDM and is especially knowledgeable about frequent formative assessment. My own interest in DDDM is focused on how individuals can use data to lead change in schools and create cultures that are more data-driven.  I wrote a monograph for NASSP in 1999 entitled "Using Data for School Improvement".  At the time I could see where the winds were blowing in the US in regards to school accountability.  Looking back, I don't think I could have predicted how far along that continuum we have traveled.

In principle, most all of us believe in the ideals of DDDM which Scott has discussed extensively on this blog.  The problem is that for many policy makers and even educators, the only "data" that counts in DDDM are student test scores.  While I agree that academic success should be the primary focus, this perspective dismisses so much of what teaching and learning is really about.  My first grade daughter is an incredible artist (yes I am biased) and has been inspired by a wonderful public school art teacher for the last two years, but I worry that talent may not be nurtured in future grades because schools are divesting from art, music and other "non-academic" subjects to devote more resources for reading and math.

To investigate my interest in the relationships among DDDM, leadership, change, and school culture, I spent part of last year creating a survey instrument to assess teacher perceptions.  I have used the survey "Data-Driven Decision Making in Schools" in ten schools so far and I have begun factor analysis procedures.  One of the high school principals whose school completed the survey indicated that she has used the results to create her own professional development plan for personal growth.  She is building on her strengths as a leader of DDDM and is formulating action plans to address areas of concern.  I hope to conduct in-depth research and use this instrument to help school leaders understand teachers' beliefs and learn what their teachers know or do not know about DDDM.  The initial data that I have analyzed does not paint a positive picture.  Not only are teachers lacking the knowledge about DDDM and concepts like frequent formative assessment. The data also indicate that teachers and even principals in many schools do not have access to the data that they need to make informed decisions about instructional practice. These are organizational and structural barriers that have to be addressed.

While educators sometimes complain about the application of business related research and theory to educational organizations, I have found much in this literature that has informed my work in DDDM.  I am especially impressed with the Balanced Scorecard concept by Kaplan and Norton and the practical application that it holds for education.  I am about to submit a manuscript for publication with one of my doctoral students, examining the relationship between DDDM and the Balanced Scorecard. Speaking of manuscripts, I have to take my leave to edit one now. DMQ

School data tutorials

Educators today are expected to integrate the collection and analysis of student learning data into their ongoing instructional and organizational practice. Yet if you walked into almost any school in this country and asked how many teachers knew how to make a basic graph using spreadsheet software, you'd be lucky if a fourth of them raised their hands (and don't even bother asking the administrators!). So what can we do to help educators learn some basic technology skills that can help them track and monitor student learning progress over time?

CASTLE's School Data Tutorials web site was created to assist school districts and educational leadership preparation programs with the task of training K-12 educators to work with raw student and school data. The web site contains over 100 short Flash screencasts that walk educators through some Excel skills that are extremely helpful when working with classroom-, building-, or district-level data. Popular tutorials that are immediately useful to teachers and principals include

• freeze panes
• sorting
• filtering
• conditional formatting
• pivot tables and charts

Most of the screencasts are only a minute or two long and can be watched a thousand times if necessary. This is extremely useful for educators who are not that tech-savvy and may not be able to keep up with the group during face-to-face training. The online availability also allows educators to return to the site later to review something if they forget what they learned previously.

We have been getting tremendously positive feedback about this resource. Educators who previously saw no intersection of spreadsheets with their professional lives are contacting us, indignant that no one has ever showed them these skills previously. Comments like "I could have saved myself a lot of time" or "Now I have a way to do this easily" are extremely common. One of the reasons I think the tutorials are so well-received is because they're grounded in the important, real-life, data-driven decision-making needs of teachers and administrators (as opposed to the generic, context-less Excel training that occurs in many school systems).

This summer we added a section to the site that walks educators through the steps of creating their own data collection templates. This section, which is not quite as polished as the rest of the site (e.g., I think you can hear my in-laws' computer's aquarium screensaver in the background!), contains examples for principals, teachers, and grade-level teams. In the coming months, we will upload additional examples of templates that are being used in schools across the country.

The tutorials currently are being used in schools, districts, and university preparation programs across the nation. I encourage you to check out the site (maybe you'll learn something helpful!) and to spread the word about this resource. We created it to help build some capacity in school systems. Right now data analysis expertise resides primarily in central office research and assessment departments - we need more knowledge in our buildings and classrooms.

More information is available at the web site if you'd like to know more about this resource or our critical partners. Any feedback or suggestions that you have are extremely welcome - simply comment on this post!

Technology tools for data-driven teachers

In addition to the white paper that I wrote for Microsoft that summarized essential data-driven decision-making concepts for teachers and principals, I also wrote a second white paper that summarized the various technology tools that school districts are using to manage summative and formative data.

Technology Tools for Data-Driven Teachers is only seven pages long and is a brief overview of data management and analysis (DMA) systems (i.e., data warehouses) and instructional management and assessment (IMA) systems. DMA systems are primarily used for management and analysis of summative data, such as those from yearly state assessments, and typically interface with districts' student information systems, electronic gradebooks, and other district databases. IMA systems typically are not integrated with DMA systems and are used to separately store and analyze data from ongoing progess monitoring efforts. I also briefly discuss in the white paper some other options that school districts have if they can't afford these often-expensive data systems.

Technology Tools for Data-Driven Teachers is a nice overview for educators who are unfamiliar with the technology options available for storing and analyzing student and school data. The white paper is deliberately written in non-technical language to facilitate easy understanding by administrators and teachers who probably are not technology wonks. Whether your district is reexamining existing tools for managing data or is looking for new solutions, the white paper can help give your educators get the big picture of what's out there as they try to make sense of different vendor offerings.

Two other resources that may be helpful are my ongoing lists of DMA system vendors and IMA system vendors. School districts often don't have the ability or inclination to do a full market scan of possible products and thus often miss vendor solutions that may better fit their needs. I don't claim that my categorization is perfect - sometimes products blur the lines between categories. If you have a vendor to add to one of these two lists, please comment on this post!

Data-driven teachers

I've been getting a lot of positive feedback on the white paper that I wrote last year for Microsoft so I thought I would share it as a resource here. I wrote Data-Driven Teachers with the intent of trying to concisely summarize the great work being done on data-driven school reform by folks such as Mike Schmoker, Rick DuFour, Victoria Bernhardt, Douglas Reeves, Rick Stiggins, etc.

Data-Driven Teachers is only ten pages long and utilizes the data-driven change framework that I have used with thousands of educators from across the country. Each section of the paper describes what teachers and principals should be doing for that particular data-driven concept. The last page of the paper is a table from a draft of the Professional Development Framework for Data-Driven School Improvement that I created for the Chicago (IL) Public Schools.

Data-Driven Teachers is a nice introduction to the concepts of data-driven school reform and is a good reminder of some essential activities that often are missing in school organizations. School districts have been using it to quickly ramp up their teachers and administrators conceptually, to give their educators a common framework and language for discussion, and to remind staff of key actions that need to occur as they work to better utilize student learning data to improve instructional and organizational decision-making.

If you find the white paper useful, drop me a comment!

Low expectations for children

In a previous post, I commented on the perceptions of many K-12 educators that their school's academic success is hostage to their student demographics.

There's another angle to this - the belief by some (but, thankfully, not most) educators that current achievement levels are just fine. I ran across this again recently when I spoke to a school district's entire teaching staff about utilizing formative assessment data to improve overall learning outcomes and close existing achievement gaps. Frequent formative assessment has been shown by countless research studies, and by successful data-driven schools across the country, to have powerful impacts on student achievement, particularly that of low achievers.

One of the teachers left this comment on her anonymous index card at the end of my presentation:

I'm glad that my children don't attend a school where it's okay for a teacher to believe that leaving 1 in 5 kids behind is just fine. The sad and scary fact is that there are entire communities that have expectations this low (or even lower) for their children.

Defeatist schools

A recent post by Kevin Carey at The Quick and the Ed highlights one of the essential dilemmas faced by those of us who are working desperately to improve students' academic and life success: there is a pervasive attitude in K-12 organizations that outputs are dependent on inputs. You routinely hear comments from educators such as "You can't expect us to do any better than we already are with these kids" or "The reason that [school / district] is doing better than we are is because they serve those kids."

"We believe that we have no meaningful impact on the children that we serve. We are hostage to our demographics. Whatever comes in the door is essentially what's going to go out at the other end." Those are chilling words to hear, both as an educational leader and as a citizen of the most affluent and powerful nation in the world.

Despite the ritualistic mantra of educators that "all children can learn," there are large numbers of teachers and principals who don't truly believe it. If they did, they would act in ways much differently than they do now.

Luckily we have (increasingly numerous) examples of schools where this belief has been challenged at its very core, where educators have come together and said "Collectively we can make a difference!" These schools are finding ways to make it happen, despite their challenging demographics. It starts with a belief that it can be done. As Dr. Douglas Reeves notes in The Learning Leader:

Norfolk Public Schools in Virginia has the following demographic characteristics:

• 80 percent of students receive free or reduced-priced lunch
• 68 percent minority student enrollment
• 40+ languages

Between 1998 and 2005 not a single child in the school system, to the best of my knowledge, has changed his or her ethnic identity. Not a single child has won the lottery. Few if any children have adopted different languages at home. In other words, this story is not about changes in children or their families, nor is it a story about changes in demographic characteristics. This is a story about changes in teaching, leadership, and learning. While demographic characteristics remained the same, student achievement rose dramatically.

In 1998, only 11 percent of the elementary schools in Norfolk contained more than 50 percent of students who scored proficient or higher on the state's English/Language Arts assessments. In 2004, 84 percent of the elementary schools achieved that distinction, and in 2005, 100 percent of the elementary schools in the district were fully accredited. They not only had 50 percent of students meeting or exceeding state standards in English / Language Arts, but these students also met state requirements in math, science, writing, and social studies. In 1998, none of the middle schools in this district had more than 50 percent of students meeting state English / Language Arts requirements, and six years later all the middle schools met this requirement. In addition, the district more than tripled the number of middle school students taking advanced math courses in middle school. In 1998, only one out of six high schools had more than 80 percent of students passing state English graduation requirements, and six years later every high school in the system achieved this distinction. Moreover, some high schools had more than 90 percent of students passing external exams in chemistry and biology, while the dropout rate remained an astonishingly low 0.5 percent for the district - one of the lowest high school dropout rates of any urban system in the nation. The students didn't change. They were still ethnically, linguistically, and economically diverse. But something profound did change - the commitment of the leaders and teachers in this district to make a difference in the lives of students.

Yes, we absolutely need to be sensitive to overall contexts and larger societal issues. I, too, am concerned about the increasing gaps between the top and bottom income brackets of our society. I, too, am gravely worried that our political and educational policies may be causing increasing harm to our most disadvantaged populations. We must seek to understand the complex contexts in which our students live and we must fight and advocate and cooperate until progress is made. But we must never, ever give up, even mentally. For then the battle, and all hope, is lost.