Sunday, August 7, 2016

It's not the science that is junk, it's the measures, Part II

So a day or so after my last post, It's not the science that is junk, it's the measures, I came across this interview of Jesse Rothstein by Rachel Cohen in the American Prospect. There's lots of good stuff in there and it's worth reading. I don't mean to take away from the import of Jesse Rothstein's work (I am a big fan of his work and of Rachel Cohen's work) but a piece of it kind of demonstrates what I was trying to get at in my last post.

Talking about VAM, Rothstein said,
It’s very controversial and I’ve argued that one of the flaws of it is that even though VAM shows the average growth of a teacher’s student, that’s not the same thing as showing a teacher’s effect, because teachers teach very different groups of students. 
If I’m a teacher who is known to be really good with students with attention-deficit disorder, and all those kids get put in my class, they don’t, on average, gain as much as other students, and I look less effective. But that might be because I was systematically given the kids who wouldn’t gain very much.
So, yes, this is a very good point: there is a difference between showing "the growth of a teacher's student" and "showing a teacher's effect."  And yes, according to test scores, and how well students perform on them, teachers can look more effective or less effective, regardless of how good they are at teaching.

The he says, when she asks if he is skeptical of VAM,
I think the metrics are not as good as the plaintiffs made them out to be. There are bias issues, among others. One big issue is that evaluating teachers based on value-added encourages teachers to teach to the state test. 
During the Vergara trials you testified against some of Harvard economist Raj Chetty's VAM research, and the two of you have been going back and forth ever since. Can you describe what you two are arguing about?  
Raj’s testimony at the trial was very focused on his work regarding teacher VAM. After the trial, I really dug in to understand his work, and I probed into some of his assumptions, and found that they didn’t really hold up. So while he was arguing that VAM showed unbiased results, and VAM results tell you a lot about a teacher’s long-term outcomes, I concluded that what his approach really showed was that value-added scores are moderately biased, and that they don’t really tell us one way or another about a teacher’s long-term outcomes.
If you look at this response and then go back to the previous one I pulled out, you see that Rothstein is referencing "growth" and then "bias." That certain types of students won't "gain as much as other students" and that the value-added scores are "moderately biased" and that they don't tell us much about a teacher's "long-term outcomes."

Nowhere in there is there a repudiation of the measures, of the tests themselves, or even a question about their validity. His responses seem to assume that determining a teacher's effectiveness according to test scores is unfair because some students won't perform on them and that these tests can show growth and gains in learning.  Nowhere does he question that the tests themselves might not be reflective of real learning, good teaching, or of quality education.

And then the bias and assumptions critique, that has to do with the model, and not with what is being fed into the model, i.e., test scores. Arguments about the strength of statistical models are worth having but those should start with probing what's being fed into them.

If someone like Jesse Rothstein isn't questioning that, then test-based accountability isn't going away anytime soon. It will forever be a matter of tinkering with models.

Wednesday, August 3, 2016

It's not the science that is junk, it's the measures

So I recently had occasion to read a whole bunch of studies on charter schools and one type I read was about their effectiveness. I read the CREDO studies and I read critiques of the CREDO studies and I read meta-analyses and I read smaller studies.

Anyway, I want to go back to something I used to say and that I have heard others who are similarly skeptical of Big Ed Reform, and that is the notion of "junk science." A lot of us have called VAM and have called other studies of educational effectiveness "junk science." I know I did, indignantly. But you know what? I didn't really know what I was saying. (This is one reason I went back to get my PhD, so I would have more understanding of these kinds of things.)

And I was reading all of these studies on the effectiveness of charter schools, I remembered reading this post by Matt DiCarlo on the Shanker Blog from over 3 years ago. I remembered that reading it gave me pause about calling what I did "junk science" and I ceased doing so, but even so, I couldn't fully relate to what he was saying:
Now, I personally am not opposed to using these estimates in evaluations and other personnel policies, but I certainly understand opponents’ skepticism. For one thing, there are some states and districts in which design and implementation has been somewhat careless, and, in these situations, I very much share the skepticism. Moreover, the common argument that evaluations, in order to be "meaningful," must consist of value-added measures in a heavily-weighted role (e.g., 45-50 percent) is, in my view, unsupportable. 
All that said, calling value-added “junk science” completely obscures the important issues. The real questions here are less about the merits of the models per se than how they're being used. 
If value-added is “junk science” regardless of how it's employed, then a fairly large chunk of social scientific research is “junk science." If that’s your opinion, then okay – you’re entitled to it – but it’s not very compelling, at least in my (admittedly biased) view.
I am still no statistics expert and I never will be, but I have a much greater appreciation for what these models and analyses can tell us and what they don't tell us and what their limitations are. And these researchers conducting these studies, they may have different ways of conducting the studies and different opinions regarding which factors should be included and which shouldn't, but they know what they're doing, most of them at least, and they go to great pains to be thorough and thoughtful about their design and methodology and to explain the models they're using and to account for the results that these models produce. So the problem is not with the science.

DiCarlo says the problem is in how the models are being used. Yes. But another problem, as far as I could glean, is with the measures they're using. "Student learning” and “student achievement” have come to be represented by test scores. That is not my currency of educational quality, but it is the current currency in educational research and policy. I think many of these tests are of dubious quality and I don't think that they provide a true measure of what students have actually learned and or of the quality of their educational experience. Richer, deeper, more authentic student learning in charter schools, and schools in general, can be measured if we think creatively and holistically about it. But we're not doing that and we're not incentivized to do that. So much of the money for educational research, so much of the recognition, goes to researchers who use these test scores as measures. Because there's not much else. Even researchers who don't agree that they are good measures will say as much in one paragraph and then cite them as evidence of effectiveness or lack thereof in the next. 

To me, it's kind of like chicken nuggets and milkshakes. McFastFood place has a sound process for making chicken nuggets and milkshakes, but once all is said and done, how much actual quality chicken meat and milk come out of the other side? How much actual nutrition? How much actual, recognizable learning and educational quality gets funneled through these tests and comes out of the other side of these statistical analyses that use test scores as measures?

I doubt much.

Saturday, July 30, 2016

NAACP calls for a moratorium on charter schools but the devil is in the (state) details

Julian Vasquez was one of the first, if not the first, to report that the NAACP called for a moratorium on charter schools. Please read his post as well as the copy of the resolution itself that he posted.

Here are my thoughts:

This is a big deal. In many cases, or at least some, civil rights groups and those representing black and Latino communities were not initially, necessarily against charters. Even if they had reservations, many saw them as an alternative to the traditional public schools TPSs) where many black and Latino have been not been served well by traditional public schools. It is easy for those of us who have been well served by traditional public schools to oppose charter schools, but it's not been a no-brainer for those who haven't been and who can blame them. That the national NAACP has essentially put their foot down on this means something.

However . . .

1. (And their resolution acknowledges this if you examine the language) charter laws are made state-by-state.  Certainly, federal policy, such as RTTT (Race to the Top) can influence the proliferation of charter schools with funding incentives (or lack thereof) and other types of support. But it's at the state level where charters are born and made and it's state laws that determine their regulation, their proliferation, and their governance structures. Every state is different. So while this will have influence at the national level, we'll have to see what individual NAACP chapters do, what each individual's state charter laws and climate is, and what state legislatures do.

2. I could be wrong but the language seems to indicate a moratorium on "privately managed charters" and perhaps even just "for-profit" charter schools. With some exceptions (and remember, this changes depending on the state), all charter schools are privately managed, meaning their governing boards or entities are privately chosen or appointed. There is no public process. (In Virginia, charter schools are permitted but they must be approved by local school bards and then they are still subject to, although not as strictly as TPSs, their governance, they are still under democratic control--we have very sound charter school laws here.) So yes, saying no more "privately managed" charter schools is kind of like saying no more charter schools. But, the distinction between "for-profit" and "non-profit" charter schools is much blurrier. For one, they can still be financed the same. Second, non-profits can still have dealings that foster profiteering and that involve corrupting profit-motives. Third, the governance of non-profits charter schools can still be private and non-democratic, which is the real issue. The funding and profit-making generally come out of the governance. To learn more about this (or maybe see what I got wrong here :) I highly recommend you read Baker and Miron's (2015) report about this.

I have had another little charter school-themed post in the works for a while which I hope to have up sometime in the next week. Stay tuned . . .

Thursday, July 28, 2016

For what it's worth, my take on Tim Kaine

As her "contact in Virginia," Diane Ravitch asked me several days ago to write a piece on her blog about Tim Kaine and Anne Holton. I agreed.

I know he didn't give the greatest speech at the convention last night. He had too much makeup on. People on twitter made all kinds of youth-pastor-dad-jokes-substitute-teacher-who-tries-to-teach-you-something jokes about him. I laughed. (But then I felt guilty about it after.) Maybe it will be in the future, but a convention like that is not his best venue. But I think that's a good thing. It means he is not slick.  You don't walk away feeling fired up but without really knowing what you're fired up about. They certainly don't always get it right, but he and his wife are in public service and politics to actually help people and make our society a better place. What a novelty.

Anyway, I hope you will read what I wrote and I hope those skeptics out there will give him a chance:
I had never heard a politician speak so earnestly and so frankly. He told his story. What stood out the most was his emphasis on local politics. He didn’t seem to see local political work as grunt work you have to do to get to next level; he saw it as the most important type of work you can do, serving the public and serving your community. 
Please read the whole thing.

Wednesday, July 27, 2016

Of course a woman can be president

On this blog, I rarely stray from education-related topics. (I also rarely post these days, but hey, I'm trying to change that).

I get it that Hillary Clinton made history tonight by being the first woman to capture the nomination for president from one of the two major parties. I get that it will encourage other women to run. I get that it's a big deal. I have never been a Hillary hater or even disliker. In fact, I have always kind of liked her.

But I found that little video bit at the end of tonight's (Tuesday, July 26th) convention program to be, well, kind of patronizing. Because you know what? I don't need Hillary to tell my little girl (who, for the record, was not up by then) that she can be president for her to believe it. I teach her that just as my mom taught me that. It's as if we and our daughters and our mothers need to hear that to feel worthy of it. What my mother and so many other women have accomplished, was that not enough for us to know? It was certainly enough for me.

I suppose it's also hard to think of Hillary's nomination as such an achievement when it is so past due. Then again, I also come from, on both sides, a line of very strong, independent women, most of whom worked. Of course a woman could be president. Though, of course, knowing that is quite different from its actually happening.

Anyway, I am glad a woman, and a presidential one, too, finally got nominated.

Tuesday, June 28, 2016

It's all still the same reformy bubble

I followed the whole "social justice warrior" conversation that took place about a month ago from afar. I pretty much gave up being active in the national ed reform/ed reform skeptic community when I became more active locally. And then I kind of gave up social media activity and writing for pleasure--especially blogging, and tweeting--for my PhD program (as a compromise, I maintain a facebook page called "All Things Education"--you can "like" it without being my facebook friend). But, back to the topic at hand, I couldn't resist jumping into the fray on this one, even if the fray even if everyone else has already moved on.

I read (and cringed at) Robert's Robert Pondiscio's  piece, The Left's drive to push conservatives out of education reform piece the day it came out. When he was with the Core Knowledge Foundation, Robert was one of my favorite education thinkers and writers. We chatted online quite a bit and he even invited me to guest post at the Core Knowledge blog. We agreed especially about the importance of content knowledge, especially in literacy and the mis-guidedness of excessive teaching of "reading skills" at the expense of teaching literature, writing, the humanities, sciences, the arts, PE, math, foreign languages. I still read some of his work with great interest and will share it and recommend it accordingly. But this piece was really alienating and just utterly clueless.

Although there is much to rebut in the piece, I will only point out the problem with using the free market as a means to "take people out of poverty."
a) The free market, essentially where there's a profit motive, should never be used in the following service sectors (and no, I am not including the vending of clocks and what not to schools and to say that I am is just silly): education, military, healthcare, criminal justice, and social services.

b) The free market is dominated by the already powerful: white males and is designed to keep  (poor choice of words there) currently functions in a way that keeps those folks on top. The free market is not a neutral tool, it's not an equitable tool, and it's not a magical tool that you just sit back and let work its magic. It doesn't replace the hard, painstaking work that is obligated in democracy.

I am heartened that The New Schools Teachers Project Teach Plus for America Democrats for Education Reform 50CAN Stand for Children Students First Venture Project Fund are talking about and including voices from Black Lives Matters.

But when it comes down to it, I still fail to see much difference between conservative and centrist (really, centrist is a better term--most of these people aren't true liberals. If they were, they wouldn't be so cavalier about dismantling public democratic institutions) ed reformers. It's all still one bubble. I mean, I guess maybe vouchers separates them but not always. Okay, gay marriage, for sure. But that's about it.

So the centrist reformers are ready to listen and include more of those voices, but ultimately, that doesn't really matter. What matters is: power and on whose terms acts of education reform are committed.

In our society, people who aren't white males, and to a lesser extent white females, and an even lesser extent queer white folks, don't have power. Black people and Native Americans have the least power of all.

So the real questions for these ed reformers become:
1. What are you going to give up? What are you going to relinquish? (And I don't mean the reformy "edu-visionary" version of relinquish as in give up your power and kid to the free market and shut up kind of relinquishment. I mean actually giving up power because you have a unhealthily disproportionate amount of it.)

2. On whose terms are you going to enact and implement education reform policies? Will this be an actual democratic process? Or will you just seek the "buy in" of those folks and wear the titles of their movements as bumper stickers as you drive off and leave them in the dust.

Are you going to listen to actual teachers, parents, and students who don't agree with the policies you push?  To the black teachers who got fired en masse in New Orleans and DC? How about to the parents living in poverty who don't want their schools closed? To the students who don't want to go to schools that are run like prisons ? You gonna have those folks at your conferences? Running your organizations? Helping to make policies? You gonna refer those people to journalists instead of talking to them yourself? Are you going to power share? Are you going to step down from your positions of power? Or are you gonna wait until you're all done? Or will you stay in your bubble and just have representatives from powerless groups who are with The New Schools Teachers Project Teach Plus for America Democrats for Education Reform 50CAN Stand for Children Students First Venture Project Fund? Will you continue to buy journalism or will you let journalists do their jobs? Will you continue to buy educational research that supports your policies or will let the educational researchers do their work?

I know many of you are nice people who mean well, but when you actually become of a part of the communities you like to have so much say over and when you actually step down and aside so those folks can step in and have control over their own communities and schools, when you give up your power and embrace true democracy, then that will be true change.

Monday, February 1, 2016

Charter schools in Virginia whether we want them or not

Charter schools may soon be coming to Virginia communities whether those communities want them or not. This is not about whether or not to have charter schools or whether or not charter schools work. This is about power and democracy.

In Virginia, what's known as the "charter school bill," HB 3 in the Virginia House of Delegates and SB 588 in the Virginia Senate, establishes a resolution (HJ 1 and SJ R6) that will trigger a referendum on a constitutional amendment giving the Virginia State Board of Education ( a 9-member body appointed by the Governor) the power to go over the heads of local school boards and establish charter schools in local communities.  This resolution will likely be heard in the Virginia House of Delegates today (Monday, February 1st, 2016) or tomorrow, so you must contact your Delegate ASAP.

This resolution and accompanying legislation is before the General Assembly for the second year in a row. (I wrote about this last year here and before that I wrote about the concept, when it was the Opportunity Educational Institution, here.) Last year, it passed both chambers and, hence, if it passes this year—and as of this writing the House Privileges and Election Committee has sent it on to the House floor on a 10-9 vote—it will go onto the ballot this November. (Or maybe not this November if the Virginia GOP doesn't think it will pass then, but I digress.) 

“Work” is not the right way of looking at this, in any case. Like any model, some charter schools are successful and some aren’t. Some charter schools are true institutions of education, created by parents and educators, while some are real estate scams, developed by hucksters and charlatans. But given that all students are not served as they should be in public schools, I agree that conversations about the merits and disadvantages of charter schools are worth having.

But it is a conversation worth having among parents, citizens, educators, and educational leaders in the communities where charter schools are potentially to be located. Setting up schools in local communities is not a state matter. While many of its members are knowledgeable and passionate about K-12 education in Virginia and the Virginia State Board of Education may do a good job with the work they are tasked with, this is not their job.

Virginia currently has a rigorous, democratic process to establish charter schools, a process with built-in oversight, checks and balances, and accountability. Charter school proposals go before the locally, democratically elected (and in some cases, locally appointed) school boards where the charter schools are to be established. Charter schools in Virginia are overseen by these school boards and the schools are hence accountable to the public like all other public schools. Some local communities in Virginia have decided to set up charter schools. Groups in other communities have tried to set up charter schools but have not made a strong enough case to other members of their communities or to their school boards.

A school board failing to act on a matter or acting in a way that we citizens don’t agree with is not sufficient reason to put the Commonwealth through the referendum process, to amend the constitution, or to do so in a way that will disenfranchise citizens of local communities. Shall we threaten to amend the Constitution of Virginia every time our school boards do something we don’t agree with? Is that reasonable? Because I have got, like, ten amendments (don't tell my school board I said that). No, that’s not reasonable and it trivializes the amendment process.

I don’t always agree with my school board, but it is my school board, answerable to me and the members of the public it serves. Let's keep it that way. Please contact your state Senator and Delegate and tell them to VOTE NO on HB3/SB588.