Downtown Yonkers, New York. Photo by Flickr user samsebeskazal.
Public education in Mississippi has been and will continue to be a hot topic for discussion in our state. Clearly, improving the college and career readiness of Mississippi students is a critical factor for improving our state’s economy. But how exactly do we get there?
In 2014, Bud Kroll, a retired Wall Street veteran and volunteer for Yonkers Partners in Education, released a paper which examined student outcomes at New York public schools and districts in such an elegant and logical way that I believe it deserves attention here in Mississippi, too. His paper, College and Career Readiness in the New York State Public Schools, used a statistical analysis technique called regression analysis to measure how schools and districts perform, while controlling for the impact of poverty on student outcomes.
Before turning his attention to public education, Bud spent 27 years in the financial services industry, focusing on the sales and trading of precious metals, energy, fixed income and equity derivatives. He was a member of the New York Mercantile Exchange and the American Stock Exchange and currently serves on the Board of Directors of Deutsche Bank Trust Corporation. He held several positions at JP Morgan Asset Management from 1996 to 2006, including Head of U.S. Equity Quantitative Research and Global Head of Structured Equity.
Tell me a little about Yonkers and how you got involved with Yonkers Partners in Education.
KROLL: Yonkers is right on the border of the Bronx. It’s the first suburb that you come to outside of New York City when you drive north. Yonkers, itself, has a long, checkered history of bankruptcies and poor educational outcomes. So, while Yonkers sits inside a very wealthy county, Westchester County, it is indistinguishable from the Bronx, which is one of the poorest boroughs of New York City. So Yonkers is a very large, complex, high-poverty, high-immigration melting pot. It’s also the fourth-largest school district in the state.
Yonkers Partners in Education is about six years old, and it was originally founded because the mayor and the head of the board of education came to some local civic folks wanting to put together a private/public partnership to help our schools. Our mission is to increase what we call post-secondary success. It’s our mantra that, in order to be a successful adult, you have to have some sort of post-secondary education to get a decent-paying job. That could be a two-year degree, it could be a four-year, it could be a certification¬†‚Ä쬆but just graduating from high school doesn’t cut it.
I spent years as a volunteer teaching assistant in math classrooms. My background is in quantitative things, and I viewed firsthand for many years, both in the city, as well as in Mt. Vernon and Yonkers, the challenges that are faced by inner city schools and, in my specific case, in math classrooms. And that’s what led me to get involved in Yonkers Partners in Education.
Why did you decide to approach your paper the way that you did?
KROLL: I’ve got 25-plus years in financial services ‚Ä쬆a lot of it investment management ‚Ä쬆and I’ve been involved particularly in the types of investment management that rely on quantitative measures. I’m not a highly trained quant, but I’ve had the great pleasure of working with a lot of highly trained quants, and I’ve absorbed by osmosis. But what really led me to this paper was the idea that, having spent my entire career in the finance world, why wouldn’t I apply the same kind of thinking and discipline to the education world?
In my line of business, there are lots of different kinds of problems that you can solve. One approach to problem solving is something we call “right to left.‚Äù You start with what you want your outcome to be, and then you move to the left to figure out what you need to do in order to get that outcome.
For example, when you look at a mutual fund, you need to be able to figure out whether the manager of that mutual fund is doing a good job. But the problem with that is you don’t know what to compare that manager to. In the finance world, about 25 years ago, a technology emerged which allowed you to look at something called common factors: How large are the companies in your portfolio? How much are they related to finance versus insurance versus healthcare? How many of them are U.S. versus international? Those are all called common factor risks. If you can describe the portfolio to the nth degree and reduce all of the components of its common factors, you’re left with something which is specific. That’s the skill of the manager.
So you start with something called a multiple regression. You take a whole bunch of factors, and you have no idea whether or not they are going to end up affecting your outcome. In our case, the desired outcome is college and career readiness as measured by the New York State Aspirational Performance Measure or APM. Then you throw these potential explanatory variables into this sort of ‚Äúwashing machine,‚Äù right? The more observations you have ‚Äì in this case it’s data from school districts and school buildings ‚Äì the more robust that analysis is going to be.
We’re feeble as human beings in that we have a very difficult time imagining things in more than two dimensions. Which is why I reduced it from what’s called a multiple regression to a simple regression. In the worlds of science and finance, you don’t generally seek to find a single explanatory variable because usually the world isn’t that simple, right? Usually, it’s a bunch of stuff that, together, are highly predictive. It just so happened in this case, and lucky for me and my readers, that one variable was so much more predictive than the others that I was on solid ground to jump from a multiple regression to a simple regression. It allows us to look at things in just two dimensions ‚Äì x and y, instead of x, y, z, a, b, c.
You found poverty to be that strong predictor, right?
Yes. Poverty leads to low college readiness ‚Äì that’s not a stunning conclusion. It’s been shown by smarter people than me, much earlier and many, many, many times. Poverty is insidiously bad. That’s not what this paper is trying to say. What this paper is saying is this: If you now confirm that poverty is bad, and you have a quantitative measure that you can then apply to school buildings and districts, you can look at how well certain districts or buildings are doing net of this bad thing.
For someone who’s a golfer, you can think of it this way: How do you go out on a Sunday morning and play against someone who’s a much better golfer and figure out which one had a better day? The answer is your handicap. So if I go out there, and I’m a 21 handicap and the other guy’s a scratch golfer, as long as I do better versus my handicap, I had a better day. That’s what this whole paper is about. Simply coming up with a measure that can calculate net scores.
So, with that as context for the paper, there are a dozen or so ways that this type of framework can be used to identify people who are hitting above their handicap or hitting below their handicap, and we can learn lots of things from both the ones above and the ones below.
We have eight high schools in Yonkers. Now I can shine a bright light on a district, and I can go from high school to high school and say ‚ÄúYou, principal #3, are doing a lousy job with the raw material that you’re getting.” I don’t care if your students have higher poverty levels or lower poverty levels, but net strokes, you’re having a bad day on the golf course.
You plotted college readiness against the percentage of students who qualify for the National School Lunch Program (free and reduced lunch). Can you explain what that chart shows?
 
First, you see that, increasingly, as poverty gets ‚Äúless bad,‚Äù your outcomes get better. That’s what correlation is. That relationship, the red line, can be represented as an equation. What does the .65 mean? Well, it means that, when you look all the way to the left at the richest districts, the ones with no poverty, your predictive line hits that Y axis between 60 and 70 percent. The .65 tells you is that, if you have no poverty, you would expect 65 percent of your students to be college ready. And the -0.64 in the equation tells you that the poorer you are, the worse you do. That red line is downward sloping ‚Ä쬆it’s a negative slope. Finally, the R2 (0.63) tells you what percentage of the variance can be explained by that factor.
How tightly clustered are the dots around my line? An R2 of “1” will tell you that every blue dot lies directly on that predictive line. In R2 of “0” would look as if someone just fired a shotgun from 100 yards and you’d have random blue dots, which is what you have with some of the other variables we looked at.
It was interesting that you found that the R2 was higher in districts with low percentages of free and reduced lunch students and lower in the higher poverty districts.
Take a look at the dashed black line that runs north/south. Now hold your hand over everything to the right of that line. The relationship between poverty and readiness to the left of that line is pretty tight. If you now cover everything to the left of that line and just look at the right side, what you see is that it’s not nearly as tight.
We often think about poverty being used to explain the poorest districts and the outcomes therein. If a district is really poor, they do really badly. But what’s so interesting about this finding is that poverty or lack of poverty as explained by this particular variable is more highly predictive in those not-so-poor districts as it is to the right in the poor districts.
So how rich you are or how poor you are is, of course, highly important. But in these super-poor districts, you have a lot of really big outliers. This gives me great hope because it tells me that their outcomes are less imprisoned by poverty level than students in low poverty districts. And my goal in life is to reproduce outcomes of those blue dots that are above the red line. What we find in New York City is that you get these outlier dots in communities that are poor but value education very highly.
So once you control for poverty, you can figure out what those outlier schools and districts are doing right or wrong?
Absolutely. If you just put your statistician’s hat on, you might say, “okay, if I’m only looking at the population to the right of 60% (free and reduced lunch), one of two things is going on: either there’s another systematic predictor variable that I’m missing, or maybe it’s a great candidate for a multiple regression ‚Ä쬆poverty plus something else. Or maybe it’s just the ‚Äúspecial sauce‚Äù of a specific district ‚Ä쬆not a common factor, but some specific factors that just stir the pot ‚Äì a great principal, a very involved group of parents ‚Äì you know, specific things. But that would require more statistical work. You can do this stuff until your ears bleed.
So what’s the next step? Have you identified any of those special factors?
I have not done that work. I kind of put this paper in a drawer because once I had my vocabulary ‚Äì I needed to move on to services that we’re actually providing that I have some control over.
My goal in life is not to be a theoretical education researcher. I’m a volunteer, right? Our main focus is in the schools. We have a college adviser in six and a half Yonkers high schools. I collect data when a student comes in and meets with a college adviser, we check them in electronically using a cloud system. And then we can relate the services that we’re providing and how many times we provide a service to a student and what kinds of services those are, whether its SAT Prep or other things to their outcomes. Do they apply to college? Do they enroll in college? Do they persist in college? So once I had my vocabulary, I needed to move on to services that we’re actually providing that I have some control over. But that might be a very interesting area for someone to look into.
