Open Data and Policing Reform

It’s not often you get to testify to one of your most admired leaders, and to do so side by side with another amazing community leader you’ve respected and appreciated from afar, but today I get to do both of these things and I’m pretty much in awe of how much of a privilege this is. It’s harder and harder to leave my girls at home and travel but for someone deep in the work of making cities better and more equitable at a local level, there are few chances to possibly influence change at a national level. And so I’m in Cincinnati where it’s below freezing, away from sunny California where yesterday I was drinking a good beer on the balcony with my wife complaining it was too warm. In January.

Today I’m testifying to the President’s Task Force on 21st Century Policing, which includes Bryan Stevenson of the Equal Justice Initiative, a man who bought me to tears last time I heard him speak. I don’t go in for hero worship, but Bryan would be on the short list.

What does a data geek and open government advocate have to say about the future of policing reforms in our country? My testimony is below…

Dear members of the task force and other community leaders,

I’m speaking today on behalf of Urban Strategies Council, a 28 year old social justice organization in Oakland, California where I have had the privilege of being the director of research and technology for almost eight and a half years. I am also here on behalf of OpenOakland, a civic innovation organization that I co-founded with Eddie Tejeda.

My work with the council has provided me with an opportunity to see how a lack of transparency in local government affects data-driven decision making. government technology, and community engagement. I’ve had the chance to work with many local agencies and community based organizations to help them unearth their valuable data, to analyse it and put it in context and then to help communicate the story and results of those data.

Traditionally the role of government has been perceived as a collector of data for compliance and reporting purposes, yet this is no longer sufficient in our view of 21st Century government. Government now needs to be pro-active across all agencies, especially those traditionally very closed and inaccessible. For many years we have been unearthing public data for research purposes and publishing these data openly for all to access- from data on local probationer populations, to crime reports and foreclosure filings. When we obtained both open and private data and published a report on the investor acquisitions of foreclosed homes, our work led to the creation of new laws to protect tenants and monitor housing purchases. When public data is put in the hands of communities, powerful things can happen.

We led an effort to crowd source the legislation to make open data the law in Oakland and now we have local agencies actively making data available to the public free of charge or restriction. This has led to breakthrough innovations such as which when shown to our city council led to disbelief- never before had decision makers seen their own budget in such clear context and the impact was powerful. Residents of our city were able to understand a complicated 16,000 line budget for the very first time- something made possible by opening data and by engaging the community in a respectful collaboration. Hackers, city staff and advocates working together.

Another local example of what happens when government opens up valuable data and collaborates includes our earthquake safety app (http:/ that helps inform low income renters if they are living in a building susceptible to collapse in the next big quake. This app was built by the community as open source and is now being deployed in a nearby city.

You’ll notice I’ve not talked about great policing collaboration examples. For good reason. Despite generating a near real time flow of crime reports, our local police departments and sheriffs have not been eager to jump into the world of open data, yet. Given the lack of trust in the Oakland Police Department, the need for real community policing and a dearth of accessible information about policing practices and incidents, Oakland is like most other US law enforcement agencies in its need to embrace open data, to develop respectful collaborations and engagement that leads to innovation.

Given the way communities of color are impacted by crime and violence, and the number of officer- involved shootings and assaults on officers, there is a very real and urgent opportunity for data to be leveraged for their benefit. Right now there are activist groups building databases of all officer- involved incidents and homicides;, these are duplicated efforts costing hundreds of hours of community time from projects such as Oakland’s Shine in Peace to and These projects should be taken as a leading indicator of a huge and growing demand for better transparency in our law enforcement agencies – citizens are clamouring for data to inform decision making, policy reform and civic action. When communities across the country need to collect news reports of officer- involved shootings and homicides, we’re missing something. When stop and frisk data are hidden from public view and not available for community research and analysis, we’re missing something. When arrest information only sees sunlight in the form of aggregate yearly reports, we’re missing something. That something will be realized when local law enforcement adopts a policy of open by default and begins publishing (with some obvious legal limitations) record level data of all crime reports, arrests, uses of force and weapon discharges along with stop and frisk incident data.

As individuals we do not trust that which we cannot see. Publishing data alone will not lead to better insights and operations, it is not a silver bullet to restoring community trust in police departments. However, in publishing these data on an ongoing basis, we make possible new, productive collaborations, new opportunities to engage with somewhat objective truths to work from and we allow for innovations that we could not predict. My recommendation to this task force: make open by default the new norm for our police forces, support the open publishing of these data, encourage standard data formats and support these agencies taking a leading role to learn together and to work towards common goals. Toward a future where transparency is no longer a laughable concept when it comes to law enforcement, where communities trust the information coming out of police databases and where residents can see and understand patterns and problems for themselves. Then we can have informed debates and start to remake policing in the USA in ways we agree on.

Truants vs Absence

Sometimes good public policy just comes down to the money.

The Attorney General Kamala Harris has published a nicely designed report to raise the profile of chronic absenteeism (kids missing more than 10% of school per year) and truancy (being late- hmm, not much of a comparison).  It does a good job of laying out the complex realities that are allowing this problem to continue and to grow in some parts of the state and in many communities.  The underlying root causes are complex, but the system components are as simple (sorta) as poor data management systems, lack of awareness and lack of effective interventions.  While many people will not empathize with the impact that students really do suffer long term from chronic absence and the fact that is does predict graduation incredibly well at 3rd grade levels, others need to just see the dollar figure.  The AG’s estimates put the cost to our state as a result of dropouts as $46 Billion a year. Given the well established links between chronic absence and graduation (I helped build that case with some work in 2007 with Hedy Chang), we can’t pretend that this is something not worth investing in- the cost benefits are enormous.

One of the more incredible findings in the AG’s work was this:

Student record systems need repair and upgrade to accurately measure, monitor and respond to truancy.

California is one of only four states in the country that does not collect individualized student attendance records at the state level.10 Even at the local level, only half of the school districts that responded to our California School District Leadership Survey (“District Leadership Survey”)11 were able to confirm that they track student absence records longitudinally—that is, they track individual students’ attendance year after year. The failure to collect, report and monitor real-time information about student attendance renders our most at-risk children – including English learners, foster children and low-income free- and reduced-price lunch students – invisible.

There are few people who would expect that our modern society is incapable of tracking student attendance in a meaningful way, yet many of California’s school districts are struggling to do just this.  And the lowdown is that lack of data means lack of scalable understanding- a single teacher may have an awareness of a student’s absenteeism, but when a struggling kid moves up a grade the knowledge of this problem is lost, and the administrators sure aren’t aware that the same child is on the same bad trajectory year after year.  We often see fancy video screen laden operations centers on TV, but the reality described in this report is that many districts aren’t doing basic analysis of chronic absenteeism and as such are surely not working on and aware of successful interventions to assist these students.

Give it a read, it’s well written and illustrates a real crisis in our cities. My 2c is that we largely ignore the truancy data presented- it’s a red herring. Follow the money and you’ll see the outcomes of chronic absenteeism as a huge money pit – unless we really address this across our state equitably.