Show-Me Institute Podcast

Real-Time Crime Data with Jeff Asher

Susan Pendergrass speaks with Jeff Asher, data analyst and co-founder of AH Datalytics, about the Real-Time Crime Index (RTCI).

They discuss how the RTCI provides a near real-time look at crime trends across the U.S. by sampling data from hundreds of law enforcement agencies. Jeff explains the challenges of working with incomplete and imprecise data, the methodology for standardizing crime statistics across different agencies, the importance of real-time data for informed decision-making, and more.

Produced by Show-Me Opportunity

Broadcast on:: 25 Sep 2024
Audio Format:: other

(upbeat music) - Thank you so much for joining us on the podcast today. Jeff, Asher, so glad to have you. I mostly wanna talk about all of the crime data you've been collecting and analyzing 'cause I think it's fascinating, but I wanna say that right up front, we're in the Show Me Institute Space in St. Louis, Missouri. And like what I've read about in your background in Louisiana, we have this, the Show Me Institute, we think a lot about public policy and all different sorts of areas. And what do we do to make Missouri a state where families wanna come and raise their kids and businesses wanna come and start and grow? And we can talk about that all day long, but when we don't get on top of the serious crime problem that we have in our major city, St. Louis in Kansas City, no matter how good everything else is, we're not going to be in a attractive place to live. And I think that there's arguments over whether crime is going up or crime is going down. It seemed to go up a lot during the pandemic, but now folks are saying it's coming down. And we've been looking into it a little bit ourselves, but there are so many problems with crime data. And I wanna hear about like what you have done to overcome that 'cause I feel like you've put monumental effort into sort of addressing at least the availability of having good real-time statistics on crime. - Yeah, you think about any problem that we face in the country, you know, inflation, you know, the economy, employment, anything that we wanna address both locally and nationally, you gather data on it and you report it and you report it in a way that makes it possible to really dig deep and evaluate it. Crime, we don't do that. And the FBI, it's, look at my calendar. I know that it's September 25th now or September, week of September 23rd, which is the week that the FBI released their 2023 crime data. It's good to have more accurate, more complete data. There's also value in knowing fast data. And so when I give these presentations on crime data, I'd say, you know, all right, there's an open book quiz. Who won the World Series last year? You know, how many wins do the same those Cardinals have this year? And you could pull out your phone and see, oh, the Rangers won the World Series last year. I confess my ignorance, I don't know how many wins. - Make it the number. - You know, I can grab my phone right here and you get that out within a few seconds. All right, how many murders were there in the United States last year? And up until September 23rd, there was no even remote estimate of that number publicly available. And so there's one thing, you know, we look at sports, I'm a sports nut. We don't do a good job of evaluating crime trends, even as anywhere near as close to what we do with any sports. Even the least followed sports, you know, I don't want to get down like the Olympic badminton or something like that. But I think the analytics and the Olympic badminton are probably stronger than they're with crime, which I'll go out of limb and say crime is more important than the Olympic badminton. So we're in a situation where we don't collect the data. And so what we've done is we have a grant from Arnold Ventures. We built the real time crime index where we're gathering monthly crime data. Right now we have a right around 300 agencies of data. We had 305 agencies in for our June report. We had 295 for our July report. And we're publishing on a month and a half delay. It's not, it's about 45% of the murders that occur each year. It's about 35 to 40% of the violent crime. So it's not everything. But it's enough that you can evaluate trends and understand what's happening nationally in a way that still leaves room for what the FBI is doing as an important means of evaluating more recent trends. - Why do you only have 35 to 40%? It's the coverage of like the cities that you collect the data from or? - Yeah, right. So right now we have about a quarter of the US population where targeting cities of 50,000 or more, because if you get every one of those cities, there's like 850-ish of them. If you get every one of them, then you will, your average miss on in terms of like your predictive value relative to the national change is right around one percentage point where we are right now with 300 agencies. We're probably looking at two to three percent. If you get 1,500 or 2,000 agencies, you're looking at an average miss of like 3/4 of 1%. So going above this population threshold that we've set doesn't add a ton of value. If we can get 50 to 60% of the murders each year from 500 agencies, we'll be going, we'll be rolling around and you could know what the trends are. If things level off or they start to go up or they continue to deployment as we've seen, you'll be able to tell that in without having to wait another nine or 10 months for the FBI data to come out. - So I want to be perfectly clear to anyone listening. They can go to aahdatalytics.com and see all of your crime dashboards. And this is where you upload all of your data and you've built into dashboards, you've made it super user friendly and I'm looking at the year-to-date murder comparison and for folks listening in the St. Louis or Kansas City metropolitan areas, those are two of the cities that you cover. So it looks to me based on the URL that you link to, do you take from the city of St. Louis their weekly crime PDFs and then do you have to take that and key it in or what do you do in that situation? - So we're actually talking about two different products. So the real time crime index, realtimecrimeindex.com has every agency and it's standardized to about a month and a half delay. So we have data through July in mid-October we'll publish the August data. So it's a bit more delayed. For that, we're getting the data straight from the state of Missouri, which is published, so St. Louis, Kansas City, most agencies in St. Louis and Missouri are sending it up to the state. The state is publishing it with minimal delay and we just grab it from there and put it into our data set. For the year-to-date murder dash forward, we're taking more up-to-date data. There are places like Washington DC, Memphis, New Orleans that publish daily. St. Louis publishes weekly numbers. So we just weekly grab it. That one is probably gonna go away at the end of the year just 'cause it's much more of a manual lift to check every week. And the numbers aren't all that different from the realtime crime index. So both of them are showing a 17 or 18% decline this year and they showed similarly large declines last year. The realtime crime index-- - From the pandemic. - From 20, no, from last year. So we have an 18% decline this year from 2023 on top of what was a 12% decline roughly last year from 2022. So we're seeing, we saw a 30% plus increase in 2020 and we're seeing in as it develops this trend, sort of erase itself and go back to or hopefully exceed where we were in 2019. - Right, so overall, yeah, when I'm looking at murders on the realtimecrimeindex.com website, I'm looking at St. Louis murders and you have them through July of this year. It looks like, yeah, you see a big bump during the pandemic, but overall they are sort of continuing on a downward trend and is that what we're seeing nationally then? - Yes, in general, we're seeing a pretty steady net downward trend and it matches, our data matches what the FBI is producing on obviously a long delay. It matches what the gun violence archive is showing in terms of fatal and non-fatal shootings and it matches what the CDC is showing in terms of homicides. So we feel good that we're able to say with pretty strong confidence what the trend is and that it will match up to what the FBI says. - So what about, I just saw an article in the Wall Street Journal, I think it was on the National Crime Victim Survey and it says, you might think that crime is going down, but according to the National Crime Victimization Survey, it's actually going up. Did you read that and what do you think about that? - So I'll be out, I'm shortly gonna be publishing a newsletter that is talking about NCBS in a little bit more detail. So the problem is that like a poll, it's a survey. So there's a margin of error. When the New York Times came out recently with their polls of Arizona and Georgia and I can't remember, the North Carolina I think was the third state. In the show, Donald Trump is beating Kamala Harris by 5% points in Arizona right now. You can't take that as Donald Trump is going to win by 5% points, one, it's a snapshot in time. And more importantly, there's a 4.4% margin of error on each candidate. So when you actually are reading the single poll, which is coming from a survey that they're extrapolating into the larger audience, you have to incorporate the margin of error. And so what the poll is really saying is that Donald Trump could be winning by 14%, or he could be losing by 9%. And that's what we're gonna say with 95% confidence, which means that one in 20 times, the margin will be outside of that. So the poll as a survey is not giving you exactitude. And so using National Crime Victimization Survey, where they're surveying about 220,000 people over the course of a year, they're not asking them about murder, because obviously, harder victims can't take surveys. And when people apply the baseline numbers without considering the confidence intervals, it creates this level of certainty that is nowhere implied by the NCBS methodology. So yes, violent crime could be up 37%, that was the baseline finding. But the upper and the lower bounds of 2023 and 2020 show that violent crime could be barely up, or it could be down by a substantial amount. A lot of the individual crimes that are being referenced, rape and robbery and urban violent crime, they could be up. But if you account for the entirety of the margin of error, they could also be down. There's a lot of uncertainty there that is not being captured. And so I think that relying on NCBS, which again, does not take into account murder, which is the thing that changed the most. And then applying a level of certainty that is not warranted based on the methodology is doing everyone a disservice. And it's not ideal to see someone that was the, you know, that's-- - Writing about this. - You should know better. We should, people that are talking about crime trends and writing about crime trends should address the uncertainty. If you're talking about FBI data, I acknowledge that, you know, we know not every crime is reported. We know that not every agency reports every year, that these are estimates and that we aren't, there's not 100% confidence in these numbers. When you're talking about the natural crime victimization survey, a similar or more, you know, it's a, of uncertainty should be brought to bear. - Yeah, I get this all the time 'cause I do education policy and people like to look at nature also surveys of students in each state and you have to be very careful in the interpretation of those, but how far back does your data go? - So we have FBI data back to 1960 for some measures. The real-time crime index right now, we have every agency back to 2017 by month. There's certain places like New Orleans, I have murdered data back to the 19 teens. - What are you seeing? Like, if you go back, I saw, I'm not on that part of your website right now, which I just wanna say upfront, I love your website. It's so fascinating. I could spend many hours. What, I believe what I saw in the trend line is things like murders peaked in the 70s, right? And then there's been some peaks, but overall in the last 50 years, crime is down, would you agree with that? - So murder peaked in, they've had two peaks, sort of 1980 and 1990. And it's sort of dipped in between and then rose up and in the early 1990s was the highest. Murder has been falling steadily since 1993, 1994. It kind of hit a leveling off around 22,000-ish and it fell substantially in the late 90s, early 2000s, and has been at generally a substantially lower level than the 90s, slightly above where it was in the 60s and that early 70s, but in that range. Violet crime has had more of a big increase, according to the reported violent crime from the FBI, big increase through the 90s, and then it fell, and then over the last, I'd say 15 years, has not seen a ton of change. So 2014, the violent crime rate that the FBI reported this year, the violent crime rate was 363.6 per 100,000, which is the lowest since 1970. In 2023, the FBI estimated that the violent crime rate was 363.8, so that's 10 years a slight increase in 2020, and then a slight decrease, but even the increase in 2020 wasn't that big. It's murder that rose 30% in 2020, murder that has fallen the last two years at historically high rates. So we've seen major changes in murder, not so much other violent crimes. - What about autotheft? I feel like that also is up. - Autotheft has surged four straight years largely on the back. So we've had sort of the post-COVID, I don't know what's causing it, surging autothefts, and then in the summer of 2022, someone posted on TikTok how to steal Kia's and Hyundai's excellent to huge surges. You could look at places like St. Louis. - It's incredible, it's an incredible increase. - And then you see the increase, and then you start to see the decline. It was a finite increase in 2022, 2023. And so I don't know if people ran out of Kia's and Hyundai's a steal, or if people got smarter about what they were driving, or they bent their scars, either way, you had this huge increase, 12% increase in 2023. And the data that we have in 2024 points to a declining violent crime or a motor vehicle theft rate, everything else is seen declines. And what's interesting about motor vehicle thefts is we're looking at clearance rates, and clearance rates for every other type of crime increased in 2023 still reasonably low by historical standards. - The national motor vehicle theft clearance rate was 8.2%, which is the lowest percentage clearance rate that's ever been recorded. - I thought the clearance rate was very interesting. I'm gonna get to that next, but I will say like St. Louis, about 3000 cars were stolen a year for the whole time, and then somewhere in the spring of 2022 becomes 9000. - Yeah. - It's back to five. So I was like a triple for some reason. - It's, I mean, you look at DC, Philadelphia, New Orleans. It's a big city thing, but it's certainly not unique to St. Louis. - So when you're gathering this and you're building these school dashboards and you're looking at it, I assume that you're thinking maybe, maybe you're not, but what can be done about these numbers? Or what kind of policy could either is happening that's bringing it down? Or what does this imply for policy when you look at these trends, national trends in the various types of crime, both back decades and back for the last 10 or 15 years. Like what emerges in terms of, have you seen cities where it comes down faster and therefore they must be doing something right and cities where it's not coming down and they must be getting something wrong? Or what do you think this implies? - I think that's a really good question. We try to stay out of the policy debate. - Nice. - It's helpful as data analysts. I think for us, it also keeps us above the fray hopefully a little bit and my goal is to, no matter what a person's political persuasions to put data that's trusted and is honest and that where there are errors, there are honest errors and we're gonna fix them as quickly as we can. So from a policy perspective, I try and stay out of that. When I do writing though, I can certainly identify the outliers and certainly do that. I think from a, it's hard with crime data. If you were to talk last year, if we were doing this podcast last year, I would have pointed to DC and Memphis and Dallas as three cities. Like these cities have seen big increases in murder and everywhere else has seen a big decrease. And if you were to do it this year, all three of those cities have seen big decreases. So sometimes there are outliers that are just outliers and because we have such an imperfect understanding of what's driving these trends, we have an even more imperfect understanding of what can be done about it or why a certain city is an outlier when everyone else has seen the decline, they're seen an increase. - Yeah, we did a podcast recently with Charles Fain-Lehmann who actually pointed me to your take that on DC specifically, which was very interesting. And he's like, you know, you do the data work and he uses it to do the policy work, which I think is also a very interesting thing to do. But I will say that I really liked your clearance rate data that you have on one of your dashboards. That's at AH DataLitix. And also you have data on the number of police officers, both officers and civilians. Where do you get those data? - That's all coming from the FBI. - Okay. - They publish what's called the master files each year. And so that will give you all of the data monthly for every agency that each agency submitted. And so we can pull that together and build more detailed evaluations. The crime data explorer, if you've been to there, it's very difficult to use the-- - And BJS at the Bureau of Justice Assistant. - No, at the FBI. - Oh, at the FBI, okay. - The BJS actually has a great dashboard called end dash, which provides if you want any of the raw numbers for NCBS, it's really easy to get to, really easy to use. The FBI data is significantly harder to use. - But you're providing this middle approach, this middleman approach, where you're taking the hard to use FBI data and making it easy to use, essentially. - Yeah, well, and the idea being that the FBI is trying to get data from 19,000 agencies. And to understand the trends, we don't need 19,000 agencies. We can do it with 300 or 500 agencies. And so we're trying to get that data as best as we can. We're hopeful within the real time crime index to start moving into other topic areas, such as clearance rates, such as possibly shooting victimization, carjackings, staffing. So I don't know exactly, well, probably it's a three-year grant, so we're hopeful to at least do a couple more. - Something about prosecutions or anything related to the criminal justice system that way. - That data is so much harder to get at. - Is it? - I've worked with prosecutors, it's so hit or miss. It's so even the prosecutors that are doing a good job of providing the data, it's oftentimes confusing and illogical and poorly put together and, you know, takes mountains of effort to clean. So my preference is generally to stay away from a big picture standpoint in analyzing that kind of data. - Then there's always the unreported crime. - Yeah. - I mean, what are you gonna do about that? - I mean, the general idea is that most, the vast majority of murders are reported. So we feel really good about the murder number, we feel pretty good about the motor vehicle theft number because we know, you know, people need insurance claims and to get insurance claim and need a police report. So usually 3/4 to 80% of those are reported each year. The other crimes, a lot of them are unreported. And if you've read my sub-stack, like a lot of what I write about is, here's the trend of reported crime, we don't know, it may or may not be right. I think that when you get enough agencies together, you can kind of see what the trends tend to be. And the real value of the National Crime Victimization Survey, I find at least is not so much in giving us the numbers 'cause I'm a little more skeptical that a survey is doing a better job of capturing what those numbers are. But I think it gives us that table and that understanding of what share of people are and aren't reporting what crime types. - Right. - And it's invaluable because if I wanna talk about crime, you have to deal with the fact that you know that not every crime is being reported. And you can look at, okay, you know, there was 72% last year down from 80%. Again, there's a margin of error there. If you do the margin of error is overlap because there's a big mark of the error. So it's not inherently a decline, but you compare that to other property crimes where you're getting 25 to 30% of those getting reported. So it's, I think there's a lot of value there from the survey and understanding the dynamics of who does it, doesn't report. I think that there's less value in the survey in saying that there were actually X number of crimes and it's Y percent. - Well, right. But reporting also means that a report got created, right? 'Cause I know that when I first moved to St. Louis, I had a smash and grab in my car and called the police and they're like, "We don't take reports on this." You know, it's like, "We don't do reports on those doors "and we come out and welcome to St. Louis." And when I did, I was called to jury duty and they, you know, going through the whole jury pool in the court room, they're like, "Anyone, "has anyone here been the victim of a property crime "and like everybody breaks their hands?" Like that's not gonna be a qualifying question because everybody has and it's, you know, it's sometimes you don't report, sometimes you report and you don't get a police report. But I think that that's a whole different set of issues and then we get into the prosecution and that's a whole nother set of policy implications. But place to start is what you've done, which is at least collect and make it easy to access up to date accurate data to the extent possible. And I just think it's fascinating that this is coming out of essentially the private sector and the nonprofit sector versus the public sector. Maybe you'll be putting pressure on them to do a better job. - Well, our goal is to transition it to the public sector and to have someone take over it because-- - Yeah, FBI. - Either BJS or FBI, I mean, I don't wanna pressure them, but like the objective, we see it as a public good, it's why it's, we're grant funded, there's no subscription, the agencies don't have to pay or don't get paid to participate. The only money we spend is for FOIA fees, for a handful of agencies. - In your time? - Yeah, and the Arnold Ventures is paying for my time and for our staff's time. So the goal is to produce a product that is a public good and hopeful that it's useful to the public. - So well, and then you can download the data, right, off your website, which is also great. I mean, that right there makes it something that, no, hopefully, in addition to maybe the public sector taking it over, researchers will take it up and start to try to figure out what to do about it. 'Cause like I said, it's a real problem for people every single day and it's just like, well, we don't really know what's happening there, so. - And I'll throw out that the other value is that we, the sourcing is easily, easy to find, hopefully. We have a sourcing table and then at the bottom of each graph for each city, you can click on the sourcing and it takes you exactly where we got the data, which hopefully makes it a lot easier. We, you may want certain thing in the data that we aren't showing because we're aggregating big statistics, but this helps to identify, you know, hundreds of agencies where publicly available Prime Day days. - Yeah, which is fascinating. That's how I got to that weekly by neighborhood, St. Louis Prime Report and I started looking through that and going down, right? Well, unfortunately, a PDF, but very interesting. Well, I just want to be clear again, it's the realtimecrimeindex.com. - Yes. - And AH DataLitix are where your various dashboards are on murder rates, year to date, the uniform crime report. Where's that come from? Does that be JS? - That's FBI. - FBI and staffing and everything else I mentioned and I just would really encourage folks to go and check it out and I appreciate that you did it. I think it's fascinating. - Thank you. - And that you explained it all to us. I do appreciate it. - We like to hear where it's useful and it's an iterative product that will be improving. So any feedback is welcome and I can't guarantee that we'll address every piece of feedback, but we'll certainly read it and listen to it and try to improve the product as best we can. - That's awesome. Yeah, no, it's great. So thanks for coming and joining us. - My pleasure. (upbeat music) (upbeat music) (upbeat music) (upbeat music)