Data
This project bears resemblance to some other open-source projects on police data, such as OpenPoliceData, Police Data Initiative, and the Stanford Open Policing Project, however, this project has been an independent effort that focuses on
crime incidents (rather than policing efforts such as stops or use of force),
harmonizing crime categories across agencies and time,
mapping incidents to Census geographic units.
The latter two objectives make the cleaned data particularly useful for research in the social sciences. The project is currently semi-closed-source, but with plans of going open-source subject to the research progress.
Location of cities included for analysis
Survey of cities
More than 350 cities (or “incorporated places”) in the US have been surveyed to acquire crime data from their data portals. An open data portal is typically set up and run by the city (or sometimes county) to publish data of public interest. The local police departments sometimes have their own data portals, or they publish data through their respective cities’ portals. The 350 cities represent all US cities with somewhere above 100,000 population in 2023 (the cutoff is not exact due to there being multiple population estimates). The cutoff of 100,000 is by all means not convenient and is used on purpose to leverage the large scale, publicly available data that has been overlooked by researchers. Researchers have a tendency to stop data collection as soon as they have enough statistical power, and the author of this project is dismissive of such practice. Below presents the survey result. Population estimates are taken from the 5-year American Community Survey. Start Date
is the approximate time when the data starts to have substantial coverage.
Ready-to-use data
After cleaning and linking to Census variables, the product looks like the following, with New York City (NYC) in 2022 for demonstration. NYC is made up of 5 boroughs, each also being a county (Bronx - Bronx County, Brooklyn - Kings County, Manhattan - New York County, Queens - Queens County, and Staten Island - Richmond County), thus the city conveniently becomes a “nested” geographic unit.
IncidentYear | GEOID | City | Aggravated Assault | All Other Offenses | Arson | Burglary | Counterfeiting-Forgery | Criminal Homicide | Curfew-Loitering-Vagrancy | Disorderly Conduct | Driving Under the Influence | Drug-Narcotic | Family Offense | Fraud | Gambling | Kidnapping-Abduction | Larceny-Theft | Liquor Law | Motor Vehicle Theft | Non-reportable | Other Assault | Pornography-Obscene Material | Prostitution | Robbery | Sex Offense | Stolen Property | Trespass of Real Property | Unclassifiable | Vandalism | Weapon Law | pop | pop2 | nonHispCt | white | black | AIandAN | asian | NHandOPI | hisp | wtHisp | blkHisp | nomoveCt | nomove | singleFamCt | singleFam | povertyCt | poverty | incAgg | foodStampCt | foodStamp | highLaborCt | highUnemp | rentCt | rentAgg | whitePct | blkPct | asianPct | hispPct | nomovePct | singleFamPct | povertyPct | incPerCap | foodStampPct | highUnempPct | rentAvg |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2022 | 36005000100 | NYC | 0 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4446 | 4446 | 3274 | 1098 | 2000 | 9 | 123 | 0 | 1172 | 800 | 64 | 4446 | 1871 | 0 | 0 | 0 | 0 | 19849600 | 0 | 0 | 0 | 0 | 0 | 0 | 0.2469636 | 0.4498426 | 0.0276653 | 0.2636077 | 0.4208277 | NA | NA | 4464.597 | NA | NA | NA |
2022 | 36005000200 | NYC | 21 | 74 | 1 | 4 | 1 | 0 | 0 | 15 | 0 | 3 | 0 | 2 | 0 | 0 | 72 | 0 | 8 | 5 | 33 | 0 | 0 | 13 | 0 | 3 | 0 | 0 | 25 | 2 | 4870 | 4870 | 1761 | 83 | 1281 | 0 | 299 | 0 | 3109 | 667 | 192 | 4870 | 4618 | 4863 | 67 | 1076 | 68 | 163783100 | 1425 | 196 | 423 | 0 | 567 | 1004600 | 0.0170431 | 0.2630390 | 0.0613963 | 0.6383984 | 0.9482546 | 0.0137775 | 0.0631970 | 33631.027 | 0.1375439 | 0.0000000 | 1771.7813 |
2022 | 36005000400 | NYC | 13 | 54 | 0 | 5 | 1 | 0 | 0 | 9 | 0 | 1 | 0 | 1 | 0 | 0 | 74 | 0 | 17 | 1 | 26 | 0 | 0 | 6 | 0 | 1 | 0 | 0 | 17 | 1 | 6257 | 6257 | 2045 | 283 | 1559 | 0 | 103 | 0 | 4212 | 507 | 338 | 6152 | 5680 | 6238 | 193 | 1516 | 109 | 240996800 | 2309 | 384 | 408 | 0 | 900 | 1320900 | 0.0452293 | 0.2491609 | 0.0164616 | 0.6731661 | 0.9232770 | 0.0309394 | 0.0718997 | 38516.350 | 0.1663058 | 0.0000000 | 1467.6667 |
2022 | 36005001600 | NYC | 29 | 66 | 0 | 6 | 3 | 0 | 0 | 23 | 3 | 3 | 0 | 12 | 0 | 0 | 171 | 0 | 10 | 10 | 42 | 0 | 0 | 24 | 0 | 2 | 2 | 0 | 15 | 11 | 6177 | 6177 | 2670 | 106 | 2132 | 213 | 148 | 0 | 3507 | 445 | 160 | 6153 | 5617 | 5952 | 149 | 1616 | 225 | 172386200 | 2205 | 955 | 469 | 50 | 1751 | 2194900 | 0.0171604 | 0.3451514 | 0.0239599 | 0.5677513 | 0.9128880 | 0.0250336 | 0.1392327 | 27907.755 | 0.4331066 | 0.1066098 | 1253.5123 |
2022 | 36005001900 | NYC | 43 | 150 | 0 | 68 | 10 | 1 | 0 | 32 | 10 | 2 | 0 | 18 | 0 | 1 | 152 | 0 | 58 | 51 | 79 | 0 | 0 | 23 | 0 | 3 | 3 | 0 | 106 | 8 | 4064 | 4064 | 2346 | 508 | 1298 | 34 | 9 | 342 | 1718 | 212 | 253 | 4017 | 3439 | 3556 | 251 | 837 | 193 | 129047100 | 1408 | 404 | 456 | 25 | 1337 | 2408800 | 0.1250000 | 0.3193898 | 0.0022146 | 0.4227362 | 0.8561115 | 0.0705849 | 0.2305854 | 31753.716 | 0.2869318 | 0.0548246 | 1801.6455 |
2022 | 36005002000 | NYC | 39 | 191 | 2 | 18 | 2 | 0 | 0 | 56 | 1 | 25 | 0 | 12 | 0 | 2 | 88 | 0 | 22 | 11 | 82 | 0 | 0 | 20 | 0 | 3 | 3 | 0 | 46 | 9 | 8376 | 8376 | 3217 | 42 | 2738 | 0 | 0 | 0 | 5159 | 1221 | 514 | 8350 | 8159 | 8346 | 603 | 1961 | 549 | 228090100 | 3642 | 2088 | 1131 | 4 | 3144 | 2116300 | 0.0050143 | 0.3268863 | 0.0000000 | 0.6159265 | 0.9771257 | 0.0722502 | 0.2799592 | 27231.387 | 0.5733114 | 0.0035367 | 673.1234 |
2022 | 36005002300 | NYC | 17 | 68 | 1 | 6 | 2 | 0 | 0 | 18 | 0 | 1 | 0 | 2 | 0 | 0 | 22 | 0 | 6 | 5 | 28 | 0 | 0 | 7 | 0 | 0 | 1 | 0 | 22 | 2 | 4528 | 4528 | 1556 | 7 | 1499 | 0 | 0 | 0 | 2972 | 483 | 437 | 4528 | 4285 | 4523 | 397 | 1097 | 387 | 60099900 | 1923 | 1044 | 307 | 75 | 1923 | 1437000 | 0.0015459 | 0.3310512 | 0.0000000 | 0.6563604 | 0.9463339 | 0.0877736 | 0.3527803 | 13272.946 | 0.5429017 | 0.2442997 | 747.2699 |
2022 | 36005002400 | NYC | 9 | 13 | 1 | 2 | 1 | 0 | 0 | 3 | 1 | 1 | 0 | 1 | 0 | 0 | 8 | 0 | 2 | 3 | 9 | 0 | 0 | 7 | 0 | 0 | 0 | 0 | 6 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
2022 | 36005002500 | NYC | 56 | 135 | 2 | 23 | 12 | 1 | 0 | 31 | 3 | 37 | 1 | 7 | 0 | 0 | 316 | 0 | 17 | 18 | 109 | 0 | 0 | 42 | 0 | 4 | 4 | 0 | 47 | 18 | 5642 | 5642 | 1496 | 305 | 1129 | 0 | 0 | 0 | 4146 | 746 | 433 | 5562 | 4980 | 5431 | 266 | 1306 | 478 | 80228200 | 1944 | 841 | 650 | 83 | 1843 | 2124900 | 0.0540588 | 0.2001063 | 0.0000000 | 0.7348458 | 0.8953614 | 0.0489781 | 0.3660031 | 14219.816 | 0.4326132 | 0.1276923 | 1152.9571 |
2022 | 36005002701 | NYC | 22 | 67 | 1 | 7 | 3 | 1 | 0 | 41 | 1 | 11 | 0 | 7 | 0 | 0 | 51 | 0 | 6 | 7 | 43 | 0 | 0 | 19 | 0 | 3 | 0 | 1 | 22 | 6 | 3335 | 3335 | 949 | 16 | 917 | 0 | 16 | 0 | 2386 | 161 | 325 | 3270 | 3172 | 3329 | 299 | 793 | 411 | 40494700 | 1159 | 670 | 246 | 65 | 1136 | 1064500 | 0.0047976 | 0.2749625 | 0.0047976 | 0.7154423 | 0.9700306 | 0.0898168 | 0.5182850 | 12142.339 | 0.5780846 | 0.2642276 | 937.0599 |
2022 | 36005002702 | NYC | 25 | 37 | 0 | 7 | 3 | 1 | 0 | 18 | 5 | 5 | 0 | 6 | 0 | 0 | 33 | 0 | 7 | 8 | 38 | 0 | 0 | 9 | 0 | 2 | 0 | 0 | 22 | 6 | 5000 | 5000 | 1422 | 7 | 1359 | 0 | 0 | 0 | 3578 | 207 | 243 | 4862 | 4277 | 4479 | 300 | 1034 | 469 | 78081600 | 1530 | 813 | 504 | 50 | 1491 | 1784900 | 0.0014000 | 0.2718000 | 0.0000000 | 0.7156000 | 0.8796791 | 0.0669792 | 0.4535783 | 15616.320 | 0.5313725 | 0.0992063 | 1197.1160 |
2022 | 36005002800 | NYC | 19 | 79 | 0 | 5 | 3 | 0 | 0 | 17 | 3 | 1 | 0 | 8 | 0 | 0 | 115 | 0 | 24 | 13 | 51 | 0 | 0 | 20 | 0 | 1 | 1 | 1 | 33 | 6 | 5230 | 5230 | 3390 | 31 | 3359 | 0 | 0 | 0 | 1840 | 437 | 104 | 5196 | 5173 | 5225 | 275 | 1225 | 29 | 197797100 | 2542 | 252 | 464 | 0 | 2306 | 2756600 | 0.0059273 | 0.6422562 | 0.0000000 | 0.3518164 | 0.9955735 | 0.0526316 | 0.0236735 | 37819.713 | 0.0991345 | 0.0000000 | 1195.4033 |
2022 | 36005003100 | NYC | 17 | 49 | 0 | 8 | 3 | 0 | 0 | 21 | 1 | 1 | 0 | 6 | 0 | 0 | 69 | 0 | 22 | 3 | 23 | 0 | 0 | 10 | 0 | 0 | 0 | 1 | 26 | 2 | 2559 | 2559 | 1090 | 36 | 831 | 0 | 146 | 0 | 1469 | 398 | 146 | 2519 | 2487 | 2559 | 369 | 778 | 127 | 58765300 | 1031 | 488 | 280 | 114 | 900 | 1366800 | 0.0140680 | 0.3247362 | 0.0570535 | 0.5740524 | 0.9872965 | 0.1441970 | 0.1632391 | 22964.166 | 0.4733269 | 0.4071429 | 1518.6667 |
2022 | 36005003300 | NYC | 14 | 42 | 2 | 3 | 4 | 1 | 0 | 23 | 2 | 0 | 0 | 3 | 0 | 1 | 33 | 0 | 8 | 9 | 25 | 0 | 0 | 9 | 0 | 0 | 2 | 0 | 21 | 4 | 3559 | 3559 | 1262 | 85 | 1081 | 0 | 0 | 0 | 2297 | 384 | 233 | 3559 | 3484 | 3424 | 392 | 780 | 327 | 49035800 | 1220 | 704 | 180 | 0 | 1058 | 1090600 | 0.0238831 | 0.3037370 | 0.0000000 | 0.6454060 | 0.9789267 | 0.1144860 | 0.4192308 | 13777.971 | 0.5770492 | 0.0000000 | 1030.8129 |
2022 | 36005003500 | NYC | 27 | 51 | 0 | 1 | 3 | 0 | 0 | 19 | 4 | 5 | 0 | 7 | 0 | 0 | 63 | 0 | 17 | 6 | 47 | 0 | 0 | 12 | 0 | 2 | 3 | 0 | 34 | 1 | 3899 | 3899 | 919 | 112 | 705 | 0 | 0 | 0 | 2980 | 561 | 893 | 3876 | 3685 | 3855 | 487 | 1128 | 292 | 77401900 | 1478 | 754 | 351 | 0 | 1420 | 1832400 | 0.0287253 | 0.1808156 | 0.0000000 | 0.7642985 | 0.9507224 | 0.1263294 | 0.2588652 | 19851.731 | 0.5101488 | 0.0000000 | 1290.4225 |
2022 | 36005003700 | NYC | 14 | 18 | 0 | 3 | 3 | 0 | 0 | 5 | 2 | 5 | 0 | 7 | 0 | 0 | 28 | 0 | 7 | 14 | 18 | 0 | 0 | 12 | 0 | 1 | 0 | 0 | 15 | 6 | 331 | 331 | 66 | 15 | 51 | 0 | 0 | 0 | 265 | 30 | 85 | 305 | 305 | 324 | 23 | 80 | 0 | 6280500 | 188 | 102 | 0 | 0 | 188 | 119800 | 0.0453172 | 0.1540785 | 0.0000000 | 0.8006042 | 1.0000000 | 0.0709877 | 0.0000000 | 18974.320 | 0.5425532 | NA | 637.2340 |
2022 | 36005003800 | NYC | 3 | 16 | 0 | 1 | 0 | 0 | 0 | 4 | 0 | 0 | 0 | 1 | 0 | 0 | 11 | 0 | 1 | 2 | 10 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 4 | 1 | 1182 | 1182 | 228 | 16 | 176 | 0 | 36 | 0 | 954 | 450 | 24 | 1135 | 1075 | 1150 | 43 | 304 | 33 | 44416500 | 366 | 49 | 156 | 11 | 214 | 281400 | 0.0135364 | 0.1489002 | 0.0304569 | 0.8071066 | 0.9471366 | 0.0373913 | 0.1085526 | 37577.411 | 0.1338798 | 0.0705128 | 1314.9533 |
2022 | 36005003900 | NYC | 67 | 135 | 0 | 15 | 18 | 0 | 0 | 53 | 14 | 54 | 0 | 23 | 0 | 0 | 133 | 0 | 16 | 30 | 98 | 0 | 0 | 29 | 178 | 2 | 1 | 0 | 57 | 21 | 6227 | 6227 | 462 | 155 | 224 | 68 | 15 | 0 | 5765 | 674 | 122 | 6227 | 6042 | 6103 | 259 | 1131 | 390 | 129275100 | 2330 | 1544 | 819 | 50 | 2247 | 2761800 | 0.0248916 | 0.0359724 | 0.0024089 | 0.9258070 | 0.9702907 | 0.0424381 | 0.3448276 | 20760.414 | 0.6626609 | 0.0610501 | 1229.1055 |
2022 | 36005004001 | NYC | 7 | 28 | 0 | 2 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 3 | 0 | 0 | 28 | 0 | 13 | 2 | 15 | 0 | 0 | 7 | 0 | 0 | 0 | 0 | 11 | 0 | 1409 | 1409 | 609 | 60 | 158 | 17 | 343 | 0 | 800 | 145 | 36 | 1399 | 1364 | 1409 | 26 | 359 | 36 | 55978300 | 529 | 76 | 123 | 13 | 293 | 444600 | 0.0425834 | 0.1121363 | 0.2434351 | 0.5677786 | 0.9749821 | 0.0184528 | 0.1002786 | 39729.099 | 0.1436673 | 0.1056911 | 1517.4061 |
2022 | 36005004100 | NYC | 44 | 151 | 2 | 13 | 5 | 1 | 0 | 42 | 0 | 21 | 0 | 4 | 0 | 0 | 88 | 0 | 12 | 9 | 88 | 0 | 0 | 21 | 0 | 3 | 2 | 2 | 57 | 15 | 6583 | 6583 | 1982 | 30 | 1896 | 0 | 56 | 0 | 4601 | 1029 | 814 | 6534 | 5924 | 6482 | 197 | 1563 | 540 | 115443200 | 2444 | 1159 | 494 | 0 | 2260 | 2030100 | 0.0045572 | 0.2880146 | 0.0085068 | 0.6989215 | 0.9066422 | 0.0303919 | 0.3454894 | 17536.564 | 0.4742226 | 0.0000000 | 898.2743 |
For any given years (or groups of years, or groups of months if needed) and any given crimes, snapshots like the following can be produced using mapView
,