Re: [betaNYC] NYPD releases Motor Vehicle Collisions data

From: Mark H.
Sent on: Wednesday, May 7, 2014 10:10 PM
Noel,

Thanks for the questions. 

First, the name of the data set on CivicData.io is taken from the NYPD website where the source data is posted. Not meant to be misleading, but rather to stay consistent with the source. Good to know that there are other types of instances covered.

CivicData.io is built on CKAN, an open source data portal which has its own set of APIs for querying data - in that regard, it offers developers another choice as to how they want to use the data, just as other API-enabled platforms do (like CartoDB). As the data from NYPD is updated, the data on CivicData.io will also be updated - although we'll do this manually as new data gets posted on the NYPD site.

In addition, since anyone can create an account (without cost) on CivicData.io, members of BetaNYC (and any other data users in NYC for that matter) can add their own data sets to this site - allowing data from different source to commingle and be mashed up.

Providing the data through CivicData.io is simply meant to provide another option on how to use this important new data. I hope the members of BetaNYC (and others) find it useful.

Thanks!


On Wed, May 7, 2014 at 9:45 PM, Noel Hidalgo | BetaNYC <[address removed]> wrote:
Mark,

Interesting, but this isn't just motor vehicle collision data. This data also includes bike on ped collisions...

Also, what value add does civicdata.io offer that the city's data catalogue doesn't?

How is your API different?

As this data gets updated, will civicdata.io also be updated?

Not trying to be snarky, just trying to have a better understanding of civicdata.io in the NYC ecosystem. 


--
Sent from my TI–85

On 7 May 2014, at 21:27, Mark Headd <[address removed]> wrote:

For those interested, I've uploaded this data to CivicData.io. I maintained the same basic structure of the files on the NYPD site - each Excel file had to worksheets, so each borough has 2 CSV files:

http://www.civicdata.io/en/dataset/nypd-motor-vehicle-collision-data

This is a CKAN instance, so it has some really nice APIs for querying and using the data. My favorite is the SQL Search API that lets you execute SQL queries via an API call. This makes querying the data as simple as cooking up SQL statements. I've added a few to demonstrate how to use it here:


Hope folks find this useful. Any questions, just let me know. 

Thanks!



On Wed, May 7, 2014 at 6:20 PM, Chris Whong <[address removed]> wrote:
To be as predictable as Andrew, here's a quick 'n' dirty animation of my zip code's collisions in downtown Brooklyn:


:)  


-Chris




On Wed, May 7, 2014 at 5:56 PM, Andrew W. Hill <[address removed]> wrote:
Very interesting. Not to be too predictable, but here are a couple of maps





I'd love to find some time to try and link these to the data to make them refresh each day... in time. 




On Wed, May 7, 2014 at 5:29 PM, fkh <[address removed]> wrote:
Great statement John and Noel. 

Regarding update frequency, my understanding is that this will be updated daily. Not sure how quickly a crash report is entered into the database though, and whether some precincts will lag. 

Regarding locations: I believe we are getting exactly what the PD has. There's no more detailed location data available for this data set, due to limitations in how the paperwork is filled out at the scene of the crash. Some crashes get a more thorough investigation by the Collision Investigation Service (CIS), but the more accurate location data doesn't get merged into the regular crash records we have here. Options for improved location quality:
- sharing the written description of the crash from the crash report
- sharing the sketch diagram from the crash report
- sharing CIS reports in an additional data set
- collecting better location data, by instructing precincts to fill out the form more completely (could happen at the precinct level(
- switching to mobile data collection tools with gps. 

On Wednesday, May 7, 2014, Noel Hidalgo | BetaNYC <[address removed]> wrote:
This IS a big deal. We've won one more open data set!!! Some of us have been fighting for this data since 2011!!!!



On Wed, May 7, 2014 at 3:52 PM, fkh <[address removed]> wrote:
This is a big deal -- it's disaggregate crash data, including date and time. No more bundling by intersection and month. 

Locations are still the closest intersection, but it's the best we can expect from the source data. 


On Wednesday, May 7, 2014, Ellen McDermott <[address removed]> wrote:
This is pretty exciting. Andrew Nicklin tweeted this, which looks like it's being rolled out especially for the BigApps kick off tonight. You can sort data by borough, street, injury, fatality, and more. The data looks up to date as of yesterday!

https://data.cityofnewyork.us/NYC-BigApps/NYPD-Motor-Vehicle-Collisions/h9gi-nx95

Best,

Ellen. 




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Ellen McDermott ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Ellen McDermott, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by fkh ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about fkh, visit his/her member profile

To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]
This message was sent by Noel Hidalgo | BetaNYC ([address removed]) from #betaNYC, a Code for America Brigade for NYC.

To learn more about Noel Hidalgo | BetaNYC, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]




--

Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by fkh ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about fkh, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages




--

Andrew W. Hill, PhD, Senior Scientist 
www.vizzuality.com | @andrewxhill

Map, analyze and build applications with your data




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Andrew W. Hill ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Andrew W. Hill, visit his/her member profile

To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Chris Whong ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Chris Whong, visit his/her member profile

To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Mark Headd ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Mark Headd, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Noel Hidalgo | BetaNYC ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Noel Hidalgo | BetaNYC, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]



--

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy