align-toparrow-leftarrow-rightbackbellblockcalendarcamerachatcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-crosscrosseditfacebookglobegoogleimagesinstagramlocation-pinmagnifying-glassmailmoremuplabelShape 3 + Rectangle 1outlookpersonplusImported LayersImported LayersImported Layersshieldstartwitteryahoo

Re: [betaNYC] Where's the Green Book?

From: Chris W.
Sent on: Tuesday, May 6, 2014 12:09 PM
"This will be an ongoing issue as long as we consider "placing the data online" as an afterthought or an additional process."

Providing public API access into a production database (especially an old one) is going to be problematic for a number of reasons, and frankly that's always going to be secondary to whatever the actual operational reason-for-existence of a government information system.  (though I think we'll start to see it "out of the box" from more and more vendors as demand grows)  I agree, "open by default" is an attitude that can be adopted by an agency, but may not be so simple to implement on their existing information systems.

(Disclaimer: I work for Socrata) What a data publishing platform like Socrata does is accept arbitrary data of any size and shape, put it in a catalog with associated metadata, displaces it from the production system, wraps it in an standardized API, allows for multiple export formats, and allows for user-created charts, maps and filters, etc...   It handles the public accessibility piece that the systems of record don't/can't/shouldn't.  

While the "giant data dump" is the case for most datasets, several in NYC are synced on a daily basis via an ETL process, such as 311 and ECB.  We're even seeing some cities syncing data on sub-daily basis (the city of Seattle publishes 911 calls every 10 minutes).   

Of course, the goal should be to get the data syncing as close to real-time as humanly and machine-ly possible for our highest-value information systems.  


People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy