Re: [betaNYC] Where's the Green Book?

From: Andrew N.
Sent on: Tuesday, May 6, 2014 12:28 PM
Apologies for the abbreviated response. Many of NYC's datasets are updated on a daily basis (311 service requests, ECB violations, etc). ACRIS is monthly. NYC has a formal policy for open data, as required by Local Law 11 of 2012. It covers many of the considerations you raised.

-Andrew


On Tue, May 6, 2014 at 12:20 PM, Ralph Yozzo <[address removed]> wrote:
Hi Chris,

Thanks for the helpful response.  I did not know about the syncing.  I agree with you that syncing is a sustainable alternative until we build new systems with "open by default" built in.

I would suggest that the "ACRIS data", "Assessment Data" and the "Statement of Account" data should be synced daily.  Especially, since external companies get some of this data daily.  That means the process already exists.

Thank goodness for you and Noel, etc.!


On Tue, May 6, 2014 at 12:09 PM, Chris Whong <[address removed]> wrote:
"This will be an ongoing issue as long as we consider "placing the data online" as an afterthought or an additional process."

Providing public API access into a production database (especially an old one) is going to be problematic for a number of reasons, and frankly that's always going to be secondary to whatever the actual operational reason-for-existence of a government information system.  (though I think we'll start to see it "out of the box" from more and more vendors as demand grows)  I agree, "open by default" is an attitude that can be adopted by an agency, but may not be so simple to implement on their existing information systems.

(Disclaimer: I work for Socrata) What a data publishing platform like Socrata does is accept arbitrary data of any size and shape, put it in a catalog with associated metadata, displaces it from the production system, wraps it in an standardized API, allows for multiple export formats, and allows for user-created charts, maps and filters, etc...   It handles the public accessibility piece that the systems of record don't/can't/shouldn't.  

While the "giant data dump" is the case for most datasets, several in NYC are synced on a daily basis via an ETL process, such as 311 and ECB.  We're even seeing some cities syncing data on sub-daily basis (the city of Seattle publishes 911 calls every 10 minutes).   

Of course, the goal should be to get the data syncing as close to real-time as humanly and machine-ly possible for our highest-value information systems.  

-Chris







--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Chris Whong ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Chris Whong, visit his/her member profile

To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Ralph Yozzo ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Ralph Yozzo, visit his/her member profile

To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy