Re: [betaNYC] Day care map project

From: Ken
Sent on: Friday, March 28, 2014 11:19 PM
If you just need a process to run the script at a certain time, you can use cron.

http://en.wikiped...­

on Mar 28, 2014, Anita Schmid <[address removed]> wrote:
>Hi all,
>Here is an update on the day care map project and a shout out to all python developers
>(and php or javascript developers):
>I contacted the people behind another Bigapp 2013 project that Will pointed out to
>me (thanks!): http://www.datana...­
>
>They did exactly what I wanted to do, but only once last May and the data has been
>frozen since. Turns out they had a developer who wrote the scraper in php, javascript
>etc., but he seems reluctant to share the code. He also says that one cannot automate
>it and that one would need to write the code in python (this was all relayed to me
>by the project manager, not the developer).
>So, I am back to my initial plan to write scraping code in python and am looking for
>anybody with experience in python or anybody with experience with php, javascript
>etc who would be willing to help me figure out how to do this in python (I need to
>understand how that particular webpage and database works). We have a working scraper
>in php (thanks Baris) for the highest level of data (names, addresses), so we have
>a starting point. Also, I have been playing around with the scrapy package (http://scrapy.org...­) and have had some success retrieving data from the database (https://a816-healt...­), but have not succeeded in automating it.
>The project is still on github: https://github.com...­ with the data with have gathered so far and Sonya is working on geocoding the addresses.
>I also contacted the Commissioner for the Department of Health and Mental Hygiene
>(Mary Travis Bassett) over their online form and asked if they consider releasing
>the data through NYC Open Data but have not heard back.
>Also, everyone who has a NYC Open Data login, please bump the request to release this
>data here: https://nycopendat...­
>
>Email me at [address removed] if you are interested to get involved!
>Thanks, Anita
>
>
>
>
>
>---------- Forwarded message ----------
>From: 
>Colegrove, Will (ManhattanBP) <[address removed]>
> Date: Thu, Mar 20, 2014 at 5:37 PM
>Subject: RE: [betaNYC] Day care map project
>To: "[address removed]" <[address removed]>
> 
>
>Anita: 2 previous bigapps contestants have done similar projects that you might want to look
>at before starting the scraping: http://2013.nycbi...­ -Will William ColegroveDirector of Budget & TransparencyManhatta­n Borough President Gale A. Brewer1 Centre Street, 19th Fl SouthNew York, NY 10007p:[masked]f: [address removed]  From: [address removed] [mailto:[address removed]] On Behalf Of Ben Sacks
> Sent: Thursday, March 20,[masked]:34 PM
> To: [address removed]
> Subject: Re: [betaNYC] Day care map project Whoa! I don't have any experience with scraping but I am looking for day care for
>my child right now and will be super psyche to use this.  The one thing I can say is that I was under the impression that day care centers were
>licensed through the state and not the city, which is likely why it's not in nyc
>open data.  Good luck and please keep me in the loop about progress. Thanks so much for doing
>this. 
> 
> Sent from my iPhone
> On Mar 20, 2014, at 5:30 PM, Anita Schmid <[address removed]> wrote:
>Hi all, I would like to make a map of all licensed daycare centers in NYC. The data is online
>on https://a816-healt...­ but it is not on NYC Open Data despite someone suggesting it a year ago. So, I am thinking of scraping the data off the site above. Anybody interested in working
>on this with me? Anybody have experience with scraping? I am looking at Scrapy (http://scrapy.org...­), but don't know if that will work. If you are interested or have feedback, please email me at [address removed]. Thanks! Anita
> 
> 
> 
> --
> Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
> This message was sent by Anita Schmid ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
> To learn more about Anita Schmid, visit his/her  member profile
> To report this message or block the sender, please  click here
> Set my mailing list to email me  As they are sent |  In one daily email |  Don't send me mailing list messages 
> 
> Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed] 
> 
> 
> 
> --
> Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
> This message was sent by Ben Sacks ([address removed]) from  #betaNYC, a Code for America Brigade for NYC.
> To learn more about Ben Sacks, visit his/her  member profile
> To report this message or block the sender, please  click here
> Set my mailing list to email me  As they are sent |  In one daily email |  Don't send me mailing list messages 
> 
> Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed] 
>
>
> 
> 
> --
> Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
> This message was sent by Anita Schmid ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
> To learn more about Anita Schmid, visit his/her member profile
> To report this message or block the sender, please click here
> Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages 
>
>  Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]  

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy