addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Fwd: [betaNYC] Day care map project

From: Anita S.
Sent on: Friday, March 28, 2014 4:45 PM
Hi all,

Here is an update on the day care map project and a shout out to all python developers (and php or javascript developers):

I contacted the people behind another Bigapp 2013 project that Will pointed out to me (thanks!): http://www.datanarratives.org/nyc/day-care-finder/index.html

They did exactly what I wanted to do, but only once last May and the data has been frozen since. Turns out they had a developer who wrote the scraper in php, javascript etc., but he seems reluctant to share the code. He also says that one cannot automate it and that one would need to write the code in python (this was all relayed to me by the project manager, not the developer).

So, I am back to my initial plan to write scraping code in python and am looking for anybody with experience in python or anybody with experience with php, javascript etc who would be willing to help me figure out how to do this in python (I need to understand how that particular webpage and database works). We have a working scraper in php (thanks Baris) for the highest level of data (names, addresses), so we have a starting point. Also, I have been playing around with the scrapy package (http://scrapy.org/) and have had some success retrieving data from the database (https://a816-healthpsi.nyc.gov/ChildCare/SearchAction2.do), but have not succeeded in automating it.

The project is still on github: https://github.com/child-care-map/NYCmap with the data with have gathered so far and Sonya is working on geocoding the addresses.

I also contacted the Commissioner for the Department of Health and Mental Hygiene (Mary Travis Bassett) over their online form and asked if they consider releasing the data through NYC Open Data but have not heard back.

Also, everyone who has a NYC Open Data login, please bump the request to release this data here: https://nycopendata.socrata.com/nominate/2570

Email me at [address removed] if you are interested to get involved!

Thanks, Anita






---------- Forwarded message ----------
From: Colegrove, Will (ManhattanBP) <[address removed]>
Date: Thu, Mar 20, 2014 at 5:37 PM
Subject: RE: [betaNYC] Day care map project
To: "[address removed]" <[address removed]>


Anita:

 

2 previous bigapps contestants have done similar projects that you might want to look at before starting the scraping:

 

http://2013.nycbigapps.com/project/300/nyc-day-care-finder-choose-with-confidence

http://2013.nycbigapps.com/project/151/child-care-desk-find-quality-child-care-near-you

 

-Will

 

William Colegrove

Director of Budget & Transparency

Manhattan Borough President Gale A. Brewer

1 Centre Street, 19th Fl South

New York, NY 10007

p: [masked]

f: [masked]

[address removed]

 

 

From: [address removed] [mailto:[address removed]] On Behalf Of Ben Sacks
Sent: Thursday, March 20,[masked]:34 PM
To: [address removed]
Subject: Re: [betaNYC] Day care map project

 

Whoa! I don't have any experience with scraping but I am looking for day care for my child right now and will be super psyche to use this. 

 

The one thing I can say is that I was under the impression that day care centers were licensed through the state and not the city, which is likely why it's not in nyc open data. 

 

Good luck and please keep me in the loop about progress. Thanks so much for doing this. 

Sent from my iPhone


On Mar 20, 2014, at 5:30 PM, Anita Schmid <[address removed]> wrote:

Hi all,

 

I would like to make a map of all licensed daycare centers in NYC. The data is online on https://a816-healthpsi.nyc.gov/ChildCare/ChildCareList.do but it is not on NYC Open Data despite someone suggesting it a year ago.

 

So, I am thinking of scraping the data off the site above. Anybody interested in working on this with me? Anybody have experience with scraping? I am looking at Scrapy (http://scrapy.org/), but don't know if that will work.

 

If you are interested or have feedback, please email me at [address removed].

 

Thanks! Anita





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Anita Schmid ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Anita Schmid, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed] Image removed by sender.





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Ben Sacks ([address removed]) from #betaNYC, a Code for America Brigade for NYC.
To learn more about Ben Sacks, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed] Image removed by sender.


This email message originally included an attachment.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy