Re: [betaNYC] Scraping help

From: Mike C.
Sent on: Tuesday, August 5, 2014 11:24 AM
I don't know if folks are familiar with it, but ScraperWiki is also a great way to automate scraping and pull data on a consistent basis... also uses Python, but has some built in libraries to make a few tasks easier.


Mike

On Tuesday, August 5, 2014, Andy Hoang <[address removed]> wrote:
Hi Rebecca,

Is there any specific reason for you to use mechanize to make requests?
I switched to requests (http://docs.python-requests.org/en/latest/) and it's running without any problem.

If you don't mind, I will create pull request so you can see what I've changed.


On Tue, Aug 5, 2014 at 10:14 AM, Rebecca Ackerman <[address removed]> wrote:
Hey y'all,

Nikki and I have been working on a scraper for parole hearing data. However! The scraper is timing out at various points. If any of you wonderful people wanted to help troubleshoot, we'd be eternally grateful.


Please fork as you see fit!

-R





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Rebecca Ackerman ([address removed]) from #BetaNYC, a member of Code for America's Brigade Program.
To learn more about Rebecca Ackerman, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Andy Hoang ([address removed]) from #BetaNYC, a member of Code for America's Brigade Program.
To learn more about Andy Hoang, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]


--

Mike Caprio
[address removed]
http://about.me/mik3cap

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy