Activate your local Brigade, DemocracyMap

From: Steven C.
Sent on: Friday, March 15, 2013 11:13 AM
See below for your state and pass on to your local list. Is it on the dark side? :-)

If you are ready to scrape and bake, claim your city or state here:
http://forums.e-democracy.org/groups/democracymap (be sure to click the verification link)

And if you haven't check out the Knight News Challenge submission, see:
http://kng.ht/Z1zXUS
Video tour:

---------- Forwarded message ----------
From: Philip Ashlock
Date: Tuesday, March 5, 2013
Subject: Re: [DMap] DemocracyMap: Help create the first open data set of government bodies and officials
To: [address removed]


Thanks Chad. Yeah, any help to find more data sources would be much
appreciated.

Honestly just discovering the data tends to be the harder and more time
consuming part than creating the scrapers, so anyone who wants to help but
would rather do something other than scrape data would be contributing a
lot by just trying to research and find more data sources.

We're keeping track of all of this on a wiki that anyone can edit (once you
create an account) so please convene there to keep track of data sources.
I've also been taking note of data sources liked PDFs and order forms for
paid printed directories and things like that, just to provide a complete
view of known sources even if they're not accessible by a scraper. The link
to the wiki page is below along with the status of known sources that have
and haven't been scraped as well as states that we've yet to identify a
data source for. I'll see if I can find a better layout for the list on the
wiki or some other way of better delineating this moving forward.

Wiki to keep track of data sources and completed scrapers:
http://pages.e-democracy.org/DemocracyMap_Representatives

States that still need scrapers:

Alabama
Arizona
Arkansas
Colorado
Connecticut
Delaware
Georgia
Iowa
Kansas
Kentucky
Louisiana
Maryland
Mississippi
Nevada
New Jersey
South Carolina
Tennessee
Vermont

Done
x Texas
x California
x Pennsylvania
x Washington State
x District of Columbia
x Oregon
x Florida
x North Carolina
x Massachusetts

No public data available so nothing to scrape for these states:
(just based on last investigation, any help to research further would be
much appreciated)

Alaska
Hawaii
Idaho
Illinois
Indiana
Maine
Michigan
Minnesota
Missouri
Montana
Nebraska
New Hampshire
New Mexico
New York
North Dakota
Ohio
Oklahoma
Puerto Rico
Rhode Island
South Dakota
Utah
Virginia
West Virginia
Wisconsin
Wyoming

On Tue, Mar 5, 2013 at 11:23 PM, Chad Robinson
<[address removed]>wrote:

> I found this page of AL municipality data. I have no ASP understanding so I
> can't snatch this data myself but if someone with the requisite skills
> wants to take a stab at it, here you go:
> http://directory.alalm.org/directoryweb/default.aspx I will continue to
> try
> to locate decent data sources for y'all to scrape since I am mostly inept
> at that part. Is there a better place to store this information. I was
> thinking there was some kind of wiki or hackpad where I could stack these
> sources as I located them.
>
>
> On Tue, Mar 5, 2013 at 3:54 PM, Philip Ashlock <[address removed]>
> wrote:
>
> > This looks great. I made a few minor tweaks to your scraper - adding the
> > city to each record for an official, and renaming URL to source, but this
> > is good to go. This shouldn't really cause any problems, but since I
> > changed the names of the columns the only way to get the scraper
> datastore
> > to remove the old column names will be to clear the datastore and scrape
> > again. In any case, I've already gone ahead and added this as another
> data
> > source for the DemocracyMap API, so all these reps should now being
> showing
> > up there. However, incorporating the city contact data like the phone
> > numbers scraped here is something I need to do separately.
> >
> > As for the next states to scrape, you're only limited by data we've found
> > to scrape and unfortunately we haven't found anything for NY, IL, or OH.
> > Though Pennsylvania has actually been scraped already. You're welcome to
> > dig around some more to see if you can find data to scrape for any of
> those
> > states, but otherwise you can take your pick from the sources identified
> on
> > the wiki.
> >
> > Best,
> > Phil
> >
> > On Tue, Mar 5, 2013 at 8:28 AM, Dane Summers <[address removed]>
> wrote:
> >
> > > Hi Phil,
> > >
> > > I've updated the NC scraper (previous link posted) so that officials
> and
> > > counties are split out into separate tables. Could you take another
> look?
> > >
> > > It looks like the next largest un-scraped States are NY, IL, PA, OH.
> I'd
> > > be happy to take one of those on next...
> > >
> > > Thanks,
> > >
> > > Dane
> > > Dane Summers
> > > Durham
> > > About/contact Dane Summers:
> > > http://forums.e-democracy.org/p/3ys8PByIYzNZVZm7kF7rVi
> > >
> > > View full topic or share online:
> > >   http://forums.e-democracy.org/r/topic/6sW8fha99TeDLr62DFhpvC
> > >
> > > * Virtual BeNeighbors.org booth and video from a recent conference:
> > >      http://bit.ly/virtbooth
> > >      E-Democracy's BeNeighbors.org Booth at Knight Foundation's Media Learning Seminar
> > >
> > > - - - - - - - - - - - - - - - - - - - - - - - -
> > >   To post: [address removed] or "Reply-to-All" to
> > > comment.
> > >   For digest/to leave: Put "digest on" or "unsubscribe" in Subject (no
> > > quotes)
> > >   Forum Home:  http://forums.e-democracy.org/groups/democracymap
> > > - - - - - - - - - - - - - - - - - - - - - - - -
> > > Need help? http://e-democracy.org/support  Hosting:
> > > http://OnlineGroups.Net
> > >
> >
> > Philip Ashlock
> > Brooklyn, New York City
> >  http://forums.e-democracy.org/r/topic/5J8XUj21n1aHyJSGzugL9M

* Virtual BeNeighbors.org booth and video from a recent conference:
     http://bit.ly/virtbooth
     E-Democracy's BeNeighbors.org Booth at Knight Foundation's Media Learning Seminar

- - - - - - - - - - - - - - - - - - - - - - - -
  To post: [address removed] or "Reply-to-All" to comment.
  For digest/to leave: Put "digest on" or "unsubscribe" in Subject (no quotes)
  Forum Home:  http://forums.e-democracy.org/groups/democracymap
- - - - - - - - - - - - - - - - - - - - - - - -
Need help? http://e-democracy.org/support  Hosting: http://OnlineGroups.Net



--
Steven Clift - http://stevenclift.com
  Executive Director - http://E-Democracy.org 
  Twitter: http://twitter.com/democracy
  Tel/Text: [masked]

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy