addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1linklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Re: [indypy] Supervisor logging

From: Brad F.
Sent on: Thursday, June 28, 2012 2:47 PM
Matt,

Mostly, yes.  I ran into a couple minor problems:

 1. I did not read the docs well enough and missed the mkfs.s3ql
    step for awhile.

 2. I'm using it on Debian Squeeze which uses python2.6 so it was
    easiest to install from the the Ubuntu Maverick PPA (as
    documented).  I tried pulling in the copy from Wheezy (python2.7),
    but that got messy so I punted and stuck with the Maverick repo.

Couple other potential caveats and things to consider/test:

 * The files are stored in blocks on S3, so you need to mount the S3
   bucket with s3ql to get them back out in usable form.

 * It is FUSE-based, so it works on Linux, might work on OS X and
   probably doesn't work on Windows.

Probably also worth checking the throughput you get, especially if the
supervisord host isn't on EC2, to make sure it will keep up with the
supervisor logging rate.

--Brad


On Thu, Jun 28, 2012 at 02:27:40PM -0400, Matt Deboard wrote:

> Is this as straightforward as it looks?
> 
> On Thu, Jun 28, 2012 at 1:50 PM, Brad Fritz <[address removed]>wrote:
> 
> >
> > Matt,
> >
> > On Thu, Jun 28, 2012 at 01:34:05PM -0400, Matt Deboard wrote:
> >
> > > Does anyone have experience piping supervisor logs directly to external
> > > storage over the wire (i.e. not a mounted drive, a la Amazon S3)?
> >
> > Seems like you could mount an S3 bucket with something like s3ql[1]
> > and then tell supervisor to log to the S3 mount.
> >
> > I just started using s3ql for incremental backups to S3 and it's
> > working well so far.
> >
> > --Brad
> >
> > [1] http://code.googl...­

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy