No that's a GREAT write up! Thanks!  Only thing I spotted is the Chrome/Firefox plugin is called 'YSlow' not 'WhySlow' but I can definitely see how someone would think that! hahaha 

--
Frank Wiles
Revolution Systems | http://www.revsys.com/
[address removed]   | (800)[masked]

On Wed, Oct 22, 2014 at 5:30 PM, Fred Stluka <[address removed]> wrote:

PUG folks (and Frank),

Frank's slides don't include all the valuable tidbits that he
tossed at us while he was going through the slides.

Here are my notes, which also include some additional details
I dug up shortly after the talk.

Frank, did I get anything wrong, or miss anything?

    - ipython
      - Better interactive shell
      - embed
        - Enters debugger shell
            from IPython import embed
            def some_complicated_code():
                """  Maybe a daemon or other console process """
                if some_failure_condition: 
                    embed()
    - pip
      - requirements/base.txt
      - requirements/dev.txt, requirements/test.txt, requirements/prod.txt
        - Include base.txt with line:  -r base.txt
        - Add env specific lines after that 
      - pip install piptools
        - pip-review
          - Shows what versions you are using and could update to
    - PYTHONSTARTUP=/path/to/some/code.py
    - virtualenvwrapper
      - workon env_name (no need for full path)
        - Does activate and sets up post-deactivate hook
        - Runs these files from <venv>/bin
          - preactivate
          - postactivate
          - predeactivate
          - postdeactivate
      - Frank uses it, even when not doing python, just for Linux shell
        environment setup/teardown with separate env for each working
        directory that he does a workon to
    - Fabric
      - Python script for sys admin, even to remote hosts
      - local("shell command")
      - Use its "roles" instead of Puppet, Chef, Ansible, etc.
    - py.test --reuse-db
      - For speed, don't re-create the DB each time
    - Sphinx docs
      - sphinx-apidoc uses "autodoc" extension to generate Javadoc-like
        docs from Python code
    - coverage
    - Use a Git tag to record date/time of a PROD deployment
    - logging_tree
      - To analyze/debug logging config
    - Tools for streaming logs to for filtering and event notification
      - loggly.com
      - logentries.com
      - PaperTraillApp.com
    - honcho
       - Like Foreman or Heroku
       - Configure a ProcFile
    - ack
      - Better grep
      - ack --python
        - Searches only .py files
    - jq
      - Cmd line tool to parse/process/filter JSON
    - django-debug-toolbar
      - Run in browser, but server must be in debug mode
      - Reports timing, SQL queries, etc
      - Good for profiling, performance, SQL, Django server, etc.
      - What line in a template did DB access?
      - Templates used
      - Data passed to template
      - Caching
      - Signals
      - Python logging messages generated, etc.
      - Intercept and debug browser redirects
    - Gulp
      - gulpjs.com
      - livereload.listen()
        - Can do SASS preprocessing, minify, etc
        - Watch for templates that change, and do a live reload via a
          browser plugin.
          - Edit template file, browser refreshes automatically
          - Can have 3 different screen sizes and have them all autorefresh
            whenever you edit a template
    - See what's new occasionally
      - Mailing lists:
        - python-weekly
        - django-weekly
        - Do Google searches occasionally, for past month only, for tips
          on your favorite tools
    - ssh has a ControlMaster feature to reuse same SSH connection from
      another process
      - In .ssh/config
          ControlMaster auto
          ControlPath /tmp/localhost-%l-remotehost-%h-port-%p-user-%r
          host names, ports, users, etc.
    - Celery alternatives
      - rq
      - Use redis as a lightweight broker
    - Use Celery for simple task scheduling?
      - No.  Use cron to run django-admin.py or manage.py
    - IDE
      - Until recently, Frank always used vi and shell
      - Switched to sublime in vi mode
      - Doesn't like slow startup time of an IDE
      - Doesn't need IDE's debuggers
      - Has clients that use PyCharm
    - Speed
      - C and Go may be faster for some workloads, like video editing
      - For websites, developer time is much faster in Python, and speed
        is cheap: caching, server farm, etc.
      - Drupal is much slower than Django
      - Don't often need to go to Java
      - C libraries callable from Python:  numpy, scipy
    - pypy - JIT compiler
    - yslow
      - Chrome plugin for why a Web site is slow
    - pagespeed
      - Same as yslow, but by Google
    - Typical performance opportunities:
      - DB connection pooling, as of Django 1.6, or with an add-on previously
        From:
        - http://media.revsys.com/media/talks/slides/performance-slides.pdf
        DATABASES = {
            'default': {
                'ENGINE':
                'django.db.backends.postgresql_psycopg2',
                'NAME': 'rando',
                'USER': 'rando',
                'CONN_MAX_AGE': 300,
            }
        }
      - DB caching
      - Page caching
        - Varnish
      - More server processes/threads per box
        - One for Apache, one for memcached, one per remaining core for WSGI.
      - Template caching
        - django.template.loaders.cached.Loader
          - Like collectstatic for static files
          - To avoid searching all template subdirs all the time for each
            template file.
          - Details from:
            - http://media.revsys.com/media/talks/slides/performance-slides.pdf
            Change this:
                TEMPLATE_LOADERS = (
                    'django.template.loaders.filesystem.Loader',
                    'django.template.loaders.app_directories.Loader',
                )
            Do this, in PROD settings.py only, so template changes will
            still be picked up on the fly in DEV and TEST:
                TEMPLATE_LOADERS = (
                    ('django.template.loaders.cached.Loader', (
                        'django.template.loaders.filesystem.Loader',
                        'django.template.loaders.app_directories.Loader',
                    )),
                )
      - Avoid include, use {% block %} of parent template instead
      - Far future "expires" headers (at least an hour)
      - django-compress
        - {% compress css %}
           - Generates combined CSS file with a hashed name.
           - Fewer separate CSS file requests.
           - Can use a far future expires header because any change to
             any CSS file generates a different hashed composite name
      - Session caching, from:
        - http://media.revsys.com/media/talks/slides/performance-slides.pdf
        Change this:
            SESSION_ENGINE = 'django.contrib.sessions.backends.db'
        To this:
            CACHES = {
                'default': {
                    'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
                    'LOCATION': '[masked]:11211',
                }
            }
            SESSION_ENGINE = "django.contrib.sessions.backends.cache"        
        You should really be caching your data at some level anyway, so
        hopefully you have memcached or redis setup already but just in
        case here is a simple memcached setup. Just change your
        SESSION_ENGINE to store your sessions there as well. If you are
        using session data, you can use cached_db to only write to the DB
        when the session data actually changes.
      - Switch from Apache to nginx
      - Switch from MySQL to PostgreSQL
      - Switch from standard Django templates to Jinja2 templates
      - See also:
        - https://speakerdeck.com/jacobian
          /django-minus-django-djangocon-eu-2014
    - disqus is built with Django
    - Flask is like Django, but lighter weight, less capable
    - Bottle is similar to Flask
    - No concerns about Python 3.  He's dropping Python 2.

Enjoy!

--Fred

---

Fred Stluka -- mailto:[address removed] -- http://bristle.com/~fred/
Bristle Software, Inc -- http://bristle.com -- Glad to be of service!
Open Source: Without walls and fences, we need no Windows or Gates.

---

On 10/22/14 9:30 AM, Timothy Michael wrote:

Frank Wiles has posted the slides from his talk here:

http://www.revsys.com/talks/2014/philly-pug-python-pro-tips/

I must apologize - while the stream worked just fine (we had between 10-12 people remote), the recording didn't come through. That's a first for me... we'll figure it out by next time.

Regards,

-Tim

--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Timothy Michael ([address removed]) from The Philadelphia Python Users Group (PhillyPUG).
To learn more about Timothy Michael, visit his/her member profile
To report this message or block the sender, please click here
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]