Smoothing Software Project Scripting - PowerPoint PPT Presentation

Slide1 l.jpg
Download
1 / 56

Smoothing Software Project Scripting. Mark Ramm reprising his award-winning role as Kevin Dangoor BlueSkyOnMars.com. Python is not compiled. (actually, it is, but that’s not important right now). Python projects certainly do not need. Redistributable files to move around

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Smoothing Software Project Scripting

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Smoothing Software Project Scripting

  • Mark Ramm

  • reprising his award-winning role as

  • Kevin Dangoor

  • BlueSkyOnMars.com


Python is not compiled l.jpg

Python is not compiled

(actually, it is, but that’s not important right now)


Python projects certainly do not need l.jpg

Python projects certainly do not need

  • Redistributable files to move around

  • Deployment to other machines

  • Interaction with source control systems

  • Documentation that’s built from the source code

  • Documentation that’s built from a format other than its final display format


If only there was some scripting language that we already knew l.jpg

If only there was some scripting language that we already knew.


Waitasec isn t python a scripting language l.jpg

Waitasec. Isn’t Python a scripting language?


Project related scripts need l.jpg

Project-related scripts need...

  • Command line argument handling

  • Configuration

  • Often work with files

  • Sometimes need to use distutils/setuptools, but wish they could do a little more

  • Need to work with other common tools (Sphinx, svn, virtualenv)


Command line argument handling l.jpg

Command-line argument handling

  • bin/start-server --port 8675309


Configuration l.jpg

Configuration

  • [messages]

  • greeting=Hello

  • [thing1]

  • who=World

  • message=${messages.greeting}, ${who}


Lots of working with files l.jpg

Lots of working with files

  • import os

  • if not os.path.exists(“foo”):

  • os.mkdir(“foo”)

  • if not os.path.exists(os.path.join(“foo, “bar”)):

  • open(os.path.join(“foo”, “bar”), “w”).write(“Hi”)


Working with distutils setuptools l.jpg

Working with distutils/setuptools

  • #!/bin/sh

  • sphinx-build (blah blah blah)

  • python setup.py sdist upload


Other common tools l.jpg

Other common tools

  • sphinx-build ...

  • subprocess.Popen(“svn info”...

  • # and do something with the output

  • virtualenv.create_bootstrap_script(“# more code”)


Do you really want separate scripts for everything l.jpg

Do you really want separate scripts for everything?

  • /usr/local/bin/git/usr/local/bin/git-merge-resolve

  • /usr/local/bin/git-add/usr/local/bin/git-merge-stupid

  • /usr/local/bin/git-add--interactive/usr/local/bin/git-merge-subtree

  • /usr/local/bin/git-am/usr/local/bin/git-merge-tree

  • /usr/local/bin/git-annotate/usr/local/bin/git-mergetool

  • /usr/local/bin/git-apply/usr/local/bin/git-mktag

  • /usr/local/bin/git-archimport/usr/local/bin/git-mktree

  • /usr/local/bin/git-archive/usr/local/bin/git-mv

  • /usr/local/bin/git-bisect/usr/local/bin/git-name-rev

  • /usr/local/bin/git-blame/usr/local/bin/git-pack-objects

  • /usr/local/bin/git-branch/usr/local/bin/git-pack-redundant

  • /usr/local/bin/git-bundle/usr/local/bin/git-pack-refs

  • /usr/local/bin/git-cat-file/usr/local/bin/git-parse-remote

  • /usr/local/bin/git-check-attr/usr/local/bin/git-patch-id

  • /usr/local/bin/git-check-ref-format/usr/local/bin/git-peek-remote

  • /usr/local/bin/git-checkout/usr/local/bin/git-prune

  • /usr/local/bin/git-checkout-index/usr/local/bin/git-prune-packed

  • /usr/local/bin/git-cherry/usr/local/bin/git-pull

  • /usr/local/bin/git-cherry-pick/usr/local/bin/git-push

  • /usr/local/bin/git-citool/usr/local/bin/git-quiltimport

  • /usr/local/bin/git-clean/usr/local/bin/git-read-tree

  • /usr/local/bin/git-clone/usr/local/bin/git-rebase

  • /usr/local/bin/git-commit/usr/local/bin/git-rebase--interactive

  • /usr/local/bin/git-commit-tree/usr/local/bin/git-receive-pack

  • /usr/local/bin/git-config/usr/local/bin/git-reflog

  • /usr/local/bin/git-convert-objects/usr/local/bin/git-relink

  • /usr/local/bin/git-count-objects/usr/local/bin/git-remote

  • /usr/local/bin/git-cvsexportcommit/usr/local/bin/git-repack

  • /usr/local/bin/git-cvsimport/usr/local/bin/git-repo-config

  • /usr/local/bin/git-cvsserver/usr/local/bin/git-request-pull

  • /usr/local/bin/git-daemon/usr/local/bin/git-rerere

  • /usr/local/bin/git-describe/usr/local/bin/git-reset

  • /usr/local/bin/git-diff/usr/local/bin/git-rev-list

  • /usr/local/bin/git-diff-files/usr/local/bin/git-rev-parse

  • /usr/local/bin/git-diff-index/usr/local/bin/git-revert

  • /usr/local/bin/git-diff-tree/usr/local/bin/git-rm

  • /usr/local/bin/git-fast-import/usr/local/bin/git-runstatus

  • /usr/local/bin/git-fetch/usr/local/bin/git-send-email

  • /usr/local/bin/git-fetch--tool/usr/local/bin/git-send-pack

  • /usr/local/bin/git-fetch-pack/usr/local/bin/git-sh-setup

  • /usr/local/bin/git-filter-branch/usr/local/bin/git-shell

  • /usr/local/bin/git-fmt-merge-msg/usr/local/bin/git-shortlog

  • /usr/local/bin/git-for-each-ref/usr/local/bin/git-show

  • /usr/local/bin/git-format-patch/usr/local/bin/git-show-branch

  • /usr/local/bin/git-fsck/usr/local/bin/git-show-index

  • /usr/local/bin/git-fsck-objects/usr/local/bin/git-show-ref

  • /usr/local/bin/git-gc/usr/local/bin/git-ssh-fetch

  • /usr/local/bin/git-get-tar-commit-id/usr/local/bin/git-ssh-pull

  • /usr/local/bin/git-grep/usr/local/bin/git-ssh-push

  • /usr/local/bin/git-gui/usr/local/bin/git-ssh-upload

  • /usr/local/bin/git-hash-object/usr/local/bin/git-stash

  • /usr/local/bin/git-http-fetch/usr/local/bin/git-status

  • /usr/local/bin/git-imap-send/usr/local/bin/git-stripspace

  • /usr/local/bin/git-index-pack/usr/local/bin/git-submodule

  • /usr/local/bin/git-init/usr/local/bin/git-svn

  • /usr/local/bin/git-init-db/usr/local/bin/git-svnimport

  • /usr/local/bin/git-instaweb/usr/local/bin/git-symbolic-ref

  • /usr/local/bin/git-local-fetch/usr/local/bin/git-tag

  • /usr/local/bin/git-log/usr/local/bin/git-tar-tree

  • /usr/local/bin/git-lost-found/usr/local/bin/git-unpack-file

  • /usr/local/bin/git-ls-files/usr/local/bin/git-unpack-objects

  • /usr/local/bin/git-ls-remote/usr/local/bin/git-update-index

  • /usr/local/bin/git-ls-tree/usr/local/bin/git-update-ref

  • /usr/local/bin/git-mailinfo/usr/local/bin/git-update-server-info

  • /usr/local/bin/git-mailsplit/usr/local/bin/git-upload-archive

  • /usr/local/bin/git-merge/usr/local/bin/git-upload-pack

  • /usr/local/bin/git-merge-base/usr/local/bin/git-var

  • /usr/local/bin/git-merge-file/usr/local/bin/git-verify-pack

  • /usr/local/bin/git-merge-index/usr/local/bin/git-verify-tag

  • /usr/local/bin/git-merge-octopus/usr/local/bin/git-whatchanged

  • /usr/local/bin/git-merge-one-file/usr/local/bin/git-write-tree

  • /usr/local/bin/git-merge-ours/usr/local/bin/gitk

  • /usr/local/bin/git-merge-recursive


Easy install paver l.jpg

easy_install Paver


Paver paverdocs l.jpg

paver paverdocs


Pavement py l.jpg

pavement.py


Why python build files l.jpg

Why Python build files?

  • You already know Python

  • The language rules are well-defined

  • The language rules are well-documented

  • Python is powerful, so you’ll never be left hanging or need an escape hatch


Configuration19 l.jpg

Configuration

  • options(

  • setup = setup_meta,

  • minilib=Bunch(

  • extra_files=['doctools', 'virtual']

  • ),

  • sphinx=Bunch(

  • builddir="build",

  • sourcedir="source"

  • ),

  • virtualenv=Bunch(

  • packages_to_install=["nose", "sphinx", "docutils", "virtualenv"],

  • install_paver=False,

  • script_name='bootstrap.py',

  • paver_command_line=None

  • )

  • )


Dynamic config values l.jpg

Dynamic config values

  • >>> from paver.defaults import *

  • >>> import time

  • >>> options(current=lambda: time.time())

  • >>> options.current

  • 1216726815.0027969


Namespace searching l.jpg

Namespace searching

  • >>> options(

  • ... setup=Bunch(version="1.0"),

  • ... sphinx=Bunch(builddir="docbuild")

  • ... )

  • >>> options.version

  • '1.0'


Namespace searching continued l.jpg

Namespace searching (continued)

  • >>> options(

  • ... setup=Bunch(version="1.0"),

  • ... sphinx=Bunch(builddir="docbuild")

  • ... )

  • >>> options.order('sphinx')

  • >>> options.version

  • Traceback (most recent call last):

  • File "<stdin>", line 1, in <module>

  • File "/Users/admin/projects/paver/paver/runtime.py", line 31, in __getattr__

  • raise AttributeError(name)

  • AttributeError: version


Configuration is still standard python l.jpg

Configuration is still standard Python

  • You can treat options like a normal, nested dictionary

  • The only unusual thing would be that callables are called.


Tasks l.jpg

Tasks

  • @task

  • def clean():

  • """Cleans up this paver directory. Removes the virtualenv traces and

  • the build directory."""

  • pass


Paver help l.jpg

paver help

  • paver help

  • ---> paver.tasks.help

  • Usage: paver [global options] taskname [task options] [taskname [taskoptions]]

  • Options:

  • --version show program's version number and exit

  • -n, --dry-run don't actually do anything

  • -v, --verbose display all logging output

  • -q, --quiet display only errors

  • -i, --interactive enable prompting

  • -f FILE, --file=FILE read tasks from FILE [pavement.py]

  • -h, --help display this help information

  • Tasks from paver.misctasks:

  • generate_setup - Generates a setup


Paver help taskname l.jpg

paver help <taskname>

  • Usage: paver paver.misctasks.minilib [options]

  • Options:

  • -h, --help display this help information

  • Create a Paver mini library that contains enough for a simple

  • pavement.py to be installed using a generated setup.py. This

  • is a good temporary measure until more people have deployed paver.

  • The output file is 'paver-minilib.zip' in the current directory.

  • Options:

  • extra_files

  • list of other paver modules to include (don't include the .py

  • extension). By default, the following modules are included:

  • defaults, path, release, setuputils, misctasks, options,

  • tasks, easy


@needs l.jpg

@needs

  • @task

  • @needs("uncog")

  • def commit():

  • """Removes the generated code from the docs and then commits to bzr."""

  • pass


@cmdopts l.jpg

@cmdopts

  • @task

  • @cmdopts([("username=", "u", "Username for remote server"),

  • ("server=", "s", "Server to deploy to")])

  • def deploy():

  • """Copy the Paver website up."""

    • pass


Paver taskname l.jpg

paver <taskname>

  • $ paver generate_setup minilib

  • ---> generate_setup

  • Write setup.py

  • ---> minilib

  • Generate paver-minilib.zip


Paver and distutils setuptools l.jpg

Paver and Distutils/Setuptools


Python setup py install l.jpg

python setup.py install


Paver embraces and extends distutils l.jpg

Paver embraces and extends Distutils

(don’t worry, it’s not evil)


Paver install l.jpg

paver install


Setup py example l.jpg

setup.py example

  • from distutils.core import setup

  • setup(name='foo', version='1.0', py_modules=['foo'], )


Upgrading to paver l.jpg

Upgrading to Paver

  • from paver.setuputils import setupsetup(name='foo', version='1.0', py_modules=['foo'], )


Keeping it simple for users l.jpg

Keeping it simple for users

  • $ paver generate_setup minilib

  • ---> generate_setup

  • Write setup.py

  • ---> minilib

  • Generate paver-minilib.zip


Slide38 l.jpg

The Paver Standard Library

(is actually newer and less musty)


The paver standard library l.jpg

The Paver Standard Library

  • Easy scripting (paver.easy)

  • Distutils integration (paver.setuputils)

  • File handling (paver.path)

  • Documentation tools (paver.doctools)

  • Subversion (paver.svn)

  • SSH (paver.ssh)

  • Virtualenv (paver.virtual)

  • Miscellaneous Tasks (paver.misctasks)


Paver easy l.jpg

paver.easy

  • from paver.easy import *

  • # display text if verbose is set

  • debug(“Hi there. Feeling chatty today?”)

  • # display text if quiet is not set

  • info(“Glad we don’t have to keep quiet”)

  • # display text regardless of setting

  • error(“HA! I’M SHOUTING AND YOU CAN’T STOP ME!”)


Paver easy continued l.jpg

paver.easy (continued)

  • # run a command, as long as dry-run is off

  • # capture the output into myval

  • myval = sh(“cat /tmp/foo”, capture=True)

  • # run a function (delete_all(‘/’) to be exact).

  • # if dry-run is set, then

  • # just print the message instead

  • dry(“Delete everything”, delete_all, ‘/’)


Paver path l.jpg

paver.path

  • Jason Orendorff’s path.py module (also available separately)

  • It’s a subclass of string!

  • Fun use of operator overloading

  • Lots of great methods

  • Makes working with files/directories fun and easy


Paver path examples l.jpg

paver.path examples

  • p = path("docs")tmpdir = p /"tmp"tmpdir.mkdir()fn = tmpdir /"myfile.txt"fn.write_text("Hi there!")


Paver doctools l.jpg

paver.doctools

  • # add this to use

  • import paver.doctools


Paver doctools sphinx l.jpg

paver.doctools – Sphinx

  • paver.doctools.html()

  • Build HTML documentation using Sphinx. This uses the following options in a “sphinx” section of the options.

  • docroot

  • the root under which Sphinx will be working. Default: docs

  • builddir

  • directory under the docroot where the resulting files are put. default: build

  • sourcedir

  • directory under the docroot for the source files default: (empty string)

  • paver.doctools.doc_clean()¶

  • Clean (delete) the built docs. Specifically, this deletes the build directory under the docroot. See the html task for the options list.


Paver doctools cog sectionedfile l.jpg

paver.doctools – Cog & SectionedFile

  • Solutions to common problems of creating high-quality docs

  • Ideally, your code samples will be in convenient runnable code files and have unit tests.

  • But you also want nice, minimal code samples in your text as you’re writing.

  • Ned Batchelder’s Cog (included with Paver) and paver.doctools.SectionedFile make these easy.

  • Not Sphinx-specific at all


Paver doctools sample code l.jpg

paver.doctools sample code

  • # mysample.py# [[[section mysample]]]defsample_func():print "To sample, or not to sample?"# [[[endsection]]]


Paver doctools in docs l.jpg

paver.doctools in docs

  • And then when you want to print the sample string, you just call::

  • # [[[cog include(“code/mysample.py”, “mysample”)]]]

  • # [[[end]]]


Paver doctools in docs 2 l.jpg

paver.doctools in docs (2)

  • And then when you want to print the sample string, you just call::

  • # [[[cog include(“code/mysample.py”, “mysample”)]]]

  • def sample_func():

  • print "To sample, or not to sample?"

  • # [[[end]]]


Paver doctools uncog before commit l.jpg

paver.doctools uncog before commit

  • @task

  • @needs("uncog")

  • def commit():

  • """Removes the generated code from the docs and then commits to bzr."""

  • sh("bzr commit")


Pulling it all together l.jpg

Pulling it all together

  • write functions with the @task decorator.

  • Just plain old Python with lots of conveniences

  • Good example in projects like Bespin, PyMOTW

  • Even better, take a look at “Getting Started with Paver”http://bit.ly/starting_paver


Remember this l.jpg

Remember this?

  • #!/bin/sh

  • sphinx-build (blah blah blah)

  • python setup.py sdist upload


Paver needs to do that too l.jpg

Paver needs to do that too

  • @task

  • @needs(['html', "minilib", "generate_setup", “setuptools.command.sdist”])

  • def sdist():

  • """Builds the documentation and the tarball."""

  • pass


But wait it does more l.jpg

But wait, it does more!

  • @task

  • @needs(['cog', 'paver.doctools.html'])

  • def html():

  • """Build Paver's documentation and install it into paver/docs"""

  • builtdocs = path("docs") / options.sphinx.builddir / "html"

  • destdir = path("paver") / "docs"

  • destdir.rmtree()

  • builtdocs.move(destdir)


Credits l.jpg

Credits

  • Thanks to SitePen for supporting Paver

  • Bumpy road http://flickr.com/photos/ilya/442034/sizes/o/

  • Smooth roadhttp://flickr.com/photos/nicholas_t/317528561/

  • Taskshttp://flickr.com/photos/mpwillis/470557535/


Http www blueskyonmars com projects paver l.jpg

http://www.blueskyonmars.com/projects/paver/


  • Login