Smoothing Software Project Scripting
Download
1 / 56

mark rammreprising his award-winning role askevin dangoorblueskyonmars.com - PowerPoint PPT Presentation


  • 314 Views
  • Uploaded on

Smoothing Software Project Scripting. Mark Ramm reprising his award-winning role as Kevin Dangoor BlueSkyOnMars.com. Python is not compiled. (actually, it is, but that’s not important right now). Python projects certainly do not need. Redistributable files to move around

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'mark rammreprising his award-winning role askevin dangoorblueskyonmars.com' - Angelica


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Smoothing Software Project Scripting

  • Mark Ramm

  • reprising his award-winning role as

  • Kevin Dangoor

  • BlueSkyOnMars.com


Python is not compiled l.jpg
Python is not compiled

(actually, it is, but that’s not important right now)


Python projects certainly do not need l.jpg
Python projects certainly do not need

  • Redistributable files to move around

  • Deployment to other machines

  • Interaction with source control systems

  • Documentation that’s built from the source code

  • Documentation that’s built from a format other than its final display format




Project related scripts need l.jpg
Project-related scripts need... knew.

  • Command line argument handling

  • Configuration

  • Often work with files

  • Sometimes need to use distutils/setuptools, but wish they could do a little more

  • Need to work with other common tools (Sphinx, svn, virtualenv)


Command line argument handling l.jpg
Command-line argument handling knew.

  • bin/start-server --port 8675309


Configuration l.jpg
Configuration knew.

  • [messages]

  • greeting=Hello

  • [thing1]

  • who=World

  • message=${messages.greeting}, ${who}


Lots of working with files l.jpg
Lots of working with files knew.

  • import os

  • if not os.path.exists(“foo”):

  • os.mkdir(“foo”)

  • if not os.path.exists(os.path.join(“foo, “bar”)):

  • open(os.path.join(“foo”, “bar”), “w”).write(“Hi”)


Working with distutils setuptools l.jpg
Working with distutils/setuptools knew.

  • #!/bin/sh

  • sphinx-build (blah blah blah)

  • python setup.py sdist upload


Other common tools l.jpg
Other common tools knew.

  • sphinx-build ...

  • subprocess.Popen(“svn info”...

  • # and do something with the output

  • virtualenv.create_bootstrap_script(“# more code”)


Do you really want separate scripts for everything l.jpg
Do you really want separate scripts for everything? knew.

  • /usr/local/bin/git /usr/local/bin/git-merge-resolve

  • /usr/local/bin/git-add /usr/local/bin/git-merge-stupid

  • /usr/local/bin/git-add--interactive /usr/local/bin/git-merge-subtree

  • /usr/local/bin/git-am /usr/local/bin/git-merge-tree

  • /usr/local/bin/git-annotate /usr/local/bin/git-mergetool

  • /usr/local/bin/git-apply /usr/local/bin/git-mktag

  • /usr/local/bin/git-archimport /usr/local/bin/git-mktree

  • /usr/local/bin/git-archive /usr/local/bin/git-mv

  • /usr/local/bin/git-bisect /usr/local/bin/git-name-rev

  • /usr/local/bin/git-blame /usr/local/bin/git-pack-objects

  • /usr/local/bin/git-branch /usr/local/bin/git-pack-redundant

  • /usr/local/bin/git-bundle /usr/local/bin/git-pack-refs

  • /usr/local/bin/git-cat-file /usr/local/bin/git-parse-remote

  • /usr/local/bin/git-check-attr /usr/local/bin/git-patch-id

  • /usr/local/bin/git-check-ref-format /usr/local/bin/git-peek-remote

  • /usr/local/bin/git-checkout /usr/local/bin/git-prune

  • /usr/local/bin/git-checkout-index /usr/local/bin/git-prune-packed

  • /usr/local/bin/git-cherry /usr/local/bin/git-pull

  • /usr/local/bin/git-cherry-pick /usr/local/bin/git-push

  • /usr/local/bin/git-citool /usr/local/bin/git-quiltimport

  • /usr/local/bin/git-clean /usr/local/bin/git-read-tree

  • /usr/local/bin/git-clone /usr/local/bin/git-rebase

  • /usr/local/bin/git-commit /usr/local/bin/git-rebase--interactive

  • /usr/local/bin/git-commit-tree /usr/local/bin/git-receive-pack

  • /usr/local/bin/git-config /usr/local/bin/git-reflog

  • /usr/local/bin/git-convert-objects /usr/local/bin/git-relink

  • /usr/local/bin/git-count-objects /usr/local/bin/git-remote

  • /usr/local/bin/git-cvsexportcommit /usr/local/bin/git-repack

  • /usr/local/bin/git-cvsimport /usr/local/bin/git-repo-config

  • /usr/local/bin/git-cvsserver /usr/local/bin/git-request-pull

  • /usr/local/bin/git-daemon /usr/local/bin/git-rerere

  • /usr/local/bin/git-describe /usr/local/bin/git-reset

  • /usr/local/bin/git-diff /usr/local/bin/git-rev-list

  • /usr/local/bin/git-diff-files /usr/local/bin/git-rev-parse

  • /usr/local/bin/git-diff-index /usr/local/bin/git-revert

  • /usr/local/bin/git-diff-tree /usr/local/bin/git-rm

  • /usr/local/bin/git-fast-import /usr/local/bin/git-runstatus

  • /usr/local/bin/git-fetch /usr/local/bin/git-send-email

  • /usr/local/bin/git-fetch--tool /usr/local/bin/git-send-pack

  • /usr/local/bin/git-fetch-pack /usr/local/bin/git-sh-setup

  • /usr/local/bin/git-filter-branch /usr/local/bin/git-shell

  • /usr/local/bin/git-fmt-merge-msg /usr/local/bin/git-shortlog

  • /usr/local/bin/git-for-each-ref /usr/local/bin/git-show

  • /usr/local/bin/git-format-patch /usr/local/bin/git-show-branch

  • /usr/local/bin/git-fsck /usr/local/bin/git-show-index

  • /usr/local/bin/git-fsck-objects /usr/local/bin/git-show-ref

  • /usr/local/bin/git-gc /usr/local/bin/git-ssh-fetch

  • /usr/local/bin/git-get-tar-commit-id /usr/local/bin/git-ssh-pull

  • /usr/local/bin/git-grep /usr/local/bin/git-ssh-push

  • /usr/local/bin/git-gui /usr/local/bin/git-ssh-upload

  • /usr/local/bin/git-hash-object /usr/local/bin/git-stash

  • /usr/local/bin/git-http-fetch /usr/local/bin/git-status

  • /usr/local/bin/git-imap-send /usr/local/bin/git-stripspace

  • /usr/local/bin/git-index-pack /usr/local/bin/git-submodule

  • /usr/local/bin/git-init /usr/local/bin/git-svn

  • /usr/local/bin/git-init-db /usr/local/bin/git-svnimport

  • /usr/local/bin/git-instaweb /usr/local/bin/git-symbolic-ref

  • /usr/local/bin/git-local-fetch /usr/local/bin/git-tag

  • /usr/local/bin/git-log /usr/local/bin/git-tar-tree

  • /usr/local/bin/git-lost-found /usr/local/bin/git-unpack-file

  • /usr/local/bin/git-ls-files /usr/local/bin/git-unpack-objects

  • /usr/local/bin/git-ls-remote /usr/local/bin/git-update-index

  • /usr/local/bin/git-ls-tree /usr/local/bin/git-update-ref

  • /usr/local/bin/git-mailinfo /usr/local/bin/git-update-server-info

  • /usr/local/bin/git-mailsplit /usr/local/bin/git-upload-archive

  • /usr/local/bin/git-merge /usr/local/bin/git-upload-pack

  • /usr/local/bin/git-merge-base /usr/local/bin/git-var

  • /usr/local/bin/git-merge-file /usr/local/bin/git-verify-pack

  • /usr/local/bin/git-merge-index /usr/local/bin/git-verify-tag

  • /usr/local/bin/git-merge-octopus /usr/local/bin/git-whatchanged

  • /usr/local/bin/git-merge-one-file /usr/local/bin/git-write-tree

  • /usr/local/bin/git-merge-ours /usr/local/bin/gitk

  • /usr/local/bin/git-merge-recursive





Why python build files l.jpg
Why Python build files? knew.

  • You already know Python

  • The language rules are well-defined

  • The language rules are well-documented

  • Python is powerful, so you’ll never be left hanging or need an escape hatch


Configuration19 l.jpg
Configuration knew.

  • options(

  • setup = setup_meta,

  • minilib=Bunch(

  • extra_files=['doctools', 'virtual']

  • ),

  • sphinx=Bunch(

  • builddir="build",

  • sourcedir="source"

  • ),

  • virtualenv=Bunch(

  • packages_to_install=["nose", "sphinx", "docutils", "virtualenv"],

  • install_paver=False,

  • script_name='bootstrap.py',

  • paver_command_line=None

  • )

  • )


Dynamic config values l.jpg
Dynamic config values knew.

  • >>> from paver.defaults import *

  • >>> import time

  • >>> options(current=lambda: time.time())

  • >>> options.current

  • 1216726815.0027969


Namespace searching l.jpg
Namespace searching knew.

  • >>> options(

  • ... setup=Bunch(version="1.0"),

  • ... sphinx=Bunch(builddir="docbuild")

  • ... )

  • >>> options.version

  • '1.0'


Namespace searching continued l.jpg
Namespace searching (continued) knew.

  • >>> options(

  • ... setup=Bunch(version="1.0"),

  • ... sphinx=Bunch(builddir="docbuild")

  • ... )

  • >>> options.order('sphinx')

  • >>> options.version

  • Traceback (most recent call last):

  • File "<stdin>", line 1, in <module>

  • File "/Users/admin/projects/paver/paver/runtime.py", line 31, in __getattr__

  • raise AttributeError(name)

  • AttributeError: version


Configuration is still standard python l.jpg
Configuration is still standard Python knew.

  • You can treat options like a normal, nested dictionary

  • The only unusual thing would be that callables are called.


Tasks l.jpg
Tasks knew.

  • @task

  • def clean():

  • """Cleans up this paver directory. Removes the virtualenv traces and

  • the build directory."""

  • pass


Paver help l.jpg
paver help knew.

  • paver help

  • ---> paver.tasks.help

  • Usage: paver [global options] taskname [task options] [taskname [taskoptions]]

  • Options:

  • --version show program's version number and exit

  • -n, --dry-run don't actually do anything

  • -v, --verbose display all logging output

  • -q, --quiet display only errors

  • -i, --interactive enable prompting

  • -f FILE, --file=FILE read tasks from FILE [pavement.py]

  • -h, --help display this help information

  • Tasks from paver.misctasks:

  • generate_setup - Generates a setup


Paver help taskname l.jpg
paver help <taskname> knew.

  • Usage: paver paver.misctasks.minilib [options]

  • Options:

  • -h, --help display this help information

  • Create a Paver mini library that contains enough for a simple

  • pavement.py to be installed using a generated setup.py. This

  • is a good temporary measure until more people have deployed paver.

  • The output file is 'paver-minilib.zip' in the current directory.

  • Options:

  • extra_files

  • list of other paver modules to include (don't include the .py

  • extension). By default, the following modules are included:

  • defaults, path, release, setuputils, misctasks, options,

  • tasks, easy


@needs l.jpg
@needs knew.

  • @task

  • @needs("uncog")

  • def commit():

  • """Removes the generated code from the docs and then commits to bzr."""

  • pass


@cmdopts l.jpg
@cmdopts knew.

  • @task

  • @cmdopts([("username=", "u", "Username for remote server"),

  • ("server=", "s", "Server to deploy to")])

  • def deploy():

  • """Copy the Paver website up."""

    • pass


Paver taskname l.jpg
paver <taskname> knew.

  • $ paver generate_setup minilib

  • ---> generate_setup

  • Write setup.py

  • ---> minilib

  • Generate paver-minilib.zip




Paver embraces and extends distutils l.jpg
Paver embraces and extends Distutils knew.

(don’t worry, it’s not evil)



Setup py example l.jpg
setup.py example knew.

  • from distutils.core import setup

  • setup(name='foo', version='1.0', py_modules=['foo'], )


Upgrading to paver l.jpg
Upgrading to Paver knew.

  • from paver.setuputils import setupsetup(name='foo', version='1.0', py_modules=['foo'], )


Keeping it simple for users l.jpg
Keeping it simple for users knew.

  • $ paver generate_setup minilib

  • ---> generate_setup

  • Write setup.py

  • ---> minilib

  • Generate paver-minilib.zip


Slide38 l.jpg

The Paver Standard Library knew.

(is actually newer and less musty)


The paver standard library l.jpg
The Paver Standard Library knew.

  • Easy scripting (paver.easy)

  • Distutils integration (paver.setuputils)

  • File handling (paver.path)

  • Documentation tools (paver.doctools)

  • Subversion (paver.svn)

  • SSH (paver.ssh)

  • Virtualenv (paver.virtual)

  • Miscellaneous Tasks (paver.misctasks)


Paver easy l.jpg
paver.easy knew.

  • from paver.easy import *

  • # display text if verbose is set

  • debug(“Hi there. Feeling chatty today?”)

  • # display text if quiet is not set

  • info(“Glad we don’t have to keep quiet”)

  • # display text regardless of setting

  • error(“HA! I’M SHOUTING AND YOU CAN’T STOP ME!”)


Paver easy continued l.jpg
paver.easy (continued) knew.

  • # run a command, as long as dry-run is off

  • # capture the output into myval

  • myval = sh(“cat /tmp/foo”, capture=True)

  • # run a function (delete_all(‘/’) to be exact).

  • # if dry-run is set, then

  • # just print the message instead

  • dry(“Delete everything”, delete_all, ‘/’)


Paver path l.jpg
paver.path knew.

  • Jason Orendorff’s path.py module (also available separately)

  • It’s a subclass of string!

  • Fun use of operator overloading

  • Lots of great methods

  • Makes working with files/directories fun and easy


Paver path examples l.jpg
paver.path examples knew.

  • p = path("docs")tmpdir = p /"tmp"tmpdir.mkdir()fn = tmpdir /"myfile.txt"fn.write_text("Hi there!")


Paver doctools l.jpg
paver.doctools knew.

  • # add this to use

  • import paver.doctools


Paver doctools sphinx l.jpg
paver.doctools – Sphinx knew.

  • paver.doctools.html()

  • Build HTML documentation using Sphinx. This uses the following options in a “sphinx” section of the options.

  • docroot

  • the root under which Sphinx will be working. Default: docs

  • builddir

  • directory under the docroot where the resulting files are put. default: build

  • sourcedir

  • directory under the docroot for the source files default: (empty string)

  • paver.doctools.doc_clean()¶

  • Clean (delete) the built docs. Specifically, this deletes the build directory under the docroot. See the html task for the options list.


Paver doctools cog sectionedfile l.jpg
paver.doctools – Cog & SectionedFile knew.

  • Solutions to common problems of creating high-quality docs

  • Ideally, your code samples will be in convenient runnable code files and have unit tests.

  • But you also want nice, minimal code samples in your text as you’re writing.

  • Ned Batchelder’s Cog (included with Paver) and paver.doctools.SectionedFile make these easy.

  • Not Sphinx-specific at all


Paver doctools sample code l.jpg
paver.doctools sample code knew.

  • # mysample.py# [[[section mysample]]]defsample_func():print "To sample, or not to sample?"# [[[endsection]]]


Paver doctools in docs l.jpg
paver.doctools in docs knew.

  • And then when you want to print the sample string, you just call::

  • # [[[cog include(“code/mysample.py”, “mysample”)]]]

  • # [[[end]]]


Paver doctools in docs 2 l.jpg
paver.doctools in docs (2) knew.

  • And then when you want to print the sample string, you just call::

  • # [[[cog include(“code/mysample.py”, “mysample”)]]]

  • def sample_func():

  • print "To sample, or not to sample?"

  • # [[[end]]]


Paver doctools uncog before commit l.jpg
paver.doctools uncog before commit knew.

  • @task

  • @needs("uncog")

  • def commit():

  • """Removes the generated code from the docs and then commits to bzr."""

  • sh("bzr commit")


Pulling it all together l.jpg
Pulling it all together knew.

  • write functions with the @task decorator.

  • Just plain old Python with lots of conveniences

  • Good example in projects like Bespin, PyMOTW

  • Even better, take a look at “Getting Started with Paver”http://bit.ly/starting_paver


Remember this l.jpg
Remember this? knew.

  • #!/bin/sh

  • sphinx-build (blah blah blah)

  • python setup.py sdist upload


Paver needs to do that too l.jpg
Paver needs to do that too knew.

  • @task

  • @needs(['html', "minilib", "generate_setup", “setuptools.command.sdist”])

  • def sdist():

  • """Builds the documentation and the tarball."""

  • pass


But wait it does more l.jpg
But wait, it does more! knew.

  • @task

  • @needs(['cog', 'paver.doctools.html'])

  • def html():

  • """Build Paver's documentation and install it into paver/docs"""

  • builtdocs = path("docs") / options.sphinx.builddir / "html"

  • destdir = path("paver") / "docs"

  • destdir.rmtree()

  • builtdocs.move(destdir)


Credits l.jpg
Credits knew.

  • Thanks to SitePen for supporting Paver

  • Bumpy road http://flickr.com/photos/ilya/442034/sizes/o/

  • Smooth roadhttp://flickr.com/photos/nicholas_t/317528561/

  • Taskshttp://flickr.com/photos/mpwillis/470557535/



ad