slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Smoothing Software Project Scripting PowerPoint Presentation
Download Presentation
Smoothing Software Project Scripting

Loading in 2 Seconds...

play fullscreen
1 / 56

Smoothing Software Project Scripting - PowerPoint PPT Presentation


  • 316 Views
  • Uploaded on

Smoothing Software Project Scripting. Mark Ramm reprising his award-winning role as Kevin Dangoor BlueSkyOnMars.com. Python is not compiled. (actually, it is, but that’s not important right now). Python projects certainly do not need. Redistributable files to move around

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Smoothing Software Project Scripting' - Angelica


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Smoothing Software Project Scripting

  • Mark Ramm
  • reprising his award-winning role as
  • Kevin Dangoor
  • BlueSkyOnMars.com
python is not compiled
Python is not compiled

(actually, it is, but that’s not important right now)

python projects certainly do not need
Python projects certainly do not need
  • Redistributable files to move around
  • Deployment to other machines
  • Interaction with source control systems
  • Documentation that’s built from the source code
  • Documentation that’s built from a format other than its final display format
project related scripts need
Project-related scripts need...
  • Command line argument handling
  • Configuration
  • Often work with files
  • Sometimes need to use distutils/setuptools, but wish they could do a little more
  • Need to work with other common tools (Sphinx, svn, virtualenv)
command line argument handling
Command-line argument handling
  • bin/start-server --port 8675309
configuration
Configuration
  • [messages]
  • greeting=Hello
  • [thing1]
  • who=World
  • message=${messages.greeting}, ${who}
lots of working with files
Lots of working with files
  • import os
  • if not os.path.exists(“foo”):
  • os.mkdir(“foo”)
  • if not os.path.exists(os.path.join(“foo, “bar”)):
  • open(os.path.join(“foo”, “bar”), “w”).write(“Hi”)
working with distutils setuptools
Working with distutils/setuptools
  • #!/bin/sh
  • sphinx-build (blah blah blah)
  • python setup.py sdist upload
other common tools
Other common tools
  • sphinx-build ...
  • subprocess.Popen(“svn info”...
  • # and do something with the output
  • virtualenv.create_bootstrap_script(“# more code”)
do you really want separate scripts for everything
Do you really want separate scripts for everything?
  • /usr/local/bin/git /usr/local/bin/git-merge-resolve
  • /usr/local/bin/git-add /usr/local/bin/git-merge-stupid
  • /usr/local/bin/git-add--interactive /usr/local/bin/git-merge-subtree
  • /usr/local/bin/git-am /usr/local/bin/git-merge-tree
  • /usr/local/bin/git-annotate /usr/local/bin/git-mergetool
  • /usr/local/bin/git-apply /usr/local/bin/git-mktag
  • /usr/local/bin/git-archimport /usr/local/bin/git-mktree
  • /usr/local/bin/git-archive /usr/local/bin/git-mv
  • /usr/local/bin/git-bisect /usr/local/bin/git-name-rev
  • /usr/local/bin/git-blame /usr/local/bin/git-pack-objects
  • /usr/local/bin/git-branch /usr/local/bin/git-pack-redundant
  • /usr/local/bin/git-bundle /usr/local/bin/git-pack-refs
  • /usr/local/bin/git-cat-file /usr/local/bin/git-parse-remote
  • /usr/local/bin/git-check-attr /usr/local/bin/git-patch-id
  • /usr/local/bin/git-check-ref-format /usr/local/bin/git-peek-remote
  • /usr/local/bin/git-checkout /usr/local/bin/git-prune
  • /usr/local/bin/git-checkout-index /usr/local/bin/git-prune-packed
  • /usr/local/bin/git-cherry /usr/local/bin/git-pull
  • /usr/local/bin/git-cherry-pick /usr/local/bin/git-push
  • /usr/local/bin/git-citool /usr/local/bin/git-quiltimport
  • /usr/local/bin/git-clean /usr/local/bin/git-read-tree
  • /usr/local/bin/git-clone /usr/local/bin/git-rebase
  • /usr/local/bin/git-commit /usr/local/bin/git-rebase--interactive
  • /usr/local/bin/git-commit-tree /usr/local/bin/git-receive-pack
  • /usr/local/bin/git-config /usr/local/bin/git-reflog
  • /usr/local/bin/git-convert-objects /usr/local/bin/git-relink
  • /usr/local/bin/git-count-objects /usr/local/bin/git-remote
  • /usr/local/bin/git-cvsexportcommit /usr/local/bin/git-repack
  • /usr/local/bin/git-cvsimport /usr/local/bin/git-repo-config
  • /usr/local/bin/git-cvsserver /usr/local/bin/git-request-pull
  • /usr/local/bin/git-daemon /usr/local/bin/git-rerere
  • /usr/local/bin/git-describe /usr/local/bin/git-reset
  • /usr/local/bin/git-diff /usr/local/bin/git-rev-list
  • /usr/local/bin/git-diff-files /usr/local/bin/git-rev-parse
  • /usr/local/bin/git-diff-index /usr/local/bin/git-revert
  • /usr/local/bin/git-diff-tree /usr/local/bin/git-rm
  • /usr/local/bin/git-fast-import /usr/local/bin/git-runstatus
  • /usr/local/bin/git-fetch /usr/local/bin/git-send-email
  • /usr/local/bin/git-fetch--tool /usr/local/bin/git-send-pack
  • /usr/local/bin/git-fetch-pack /usr/local/bin/git-sh-setup
  • /usr/local/bin/git-filter-branch /usr/local/bin/git-shell
  • /usr/local/bin/git-fmt-merge-msg /usr/local/bin/git-shortlog
  • /usr/local/bin/git-for-each-ref /usr/local/bin/git-show
  • /usr/local/bin/git-format-patch /usr/local/bin/git-show-branch
  • /usr/local/bin/git-fsck /usr/local/bin/git-show-index
  • /usr/local/bin/git-fsck-objects /usr/local/bin/git-show-ref
  • /usr/local/bin/git-gc /usr/local/bin/git-ssh-fetch
  • /usr/local/bin/git-get-tar-commit-id /usr/local/bin/git-ssh-pull
  • /usr/local/bin/git-grep /usr/local/bin/git-ssh-push
  • /usr/local/bin/git-gui /usr/local/bin/git-ssh-upload
  • /usr/local/bin/git-hash-object /usr/local/bin/git-stash
  • /usr/local/bin/git-http-fetch /usr/local/bin/git-status
  • /usr/local/bin/git-imap-send /usr/local/bin/git-stripspace
  • /usr/local/bin/git-index-pack /usr/local/bin/git-submodule
  • /usr/local/bin/git-init /usr/local/bin/git-svn
  • /usr/local/bin/git-init-db /usr/local/bin/git-svnimport
  • /usr/local/bin/git-instaweb /usr/local/bin/git-symbolic-ref
  • /usr/local/bin/git-local-fetch /usr/local/bin/git-tag
  • /usr/local/bin/git-log /usr/local/bin/git-tar-tree
  • /usr/local/bin/git-lost-found /usr/local/bin/git-unpack-file
  • /usr/local/bin/git-ls-files /usr/local/bin/git-unpack-objects
  • /usr/local/bin/git-ls-remote /usr/local/bin/git-update-index
  • /usr/local/bin/git-ls-tree /usr/local/bin/git-update-ref
  • /usr/local/bin/git-mailinfo /usr/local/bin/git-update-server-info
  • /usr/local/bin/git-mailsplit /usr/local/bin/git-upload-archive
  • /usr/local/bin/git-merge /usr/local/bin/git-upload-pack
  • /usr/local/bin/git-merge-base /usr/local/bin/git-var
  • /usr/local/bin/git-merge-file /usr/local/bin/git-verify-pack
  • /usr/local/bin/git-merge-index /usr/local/bin/git-verify-tag
  • /usr/local/bin/git-merge-octopus /usr/local/bin/git-whatchanged
  • /usr/local/bin/git-merge-one-file /usr/local/bin/git-write-tree
  • /usr/local/bin/git-merge-ours /usr/local/bin/gitk
  • /usr/local/bin/git-merge-recursive
why python build files
Why Python build files?
  • You already know Python
  • The language rules are well-defined
  • The language rules are well-documented
  • Python is powerful, so you’ll never be left hanging or need an escape hatch
configuration19
Configuration
  • options(
  • setup = setup_meta,
  • minilib=Bunch(
  • extra_files=['doctools', 'virtual']
  • ),
  • sphinx=Bunch(
  • builddir="build",
  • sourcedir="source"
  • ),
  • virtualenv=Bunch(
  • packages_to_install=["nose", "sphinx", "docutils", "virtualenv"],
  • install_paver=False,
  • script_name='bootstrap.py',
  • paver_command_line=None
  • )
  • )
dynamic config values
Dynamic config values
  • >>> from paver.defaults import *
  • >>> import time
  • >>> options(current=lambda: time.time())
  • >>> options.current
  • 1216726815.0027969
namespace searching
Namespace searching
  • >>> options(
  • ... setup=Bunch(version="1.0"),
  • ... sphinx=Bunch(builddir="docbuild")
  • ... )
  • >>> options.version
  • '1.0'
namespace searching continued
Namespace searching (continued)
  • >>> options(
  • ... setup=Bunch(version="1.0"),
  • ... sphinx=Bunch(builddir="docbuild")
  • ... )
  • >>> options.order('sphinx')
  • >>> options.version
  • Traceback (most recent call last):
  • File "<stdin>", line 1, in <module>
  • File "/Users/admin/projects/paver/paver/runtime.py", line 31, in __getattr__
  • raise AttributeError(name)
  • AttributeError: version
configuration is still standard python
Configuration is still standard Python
  • You can treat options like a normal, nested dictionary
  • The only unusual thing would be that callables are called.
tasks
Tasks
  • @task
  • def clean():
  • """Cleans up this paver directory. Removes the virtualenv traces and
  • the build directory."""
  • pass
paver help
paver help
  • paver help
  • ---> paver.tasks.help
  • Usage: paver [global options] taskname [task options] [taskname [taskoptions]]
  • Options:
  • --version show program's version number and exit
  • -n, --dry-run don't actually do anything
  • -v, --verbose display all logging output
  • -q, --quiet display only errors
  • -i, --interactive enable prompting
  • -f FILE, --file=FILE read tasks from FILE [pavement.py]
  • -h, --help display this help information
  • Tasks from paver.misctasks:
  • generate_setup - Generates a setup
paver help taskname
paver help <taskname>
  • Usage: paver paver.misctasks.minilib [options]
  • Options:
  • -h, --help display this help information
  • Create a Paver mini library that contains enough for a simple
  • pavement.py to be installed using a generated setup.py. This
  • is a good temporary measure until more people have deployed paver.
  • The output file is 'paver-minilib.zip' in the current directory.
  • Options:
  • extra_files
  • list of other paver modules to include (don't include the .py
  • extension). By default, the following modules are included:
  • defaults, path, release, setuputils, misctasks, options,
  • tasks, easy
@needs
@needs
  • @task
  • @needs("uncog")
  • def commit():
  • """Removes the generated code from the docs and then commits to bzr."""
  • pass
@cmdopts
@cmdopts
  • @task
  • @cmdopts([("username=", "u", "Username for remote server"),
  • ("server=", "s", "Server to deploy to")])
  • def deploy():
  • """Copy the Paver website up."""
    • pass
paver taskname
paver <taskname>
  • $ paver generate_setup minilib
  • ---> generate_setup
  • Write setup.py
  • ---> minilib
  • Generate paver-minilib.zip
paver embraces and extends distutils
Paver embraces and extends Distutils

(don’t worry, it’s not evil)

setup py example
setup.py example
  • from distutils.core import setup
  • setup(name='foo', version='1.0', py_modules=['foo'], )
upgrading to paver
Upgrading to Paver
  • from paver.setuputils import setupsetup(name='foo', version='1.0', py_modules=['foo'], )
keeping it simple for users
Keeping it simple for users
  • $ paver generate_setup minilib
  • ---> generate_setup
  • Write setup.py
  • ---> minilib
  • Generate paver-minilib.zip
slide38

The Paver Standard Library

(is actually newer and less musty)

the paver standard library
The Paver Standard Library
  • Easy scripting (paver.easy)
  • Distutils integration (paver.setuputils)
  • File handling (paver.path)
  • Documentation tools (paver.doctools)
  • Subversion (paver.svn)
  • SSH (paver.ssh)
  • Virtualenv (paver.virtual)
  • Miscellaneous Tasks (paver.misctasks)
paver easy
paver.easy
  • from paver.easy import *
  • # display text if verbose is set
  • debug(“Hi there. Feeling chatty today?”)
  • # display text if quiet is not set
  • info(“Glad we don’t have to keep quiet”)
  • # display text regardless of setting
  • error(“HA! I’M SHOUTING AND YOU CAN’T STOP ME!”)
paver easy continued
paver.easy (continued)
  • # run a command, as long as dry-run is off
  • # capture the output into myval
  • myval = sh(“cat /tmp/foo”, capture=True)
  • # run a function (delete_all(‘/’) to be exact).
  • # if dry-run is set, then
  • # just print the message instead
  • dry(“Delete everything”, delete_all, ‘/’)
paver path
paver.path
  • Jason Orendorff’s path.py module (also available separately)
  • It’s a subclass of string!
  • Fun use of operator overloading
  • Lots of great methods
  • Makes working with files/directories fun and easy
paver path examples
paver.path examples
  • p = path("docs")tmpdir = p /"tmp"tmpdir.mkdir()fn = tmpdir /"myfile.txt"fn.write_text("Hi there!")
paver doctools
paver.doctools
  • # add this to use
  • import paver.doctools
paver doctools sphinx
paver.doctools – Sphinx
  • paver.doctools.html()
  • Build HTML documentation using Sphinx. This uses the following options in a “sphinx” section of the options.
  • docroot
  • the root under which Sphinx will be working. Default: docs
  • builddir
  • directory under the docroot where the resulting files are put. default: build
  • sourcedir
  • directory under the docroot for the source files default: (empty string)
  • paver.doctools.doc_clean()¶
  • Clean (delete) the built docs. Specifically, this deletes the build directory under the docroot. See the html task for the options list.
paver doctools cog sectionedfile
paver.doctools – Cog & SectionedFile
  • Solutions to common problems of creating high-quality docs
  • Ideally, your code samples will be in convenient runnable code files and have unit tests.
  • But you also want nice, minimal code samples in your text as you’re writing.
  • Ned Batchelder’s Cog (included with Paver) and paver.doctools.SectionedFile make these easy.
  • Not Sphinx-specific at all
paver doctools sample code
paver.doctools sample code
  • # mysample.py# [[[section mysample]]]defsample_func():print "To sample, or not to sample?"# [[[endsection]]]
paver doctools in docs
paver.doctools in docs
  • And then when you want to print the sample string, you just call::
  • # [[[cog include(“code/mysample.py”, “mysample”)]]]
  • # [[[end]]]
paver doctools in docs 2
paver.doctools in docs (2)
  • And then when you want to print the sample string, you just call::
  • # [[[cog include(“code/mysample.py”, “mysample”)]]]
  • def sample_func():
  • print "To sample, or not to sample?"
  • # [[[end]]]
paver doctools uncog before commit
paver.doctools uncog before commit
  • @task
  • @needs("uncog")
  • def commit():
  • """Removes the generated code from the docs and then commits to bzr."""
  • sh("bzr commit")
pulling it all together
Pulling it all together
  • write functions with the @task decorator.
  • Just plain old Python with lots of conveniences
  • Good example in projects like Bespin, PyMOTW
  • Even better, take a look at “Getting Started with Paver”http://bit.ly/starting_paver
remember this
Remember this?
  • #!/bin/sh
  • sphinx-build (blah blah blah)
  • python setup.py sdist upload
paver needs to do that too
Paver needs to do that too
  • @task
  • @needs(['html', "minilib", "generate_setup", “setuptools.command.sdist”])
  • def sdist():
  • """Builds the documentation and the tarball."""
  • pass
but wait it does more
But wait, it does more!
  • @task
  • @needs(['cog', 'paver.doctools.html'])
  • def html():
  • """Build Paver's documentation and install it into paver/docs"""
  • builtdocs = path("docs") / options.sphinx.builddir / "html"
  • destdir = path("paver") / "docs"
  • destdir.rmtree()
  • builtdocs.move(destdir)
credits
Credits
  • Thanks to SitePen for supporting Paver
  • Bumpy road http://flickr.com/photos/ilya/442034/sizes/o/
  • Smooth roadhttp://flickr.com/photos/nicholas_t/317528561/
  • Taskshttp://flickr.com/photos/mpwillis/470557535/