Packaging with the Build System

Currently the µracoli release packages are generated with a external script (Tools/makerelease.sh). The script is a shell script and is tailored to Linux systems. Nevertheless the SCons build system should be capable to build the ZIP-archives directly on request.

Step 1: Which archive format ?

We started already with the ZIP format, because it is a very common format. Other exotic
formats may achieve a better compression ratio but force the user to find and install the right tool for handling the archive. So we stick in the tradition of the good old PKZIP
(http://en.wikipedia.org/wiki/List_of_archive_formats)

Step 2: Finding the right tool for building Zip Files.

If you google for Scons and Zip, then you are often guided to the distutils.make_zip module.
After some experiments it had shown that this funktion lacks some flexibility.
The Requirments for the uracoli package builder was:

• Zipping a file located somewhere into a virtual folder in the Zipfile.
• Renaming files.
• Adding a path prefix to all files.
• No need to bulid a seperate tree and zip this tree.
  

It was found after some experiments, that all this can be achieved with the python ZipFile
module (http://docs.python.org/library/zipfile.html)

Step 3: Integrating it into Scons.

Next a Scons builder had to be written.

import zipfile
##
# This the Zipfile Builder Function for generating
# arbitrary Zipfiles.
#
# The function walk_tree is used to traverse any directory
# given and recursively adding the files therein.
#
# @param target
#        Name of the Zipfile
# @param source
#        List of the source files packed into the archive
# @param env
#        From env the following keys will be used:
#        - PREFIX :
#          prefix that is prepended to each file packed in the archiv.
#          It must not start with "/"
#        - RELOCATE :
#          Dicitionry that is used for translating file and directory names.
#        - COMMENT :
#          maximum 65535 byte long string that is added as comment.
#
def create_zip_file(target, source, env):
 target = str(target[0])
 sources = map(str,source)
 prefix = env.get("PREFIX","")
 relocate = env.get("RELOCATE",{})
 comment = env.get("COMMENT",None)
 # generate list of sourcefiles and its name mappings in zipfile
 ziplist = []
 for src in sources:
     for indirname, filelst in walk_tree(src):
         # normalize dirname to have / seperator to do dictionary lookup
         indn = indirname.replace("\\","/")
         outdirname = relocate.get(indn, indn)
         for infilename in filelst:
             outfilename = relocate.get(infilename,infilename)
             infn = os.path.join(indirname,infilename)
             outfn = os.path.join(prefix, outdirname, outfilename)
             ziplist.append( (infn, outfn) )

 # write zipfile
 zf = zipfile.ZipFile(target,"w")
 for fn_in, fn_out in ziplist:
     zf.write(fn_in, fn_out, zipfile.ZIP_DEFLATED)
 if comment != None:
     zipfile.ZipFile.comment = comment
 zf.close()
##
# This function comes from:
# http://code.activestate.com/recipes/200131-recursive-directory-listing-in-html/
def walk_tree(top = ".", depthfirst = True):
 # This  function recursively traverses a directory tree.
....
## Using the
com['BUILDERS']['ZipFile'] =  Builder(action = create_zip_file, suffix = '.zip')

pname = "mypkg"
flist = "install/inc/foo.h install/bin/bar.hex ....".split()
pkg = com.ZipFile("#/install/%s" % pname, flist,
       PREFIX = pname,
       RELOCATE = {"install/inc":"inc", .... },
       COMMENT = "A Pacakge")

Step 4: Defining Packages outside Scons.

The packages should be defined in seperate config file which is not in the SConstruct/SConscript files. Beside the dividing of code and data, this offers, that a user can easily add its own pachkage, A working example can be found in the file packages.cfg in the root directory that contains all package description data.

The File has win.ini Style format and can be from the python module ConfigParser, that comes natively with the python install package.

Voila, and thats in a few words the essence of the new package builder.

There are of course the last 20% to perfection missing, e.g. listing the packages in scons help
or finding a way to build the clean source code archive, frehsly checked out from CVS, but somewhere in a silent minute an idea to that will come.