Things to do for piuparts
=========================

Please also check the BTS, especially the bugs with a severity higher than
wishlist!


for 0.5x:

- Create links in htdocs from $codename to stable/testing/unstable used
  for all the sections. (Closes: #696093)
  Create simple master cronjob to this daily.

- use /sys/fs/selinux from wheezy onwards - see #682068 for more info

- make piuparts-master + piuparts-slave packages work with documented steps:
  - do not enable sudoers.d/piuparts-slave in p-s.deb, neither the cronjobs.
    - this is manual work one has done once after installation. we only
      document whats needs to be done how.
  - the shipped crontabs are not suitable from cron.d/ due to missing user
    column -> provide instructions how to install them as piuparts(s|m) user
    from /u/s/doc/p-(s|m)/examples
  - verify that all scripts in $user/bin/ also work from master-slave packages
    eventually enable some conjobs in the packages
  - slave.postinst should setup '~piupartsm/.ssh/authorized_keys' using triggers
    dpkg-trigger piuparts-master-please-install-the-slave-key
    + the current method only works if -slave is configured after -master
    - maybe do this manually again as well?

- Documentation related:
  - examples are duplicated in piuparts.1.txt and README.txt - only keep one copy.
  - README_server has some duplication information on configuration as well.

- more stats and graphs:
  - packages processed per day and section
    - master writes submissions.txt per section since 0.45
  - # of open bugs with tag piuparts
  - generate http://piuparts.debian.org/stable/states.png + testing.png from
    existing data
  - generate simple diagrams: number of source + binary package in all single distros:
    squeeze, wheezy, jessie, sid.
  - move counts.txt from htdocs to master.

- add a sample config with all possible keys set to some useful value
  (like /usr/share/doc/apt/examples/configure-index.gz)

- piuparts.conf.pejacevic: use mirror via nfs (faster)

- look for a solution to use the global debian mirror for debian-backports,
  too, to avoid hardcoding a specific mirror in distros.conf

- install from git/Makefile: remove the need for /etc/piuparts

- maybe compress all logfiles

- link wiki:piuparts/UseCases||UsageTips in README_1st (after we agreed on a name)


for 0.6x:

- enable unit tests again.

- if it weren't for 'slave-bin/slave_cleanup', the slave would only need
  rights to run "sudo piuparts" but nothing else. If we can clean this up,
  the sudoers.d should recommend sudo (lsof|kill|umount) for admins.

- if there were real schroot support, piuparts could be used without sudo.
  (#708663)

- generate piuparts.1.txt automatically from piuparts.py - see this blog post
  for a nice howto:
  http://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/
  - though this seems pretty complicated... maybe rather grep for
    parser.add_option and help= in piuparts.py ?!
    - requires merging all the additional infomation in piuparts.1.txt into piuarts.py
    - parsing piuparts --help output may be easier than parsing piuparts.py

- rework known_problems:
  - split detect_well_known_errors
    - parsing the logfiles should stay there
    - generating the html should be integrated into piuparts-report
  - use a number prefix for sorting
  - add title information
  - piuparts-report: "discover" the available known_problems, dont hardcode the
    list
  - drop _issue/_error duplication, have flags inside to indicate thether to
    generate _issues.tpl (pass/) and/or _error.tpl (fail/ bugged/ affected/)
  - rework known problems to a python-friendlier format

- accept a PIUPARTS_CONF environment variable everywhere to point to a different
  piuparts.conf

- write reportbug-like wrapper for mass bug filing (start simple, make it more
  sophisticated later).

- rewrite piuparts-analyze to run over all sections and cache BTS responses

- "decorate" (strike-through) bug links generated by piuparts-analyze to
  indicate resolved state (take package version into account!)

- verify that find_default_debian_mirrors does something sane

- find_default_debian_mirrors: if parts[2] contains a / (think stable/updates
  for security.d.o), you can't ignore this, it will break later...
  - with distros.conf this may no longer be needed
  - check whether find_default_debian_mirrors produces something useful if
    sources.list does not exist (and sources.list.d/*.list is there instead)
  - maybe parse 'apt-cache policy' to get the mirror list instead

- make it possible to call aptitude (or similar) instead of apt-get and allow to
  override the commandline arguments.

- the templates used by update-reports.py and detect_well_known_errors should
  be taken from /etc/piuparts/templates/ and not be included in the python source

- check the logfiles (especially pass/) for
  - "Exception in thread"
  - java stacktraces
  - "Can't locate .* in @INC"


for 0.7x and later:

- mounting /proc and perhaps others (usbfs, sysfs, /dev/pts, etc.) in the chroot
  might be a good idea because some packages might need this.

- report:
  - write stats about the reasons for failures, as its done with shell scripts
    now (piuparts-analyze.py is an existing "fragment".)
  - RSS feeds of logs
  - do more fancy R graphs, eg. also per state
  - link (and target) to piuparts.d.o configuration is static to pejacevic. should
    refer to the actual hosts configuration if running somewhere else

- a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would
  be nice

- really support multiple architectures:
  - piuparts-report should have a list of available arch and list packages only
    available on untested archs in a new state
    "(depends-)not-available-on-tested-archs"
  - master should (per default) only schedule packages which are not available
    on the master arch to slaves of different archs ->
    "schedule-evenly-to-slaves = no"

- piuparts-master: keep track of to whom a reservation was given

- piuparts can't currently test upgrades of required packages.  (Because they
  cannot be removed, it assumes these are untestable, which is only true for
  removal tests...
  - all distupgrade tests implicitly tests these upgrades, although not
    individually per package

- not sure if it's a sensible thing to to, but provide a way to turn off
  debugging output for piuparts.py - see
  http://docs.python.org/library/logging.html

- commandline-switches for all programms

- move shell cronjobs functionality into master, slave & report


1.0 should really have automated testing of piuparts itself...
-----------------------------------------------------------------

- create archive of broken packages to provide test cases for piuparts testing.

- create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later
  put more broken packages in there and use that for testing piuparts)

