Provided by: checkbox-ng_0.3-2_all 
NAME
checkbox_ng - CheckBoxNG Documentation
CheckBoxNG is a hardware testing tool useful for certifying laptops, desktops and servers
with Ubuntu. It is a new version of CheckBox that is built directly on top of PlainBox
CheckBoxNG replaces CheckBox, where applicable.
WARNING:
Documentation is under development. Some things are wrong, inaccurate or describe
development goals rather than current state.
INSTALLATION
CheckBoxNG can be installed from a PPA (recommended) or pypi on Ubuntu Precise (12.04) or
newer.
$ sudo add-apt-repository ppa:checkbox-dev/ppa && sudo apt-get update && sudo apt-get install checkbox-ng
RUNNING STABLE RELEASE UPDATE TESTS
CheckBoxNG has special support for running stable release updates tests in an automated
manner. This runs all the jobs from the sru.whitelist and sends the results to the
certification website.
To run SRU tests you will need to know the so-called Secure ID of the device you are
testing. Once you know that all you need to do is run:
$ checkbox sru $secure_id submission.xml
The second argument, submission.xml, is a name of the fallback file that is only created
when sending the data to the certification website fails to work for any reason.
TEST SCRIPTS
Test 'scripts' are small programs which are used to aid in implementing tests.
brightness_test
This script tests the brightness of the systems backlight can be changed by using the
kernel interfaces in /sys/class/backlight. There may be more than one interface to choose
from, so the correct interface to use is selected by using the heuristic prescribed in
https://www.kernel.org/doc/Documentation/ABI/stable/sysfs-class-backlight. The brightness
is manipulated by updating the brightness file of the interface and the actual_brightness
file is checked to see if the value was modified to the brightness selected.
CHECKBOX RELEASE PROCESS
This page describes the necessary steps for releasing versions of CheckBox and CheckBox
Certification to the stable PPA belonging to the Hardware Certification team, on a regular
basis. Throughout this document the term 'CheckBox' is used as a catch-all term to cover
all versions of CheckBox owned by the Hardware Certification team, currently CheckBox
itself and the CheckBox Certification extensions.
Overview
Currently the process runs on a bi-weekly cadence, with a new release of Checkbox every
two weeks. This covers ten working days, and the tasks carried out on each day or group of
days is described below:
• Days 1-4: Time allowed for new changes to be introduced into trunk.
• Day 5: Changes are merged from the trunk of lp:checkbox and lp:checkbox-certification to
their respective release branches. The changelogs for both are bumped at this point and
revisions are tagged. At this stage it may also be necessary to copy the package 'fwts'
from the FWTS Stable PPA to the Checkbox Release Testing PPA.
• Days 6-9: Testing is performed by the release manager for the Hardware Certification
team, and a representative of the CE QA team (the main customer for CheckBox within
Canonical)
• Day 9: A release meeting is held between the release manager for the Hardware
Certification team and the representative of the CE QA team. Potential issues with the
release are identified and plans made to address them.
• Day 10: The tested version of CheckBox is copied to the stable PPA.
Launchpad Branches
The release process requires separate branches in Launchpad containing a semi-frozen
version of the code that was in trunk on day 5 of the process. This is so that development
can continue on trunk without jeopardising the stability of the to-be released version of
CheckBox. The relationship between all branches involved in the process is as shown below:
• lp:checkbox/release <- lp:checkbox
• lp:checkbox-certification/release <- lp:checkbox-certification
• lp:~checkbox-dev/checkbox/checkbox-packaging-release <-
lp:~checkbox-dev/checkbox/checkbox-packaging
Auditing milestoned bugs
Prior to creating the release candidate the release manager should review the list of bugs
milestoned for the next release of CheckBox. They should visit checkbox milestones and
locate the milestone dated with the release date.
• For bugs that are set to In Progress with a branch associated - liase with the branch
owner to see if the merge can be completed before the deadline.
• For bugs that are in any other non-closed status (except Fix Commited) - re-milestone
them to the following milestone.
Cutting the release
In order to cut the release, we have to merge the changes from trunk into the release
branch, commit them with a suitable message and update the changelog in trunk so that
future changes go under the correct version. For each combination of branches shown above,
do the following (the example uses lp:checkbox and lp:checkbox/release):
bzr branch lp:checkbox/release checkbox-release
bzr branch lp:checkbox checkbox-trunk
cd checkbox-release
current_stable=`head -n1 $(find . -name 'changelog') | grep -oP '(?<=\().*(?=\))'`
bzr merge lp:checkbox
at this point if no changes (other than one to debian/changelog) get merged in then we do
not perform a release of the package in question. In practice this often happens with
checkbox-certification but never with checkbox:
bzr commit -m "Merged in changes from rev$(bzr revno -r tag:$current_stable lp:checkbox) to rev$(bzr revno lp:checkbox) from lp:checkbox"
bzr push lp:checkbox/release
cd `find . -name 'debian'`; cd ..
bzr tag `head -n1 debian/changelog | grep -oP '(?<=\().*(?=\))'`
dch -r (save modified changelog)
dch -i -U 'Incremented changelog'
debcommit
bzr push lp:checkbox
The last step in the process is to perform a build of the packages in the
ppa:checkbox-dev/testing PPA. To do this we need to go to the recipe pages for the
checkbox and/or checkbox-certification release branches.
• checkbox-testing recipe
• checkbox-certification-testing recipe
The Build Now option should be available on the page. Click it to start a build.
Copying Firmware Test Suite to the Testing PPA
The Firmware Test Suite tool is a test tool for system firmware that is naturally heavily
utilised by Checkbox. To make sure the latest version which contains fixes and new
tests/features needed by Checkbox is available and also doesn't break anything in
Checkbox, we need to release it alongside Checkbox. After cutting the release if the
Firmware Testing team have notified that a new version is available and that this version
should be used for certification, we need to copy it to the Testing PPA. To do this we
need to go to the Copy packages view of the Firmware Test Suite (Stable) PPA and select
the 'fwts' packages for all releases back to Precise. We need to set the 'Destination PPA'
as 'Checkbox Release Testing [~checkbox-dev/testing]' and the 'Copy options' field to
'Copy existing binaries', then click 'Copy packages'. This step then needs to be repeated
but set the 'Destination PPA' field to 'PPA for Checkbox Developers [~checkbox-dev/ppa]'.
Next Release of Checkbox e-mail
So that everyone has the opportunity to perform whatever testing is required in a timely
manner, after the PPA builds have been completed an email should be sent to the following
mailing lists:
• hardware-certification-team@lists.canonical.com
• commercial-engineering@lists.canonical.com
The content is typically something like this:
Subject: Next Release of CheckBox (18/11/2013)
Hi,
The next release of CheckBox is available in the
https://code.launchpad.net/~checkbox-dev/+archive/testing PPA.
Please test it at your convenience. CheckBox is based on revision 2484 of
lp:checkbox and CheckBox Certification is based on revision 586 of
lp:checkbox-certification.
Thanks,
If one or the other of CheckBox and CheckBox Certification have not been updated then
there is no need to mention that package
Testing the release
Now that the release has been cut, testing should take place prior to the release meeting.
From the point of view of the certification team, what needs to be tested is
checkbox-certification-client and checkbox-certification-server which form the basis for
CE QAs OEM specific versions of CheckBox. CheckBox certification server is tested in the
CI loop CheckBox certification client needs to be tested manually.
Release Meeting
On the Thursday before the release is made, a meeting is held between a representative of
the Certification team and a representative of the Commercial Engineering QA team. The
meeting is held at 7:30 UTC as shown in this calendar invite. An agenda for the meeting
is included in the invite.
Publishing the release
To publish the release we simply need to copy a number of packages from the Checkbox
Release Testing PPA to the Hardware Certification Public PPA. To do this we go to the Copy
packages view of the Checkbox Release Testing PPA and select all versions of the following
list of packages: checkbox, checkbox-certification, fwts. Make sure that the 'Destination
PPA' field is set to 'Public PPA for Hardware Certification
[~hardware-certification/public]' and that the 'Copy options' field is set to 'Copy
existing binaries', then click 'Copy Packages'.
After that is done an announcement email should be sent to
commercial-engineering@lists.canonical.com. A template for the announcement in included
below:
Hi,
A new release of checkbox has been uploaded to the Hardware
Certification Public PPA
(https://launchpad.net/~hardware-certification/+archive/public). The
release is based on revision 2294 of lp:checkbox
Thanks,
Please attach the most recent part of the changelog as release notes
• genindex
• modindex
• search
AUTHOR
Zygmunt Krynicki
COPYRIGHT
2013, Zygmunt Krynicki