Provided by: swupdate_2024.12+dfsg-2_amd64 
NAME
swupdate - SWUpdate Documentation [image]
SWUpdate provides a reliable way to update the software on an embedded system. Sources
are hosted at https://github.com/sbabic/swupdate. Do not forget to star SWUpdate.
SOFTWARE MANAGEMENT ON EMBEDDED SYSTEMS
As Embedded Systems become more and more complex, their software reflects the augmented
complexity. It is vital that the software on an embedded system can be updated in an
absolutely reliable way, as new features and fixes are added.
On a Linux-based system, we can find in most cases the following elements:
• the boot loader.
• the kernel and the DT (Device Tree) file.
• the root file system
• other file systems, mounted at a later point
• customer data, in raw format or on a file system
• application specific software. For example, firmware to be downloaded on connected
micro-controllers, and so on.
Generally speaking, in most cases it is required to update kernel and root file system,
preserving user data - but cases vary.
In only a few cases it is required to update the boot loader, too. In fact, updating the
boot loader is quite always risky, because a failure in the update breaks the board.
Restoring a broken board is possible in some cases, but this is not left in most cases to
the end user and the system must be sent back to the manufacturer.
There are a lot of different concepts about updating the software. I like to expose some
of them, and then explain why I have implemented this project.
Updating through the boot loader
Boot loaders do much more as simply start the kernel. They have their own shell and can
be managed using a processor's peripheral, in most cases a serial line. They are often
script-able, letting possible to implement some kind of software update mechanism.
However, I found some drawbacks in this approach, that let me search for another solution,
based on an application running on Linux:
Boot loaders have limited access to peripherals
Not all peripherals supported by the kernel are available with the boot loader. When it
makes sense to add support to the kernel, because the peripheral is then available by the
main application, it does not always make sense to duplicate the effort to port the driver
to the boot loader.
Boot loader's drivers are not updated
Boot loader's drivers are mostly ported from the Linux kernel, but due to adaptations they
are not later fixed or synchronized with the kernel, while bug fixes flow regularly in the
Linux kernel. Some peripherals can then work in a not reliable ways, and fixing the
issues can be not easy. Drivers in boot loaders are more or less a fork of the respective
drivers in kernel.
As example, the UBI / UBIFS for NAND devices contains a lot of fixes in the kernel, that
are not ported back to the boot loaders. The same can be found for the USB stack. The
effort to support new peripherals or protocols is better to be used for the kernel as for
the boot loaders.
Reduced file systems
The number of supported file systems is limited and porting a file system to the boot
loader requires high effort.
Network support is limited
Network stack is limited, generally an update is possible via UDP but not via TCP.
Interaction with the operator
It is difficult to expose an interface to the operator, such as a GUI with a browser or on
a display.
A complex logic can be easier implemented inside an application else in the boot loader.
Extending the boot loader becomes complicated because the whole range of services and
libraries are not available.
Boot loader's update advantages
However, this approach has some advantages, too:
• software for update is generally simpler. - smaller footprint: a stand-alone
application only for software management requires an own kernel and a root file system.
Even if their size can be trimmed dropping what is not required for updating the
software, their size is not negligible.
Updating through a package manager
All Linux distributions are updating with a package manager. Why is it not suitable for
embedded ?
I cannot say it cannot be used, but there is an important drawback using this approach.
Embedded systems are well tested with a specific software. Using a package manager can put
weirdness because the software itself is not anymore atomic, but split into a long list of
packages. How can we be assured that an application with library version x.y works, and
also with different versions of the same library? How can it be successfully tested?
For a manufacturer, it is generally better to say that a new release of software (well
tested by its test engineers) is released, and the new software (or firmware) is available
for updating. Splitting in packages can generate nightmare and high effort for the
testers.
The ease of replacing single files can speed up the development, but it is a
software-versions nightmare at the customer site. If a customer report a bug, how can it
is possible that software is "version 2.5" when a patch for some files were sent
previously to the customer ?
An atomic update is generally a must feature for an embedded system.
Strategies for an application doing software upgrade
Instead of using the boot loader, an application can take into charge to upgrade the
system. The application can use all services provided by the OS. The proposed solution is
a stand-alone software, that follow customer rules and performs checks to determine if a
software is installable, and then install the software on the desired storage.
The application can detect if the provided new software is suitable for the hardware, and
it is can also check if the software is released by a verified authority. The range of
features can grow from small system to a complex one, including the possibility to have
pre- and post- install scripts, and so on.
Different strategies can be used, depending on the system's resources. I am listing some
of them.
Double copy with fall-back
If there is enough space on the storage to save two copies of the whole software, it is
possible to guarantee that there is always a working copy even if the software update is
interrupted or a power off occurs.
Each copy must contain the kernel, the root file system, and each further component that
can be updated. It is required a mechanism to identify which version is running.
SWUpdate should be inserted in the application software, and the application software will
trigger it when an update is required. The duty of SWUpdate is to update the stand-by
copy, leaving the running copy of the software untouched.
A synergy with the boot loader is often necessary, because the boot loader must decide
which copy should be started. Again, it must be possible to switch between the two copies.
After a reboot, the boot loader decides which copy should run. [image]
Check the chapter about boot loader to see which mechanisms can be implemented to
guarantee that the target is not broken after an update.
The most evident drawback is the amount of required space. The available space for each
copy is less than half the size of the storage. However, an update is always safe even in
case of power off.
This project supports this strategy. The application as part of this project should be
installed in the root file system and started or triggered as required. There is no need
of an own kernel, because the two copies guarantees that it is always possible to upgrade
the not running copy.
SWUpdate will set bootloader's variable to signal the that a new image is successfully
installed.
Single copy - running as standalone image
The software upgrade application consists of kernel (maybe reduced dropping not required
drivers) and a small root file system, with the application and its libraries. The whole
size is much less than a single copy of the system software. Depending on set up, I get
sizes from 2.5 until 8 MB for the stand-alone root file system. If the size is very
important on small systems, it becomes negligible on systems with a lot of storage or big
NANDs.
The system can be put in "upgrade" mode, simply signaling to the boot loader that the
upgrading software must be started. The way can differ, for example setting a boot loader
environment or using and external GPIO.
The boot loader starts "SWUpdate", booting the SWUpdate kernel and the initrd image as
root file system. Because it runs in RAM, it is possible to upgrade the whole storage.
Differently as in the double-copy strategy, the systems must reboot to put itself in
update mode.
This concept consumes less space in storage as having two copies, but it does not
guarantee a fall-back without updating again the software. However, it can be guaranteed
that the system goes automatically in upgrade mode when the productivity software is not
found or corrupted, as well as when the upgrade process is interrupted for some reason.
[image]
In fact, it is possible to consider the upgrade procedure as a transaction, and only after
the successful upgrade the new software is set as "boot-able". With these considerations,
an upgrade with this strategy is safe: it is always guaranteed that the system boots and
it is ready to get a new software, if the old one is corrupted or cannot run. With U-Boot
as boot loader, SWUpdate is able to manage U-Boot's environment setting variables to
indicate the start and the end of a transaction and that the storage contains a valid
software. A similar feature for GRUB environment block modification as well as for EFI
Boot Guard has been introduced.
SWUpdate is mainly used in this configuration. The recipes for Yocto generate an initrd
image containing the SWUpdate application, that is automatically started after mounting
the root file system. [image]
Something went wrong ?
Many things can go wrong, and it must be guaranteed that the system is able to run again
and maybe able to reload a new software to fix a damaged image. SWUpdate works together
with the boot loader to identify the possible causes of failures. Currently U-Boot, GRUB,
and EFI Boot Guard are supported.
We can at least group some of the common causes:
• damage / corrupted image during installing. SWUpdate is able to recognize it and the
update process is interrupted. The old software is preserved and nothing is really
copied into the target's storage.
• corrupted image in the storage (flash)
• remote update interrupted due to communication problem.
• power-failure
SWUpdate works as transaction process. The boot loader environment variable
"recovery_status" is set to signal the update's status to the boot loader. Of course,
further variables can be added to fine tuning and report error causes. recovery_status
can have the values "progress", "failed", or it can be unset.
When SWUpdate starts, it sets recovery_status to "progress". After an update is finished
with success, the variable is erased. If the update ends with an error, recovery_status
has the value "failed".
When an update is interrupted, independently from the cause, the boot loader recognizes it
because the recovery_status variable is in "progress" or "failed". The boot loader can
then start again SWUpdate to load again the software (single-copy case) or run the old
copy of the application (double-copy case).
Power Failure
If a power off occurs, it must be guaranteed that the system is able to work again -
starting again SWUpdate or restoring an old copy of the software.
Generally, the behavior can be split according to the chosen scenario:
• single copy: SWUpdate is interrupted and the update transaction did not end with a
success. The boot loader is able to start SWUpdate again, having the possibility to
update the software again.
• double copy: SWUpdate did not switch between stand-by and current copy. The same
version of software, that was not touched by the update, is started again.
To be completely safe, SWUpdate and the bootloader need to exchange some information. The
bootloader must detect if an update was interrupted due to a power-off, and restart
SWUpdate until an update is successful. SWUpdate supports the U-Boot, GRUB, and EFI Boot
Guard bootloaders. U-Boot and EFI Boot Guard have a power-safe environment which SWUpdate
is able to read and change in order to communicate with them. In case of GRUB, a fixed
1024-byte environment block file is used instead. SWUpdate sets a variable as flag when it
starts to update the system and resets the same variable after completion. The bootloader
can read this flag to check if an update was running before a power-off. [image]
What about upgrading SWUpdate itself ?
SWUpdate is thought to be used in the whole development process, replacing customized
process to update the software during the development. Before going into production,
SWUpdate is well tested for a project.
If SWUpdate itself should be updated, the update cannot be safe if there is only one copy
of SWUpdate in the storage. Safe update can be guaranteed only if SWUpdate is duplicated.
There are some ways to circumvent this issue if SWUpdate is part of the upgraded image:
• have two copies of SWUpdate
• take the risk, but have a rescue procedure using the boot loader.
What about upgrading the Boot loader ?
Updating the boot loader is in most cases a one-way process. On most SOCs, there is no
possibility to have multiple copies of the boot loader, and when boot loader is broken,
the board does not simply boot.
Some SOCs allow one to have multiple copies of the boot loader. But again, there is no
general solution for this because it is very hardware specific.
In my experience, most targets do not allow one to update the boot loader. It is very
uncommon that the boot loader must be updated when the product is ready for production.
It is different if the U-Boot environment must be updated, that is a common practice.
U-Boot provides a double copy of the whole environment, and updating the environment from
SWUpdate is power-off safe. Other boot loaders can or cannot have this feature.
LICENSE
SWUpdate is Free Software. It is copyrighted by Stefano Babic and many others who
contributed code (see the actual source code and the git commit messages for details).
You can redistribute SWUpdate and/or modify it under the terms of version 2 of the GNU
General Public License as published by the Free Software Foundation. Some files can also
be distributed, at your option, under any later version of the GNU General Public License
-- see individual files for exceptions.
To make this easier, license headers in the source files will be replaced with a single
line reference to Unique License Identifiers as defined by the Linux Foundation's SPDX
project [1]. For example, in a source file the full "GPL v2.0 only" header text will be
replaced by a single line:
SPDX-License-Identifier: GPL-2.0-only
Ideally, the license terms of all files in the source tree should be defined by such
License Identifiers; in no case a file can contain more than one such License Identifier
list.
If a "SPDX-License-Identifier:" line references more than one Unique License Identifier,
then this means that the respective file can be used under the terms of either of these
licenses, i. e. with
SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause
you can choose between GPL-2.0-only and BSD-3-Clause licensing.
We use the SPDX Unique License Identifiers (SPDX-Identifiers)
LICENSES
┌─────────────────────────┬───────────────────┬──────────────┐
│Full name │ SPDX Identifier │ OSI Approved │
├─────────────────────────┼───────────────────┼──────────────┤
│GNU General Public │ GPL-2.0-only │ Y │
│License v2.0 only │ │ │
├─────────────────────────┼───────────────────┼──────────────┤
│GNU Lesser General │ LGPL-2.1-or-later │ Y │
│Public License v2.1 or │ │ │
│later │ │ │
├─────────────────────────┼───────────────────┼──────────────┤
│BSD 1-Clause License │ BSD-1-Clause │ Y │
├─────────────────────────┼───────────────────┼──────────────┤
│BSD 2-Clause License │ BSD-2-Clause │ Y │
├─────────────────────────┼───────────────────┼──────────────┤
│BSD 3-Clause "New" or │ BSD-3-Clause │ Y │
│"Revised" License │ │ │
├─────────────────────────┼───────────────────┼──────────────┤
│MIT License │ MIT │ Y │
├─────────────────────────┼───────────────────┼──────────────┤
│Creative Commons Zero │ CC0-1.0 │ N │
│1.0 Universal (CC0) │ │ │
├─────────────────────────┼───────────────────┼──────────────┤
│Creative Commons │ CC-BY-SA-4.0 │ Y │
│Attribution Share Alike │ │ │
│4.0 │ │ │
├─────────────────────────┼───────────────────┼──────────────┤
│ISC License (ISC) │ ISC │ Y │
└─────────────────────────┴───────────────────┴──────────────┘
SWUPDATE: SOFTWARE UPDATE FOR EMBEDDED SYSTEM
Overview
This project is thought to help to update an embedded system from a storage media or from
network. However, it should be mainly considered as a framework, where further protocols
or installers (in SWUpdate they are called handlers) can be easily added to the
application.
One use case is to update from an external local media, as USB-Pen or SD-Card. In this
case, the update is done without any intervention by an operator: it is thought as
"one-key-update", and the software is started at reset simply pressing a key (or in any
way that can be recognized by the target), making all checks automatically. At the end,
the updating process reports only the status to the operator (successful or failed).
The output can be displayed on a LCD using the frame-buffer device or directed to a serial
line (Linux console).
It is generally used in the single copy approach, running in an initrd (recipes are
provided to generate with Yocto). However, it is possible to use it in a double-copy
approach by use of Software collections.
If started for a remote update, SWUpdate starts an embedded Web-server and waits for
requests. The operator must upload a suitable image, that SWUpdate checks and then
install. All output is notified to the operator's browser via AJAX notifications.
Features
General Overview
• Install on embedded Media (eMMC, SD, Raw NAND, NOR and SPI-NOR flashes)
• check if an image is available. The image is built in a specified format (cpio) and it
must contain a file describing the software that must be updated.
• SWUpdate is thought to update UBI volumes (mainly for NAND, but not only) and images on
devices. Passing a whole image can still be updated as a partition on the SD card, or a
MTD partition.
• new partition schema. This is bound with UBI volume. SWUpdate can recreate UBI volumes,
resizing them and copying the new software.
• support for compressed images, using the zlib and zstd library. tarball (tgz file) are
supported.
• support for partitioned USB-pen or unpartitioned (mainly used by Windows).
• support for updating a single file inside a filesystem. The filesystem where to put the
file must be described.
• checksum for the single components of an image
• use a structured language to describe the image. This is done using the libconfig
library as default parser, that uses a JSON-like description.
• use custom's choice for the description of the image. It is possible to write an own
parser using the Lua language. An example using a XML description in Lua is provided in
the examples directory.
• Support for setting / erasing U-Boot variables
• Support for setting / erasing GRUB environment block variables
• Support for setting / erasing EFI Boot Guard variables
• Support for pre and post update commands run before the update starts processing data
and after the update has finished successfully.
• Support for lua hooks, executed before any handler runs.
• Support for preinstall scripts. They run after streamed handlers have handled their
data, and before regular handlers.
• Support for postinstall scripts. They run after updating the images.
• Network installer using an embedded Web-server (Mongoose Server was chosen, in the
version under Lua license). A different Web-server can be used.
•
Multiple interfaces for getting software
• local Storage: USB, SD, UART,..
•
OTA / Remote
• integrated Web-Server
• pulling from remote Server (HTTP, HTTPS, ..)
• using a Backend. SWUpdate is open to talk with back end servers for rolling out
software updates. Current version supports the hawkBit server, but other
backend can be added.
• Can be configured to check for compatibility between software and hardware revisions.
The software image must contain an entry declaring on which HW revision the software is
allowed to run. SWUpdate refuses to install if the compatibility is not verified.
• support for image extraction. A manufacturer can require to have a single image that
contains the software for more as one device. This simplifies the manufacturer's
management and reduces their administrative costs having a single software product.
SWUpdate receives the software as stream without temporary storing, and extracts only
the required components for the device to be installed.
• allow custom handlers for installing FPGA firmware, micro-controller firmware via custom
protocols.
• Features are enabled / disabled using "make menuconfig". (Kbuild is inherited from
busybox project)
• Images are authenticated and verified before installing
• Power-Off safe
Single image delivery
The main concept is that the manufacturer delivers a single big image. All single images
are packed together (cpio was chosen for its simplicity and because can be streamed)
together with an additional file (sw-description), that contains meta information about
each single image.
The format of sw-description can be customized: SWUpdate can be configured to use its
internal parser (based on libconfig), or calling an external parser in Lua. [image]
Changing the rules to accept images with an external parser, let to extend to new image
types and how they are installed. In fact, the scope of the parser is to retrieve which
single images must be installed and how. SWUpdate implements "handlers" to install a
single image: there are handlers to install images into UBI volumes, or to a SD card, a
CFI Flash, and so on. It is then easy to add an own handler if a very special installer is
required.
For example we can think at a project with a main processor and one or several
micro-controllers. Let's say for simplicity that the main processor communicates with the
micro-controllers via UARTS using a proprietary protocol. The software on the
micro-controllers can be updated using the proprietary protocol.
It is possible to extend SWUpdate writing a handler, that implements the part of the
proprietary protocol to perform the upgrade on the micro-controller. The parser must
recognize which image must be installed with the new handler, and SWUpdate will call the
handler during the installation process.
Streaming feature
SWUpdate is thought to be able to stream the received image directly into the target,
without any temporary copy. In fact, the single installer (handler) receive as input the
file descriptor set at the beginning of the image that must be installed.
The feature can be set on image basis, that means that a user can decide which partial
images should be streamed. If not streamed (see installed-directly flag), files are
temporary extracted into the directory pointed to by the environment variable TMPDIR with
/tmp as fall-back if TMPDIR is not set. Of course, by streaming it is not possible to
make checks on the whole delivered software before installing. The temporary copy is done
only when updated from network. When the image is stored on an external storage, there is
no need of that copy.
Images fully streamed
In case of remote update, SWUpdate extracts relevant images from the stream and copies
them into the directory pointed to by the environment variable TMPDIR (if unset, to /tmp)
before calling the handlers. This guarantee that an update is initiated only if all parts
are present and correct. However, on some systems with less resources, the amount of RAM
to copy the images could be not enough, for example if the filesystem on an attached SD
Card must be updated. In this case, it will help if the images are installed directly as
stream by the corresponding handler, without temporary copies. Not all handlers support to
stream directly into the target. Streaming with zero-copy is enabled by setting the flag
"installed-directly" in the description of the single image.
Configuration and Build
Requirements
There are only a few libraries strictly required to compile SWUpdate:
• zlib (https://www.zlib.net)
• libubootenv for U-Boot environment support (https://github.com/sbabic/libubootenv)
• json-c for parsing JSON (https://github.com/json-c/json-c)
Further library dependencies may be required when activating more SWUpdate features, see
the next section on Configuring SWUpdate.
Configuring SWUpdate
SWUpdate is configurable via make menuconfig. A small footprint is realized, e.g., by
using the internal parser and disabling the web server. Every option has a small help
describing its usage. In the default configuration, many options are already activated.
To configure the options:
make menuconfig
Building
To cross-compile, set the CC and CXX variables before running make. It is also possible
to set the cross-compiler prefix as option with make menuconfig. Then, generate the code
by running
make
The result is the binary swupdate. Notably, the tools/swupdate-progress binary is built as
well. It is an example of how to build your own interface to SWUpdate to, e.g., show a
progress bar on an HMI. This example simply prints on the console the current status of
the update and, more importantly, reboots the machine after successful installation.
Building with Yocto
See the corresponding chapter meta-swupdate: building with Yocto on how to build SWUpdate
in Yocto.
Distribution Packages
SWUpdate is thought for Embedded Systems and building in an embedded distribution is the
primary use case. But apart from the most used build systems for embedded systems like
Yocto or Buildroot, in some cases, a standard Linux distro is used. SWUpdate is currently
packaged for
• Debian
• Ubuntu
Refer to the distribution source package and build system on how to build a package from
source.
Running SWUpdate
What is expected from a SWUpdate run
The whole update process can be seen as a set of pipelines. The incoming stream (the SWU
file) is processed by each pipe and passed to the next step. First, the SWU is streamed
from one of the interfaces : local (USB, filesystem), Webserver, suricatta (one of the
backend), etc. The incoming SWU is forwarded to the installer to be examined and
installed. A run of SWUpdate consists mainly of the following steps:
• extracts sw-description from the stream and verifies it It parses sw-description
creating a raw description in RAM about the activities that must be performed.
• if Signed Images is activated, extracts sw-description.sig and validate sw-description.
• check for hardware-software compatibility, if any, reading hardware revision from
hardware and matching with the table in sw-description.
• Parse sw-description to determine which artefacts in the incoming SWU are required. Not
required artifacts are simply skipped. If an "embedded-script" is defined, it is
executed at this point before parsing files. If "hooks" are defined, they are executed
as each file is parsed, even if they will be skipped. At the end of the parsing,
SWUpdate builds an internal mapping for each artifact to recognize which handler should
be called for each of them.
• runs the pre update command, if set
• runs partition handlers, if required.
•
reads through the cpio archive one file at a time and either:
• execute handlers for each file marked as "installed-directly". checksum is
checked while the data is streamed to handler, and copy will be marked as
having failed if checksum was not correct failing the rest of the install.
• copy other files to a temporary location while checking checksums, stopping if
there was a mismatch.
• iterates through all scripts and call the corresponding handler for pre-install scripts.
Please note: if artifacts are streamed, they will be extracted before this runs. If
earlier execution is required, please use the "embedded-script" or hooks features to
ensure code is run before installation takes place.
• iterates through all images and call the corresponding handler for installing on target.
• iterates through all files and call the corresponding handler for installing on target.
• iterates through all scripts and call the corresponding handler for post-install scripts
• iterates through all bootenv and updates the bootloader environment.
• reports the status to the operator through the notification interface (logging, traces)
and through the progress interface.
• runs the post update command, if set.
The first step that fails, stops the entire procedure and an error is reported.
To start SWUpdate expecting the image from a file:
swupdate -i <filename>
To start with the embedded web server:
swupdate -w "<web server options>"
The main important parameters for the web server are "document-root" and "port".
swupdate -w "--document-root ./www --port 8080"
The embedded web server is taken from the Mongoose project.
The list of available options (depending on activated features) is shown with:
swupdate -h
This uses as website the pages delivered with the code. Of course, they can be customized
and replaced. The website uses AJAX to communicate with SWUpdate, and to show the progress
of the update to the operator.
The default port of the Web-server is 8080. You can then connect to the target with:
http://<target_ip>:8080
If it works, the start page should be displayed as in next figure. [image]
If a correct image is downloaded, SWUpdate starts to process the received image. All
notifications are sent back to the browser. SWUpdate provides a mechanism to send to a
receiver the progress of the installation. In fact, SWUpdate takes a list of objects that
registers itself with the application and they will be informed any time the application
calls the notify() function. This allows also for self-written handlers to inform the
upper layers about error conditions or simply return the status. It is then simply to add
own receivers to implement customized way to display the results: displaying on a LCD (if
the target has one), or sending back to another device via network. An example of the
notifications sent back to the browser is in the next figure: [image]
Software collections can be specified by passing --select command line option. Assuming
sw-description file contains a collection named stable, with alt installation location,
SWUpdate can be called like this:
swupdate --select stable,alt
Command line parameters
┌─────────────────────────┬───────────┬─────────────────────────────────────┐
│Parameter │ Type │ Description │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-f <file> │ string │ SWUpdate configuration │
│ │ │ file to use. See │
│ │ │ examples/configuration/swupdate.cfg │
│ │ │ in the source code for │
│ │ │ details. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-b <string> │ string │ Available if CONFIG_UBIATTACH is │
│ │ │ set. It allows one to blacklist │
│ │ │ MTDs when SWUpdate searches for UBI │
│ │ │ volumes. Example: U-Boot and │
│ │ │ environment in MTD0-1: swupdate -b │
│ │ │ "0 1". │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-B <loader> │ string │ Override the default bootloader │
│ │ │ interface to use loader instead. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-e <sel> │ string │ sel is in the format │
│ │ │ <software>,<mode>. It allows one │
│ │ │ to find a subset of rules in the │
│ │ │ sw-description file. With it, │
│ │ │ multiple rules are allowed. One │
│ │ │ common usage is in case of the dual │
│ │ │ copy approach. Example: -e "stable, │
│ │ │ copy1" ==> install on copy1 -e │
│ │ │ "stable, copy2" ==> install on │
│ │ │ copy2 │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│ │ string │ sel is in the format │
│ --excluded │ │ <software>,<mode>. It sets a │
│ <sel> │ │ blacklist of selections that cannot │
│ │ │ be used for an update. Selections │
│ │ │ can be activated not only with -e, │
│ │ │ but also via IPC. Multiple │
│ │ │ --excluded are allowed │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-h │ │ Run usage with help. │
│ │ • │ │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-k <file> │ string │ Available if CONFIG_SIGNED is set. │
│ │ │ Filename with the public key. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-K <file> │ string │ Available on │
│ │ │ CONFIG_ENCRYPTED_IMAGES set. │
│ │ │ Filename with the symmetric key to │
│ │ │ be used for decryption. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│--cert-purpose <purpose> │ string │ Available if CONFIG_SIGNED_IMAGES │
│ │ │ is set. Set expected certificate │
│ │ │ purpose. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│--forced-signer-name │ string │ Available if CONFIG_SIGNED_IMAGES │
│<cn> │ │ is set. Set expected common name │
│ │ │ of signer certificate. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│--ca-path <file> │ string │ Available if CONFIG_SIGNED_IMAGES │
│ │ │ is set. Path to the Certificate │
│ │ │ Authority (PEM). │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│--get-root │ │ Detect and print the root device │
│ │ │ and exit │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│--get-emmc- boot │ │ If devic eis an eMMC, reads the CSD │
│<device> │ │ register and detect from which boot │
│ │ │ device has booted (mmcblkXboot0 or │
│ │ │ mmxblkXboot1) │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-l <level> │ int │ Set loglevel. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-L │ │ Send LOG output to syslog (local). │
│ │ • │ │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-i <file> │ string │ Run SWUpdate with a local .swu │
│ │ │ file. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-n │ │ Run SWUpdate in dry-run mode. │
│ │ • │ │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-N <version> │ string │ The minimum required version of │
│ │ │ software. This will be checked │
│ │ │ with the version of new software │
│ │ │ and forbids downgrading. Version │
│ │ │ consists of either 4 numbers │
│ │ │ (major.minor.rev.build with each │
│ │ │ field in the range 0..65535) or it │
│ │ │ is a semantic version. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│ │ string │ The maximum required version of │
│ -max-version │ │ software. This will be checked │
│ <version> │ │ with the version of new software. │
│ │ │ Version consists of either 4 │
│ │ │ numbers (major.minor.rev.build with │
│ │ │ each field in the range 0..65535) │
│ │ │ or it is a semantic version. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-R <version> │ string │ The current installed version of │
│ │ │ software. This will be checked │
│ │ │ with the version of new software │
│ │ │ and forbids reinstalling. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-o <file> │ string │ Save the stream (SWU) to a file. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-s <file> │ string │ Save installed version info to a │
│ │ │ file. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-v │ │ Activate verbose output. │
│ │ • │ │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-M │ │ Disable setting the bootloader │
│ │ • │ transaction marker. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-m │ │ Disable setting the update state in │
│ │ • │ the bootloader. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-w <parms> │ string │ Available if CONFIG_WEBSERVER is │
│ │ │ set. Start internal webserver and │
│ │ │ pass to it a command line string. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-d <parms> │ string │ Available if CONFIG_DOWNLOAD is │
│ │ │ set. Start internal downloader │
│ │ │ client and pass to it a command │
│ │ │ line string. See below the │
│ │ │ internal command line arguments for │
│ │ │ the downloader. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-u <parms> │ string │ Available if CONFIG_SURICATTA is │
│ │ │ set. Start internal suricatta │
│ │ │ client daemon and pass to it a │
│ │ │ command line string. See below the │
│ │ │ internal command line arguments for │
│ │ │ suricatta. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-H <board:rev> │ string │ Available on │
│ │ │ CONFIG_HW_COMPATIBILITY set. Set │
│ │ │ board name and hardware revision. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-c │ │ Check *.swu file. It ensures that │
│ │ • │ files referenced in sw-description │
│ │ │ are present. Usage: swupdate -c -i │
│ │ │ <file> │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-P <cmd> │ string │ Execute pre-update command. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-p <cmd> │ string │ Execute post-update command. │
├─────────────────────────┼───────────┼─────────────────────────────────────┤
│-q <sel> │ string │ │
│ │ │ List for software images set │
│ │ │ and source │
│ │ │ that are accepted via │
│ │ │ IPC Ex.: stable,main │
│ │ │ it can be set │
│ │ │ multiple times │
└─────────────────────────┴───────────┴─────────────────────────────────────┘
Downloader command line parameters
Example: swupdate -d "-u example.com"
Mandatory arguments are marked with '*':
┌───────────────┬─────────┬──────────────────────────┐
│Parameter │ Type │ Description │
├───────────────┼─────────┼──────────────────────────┤
│-u <url> │ string │ * This is the URL where │
│ │ │ new software is pulled. │
│ │ │ URL is a link to a valid │
│ │ │ .swu image │
├───────────────┼─────────┼──────────────────────────┤
│-r <retries> │ integer │ Number of retries before │
│ │ │ a download is considered │
│ │ │ broken. With "-r 0", │
│ │ │ SWUpdate will not stop │
│ │ │ until a valid software │
│ │ │ is loaded. │
├───────────────┼─────────┼──────────────────────────┤
│-w <retrywait> │ integer │ Time to wait prior to │
│ │ │ retry and resume a │
│ │ │ download (default: 5s). │
├───────────────┼─────────┼──────────────────────────┤
│-t <timeout> │ integer │ Timeout for connection │
│ │ │ lost downloader or │
│ │ │ Webserver │
├───────────────┼─────────┼──────────────────────────┤
│-a <usr:pwd> │ string │ Send user and password │
│ │ │ for Basic Auth │
├───────────────┼─────────┼──────────────────────────┤
│-n <value> │ string │ Maximum download speed │
│ │ │ to be used. Value be │
│ │ │ specified in kB/s, B/s, │
│ │ │ MB/s or GB/s. Examples: │
│ │ │ -n 100k : Set limit to │
│ │ │ 100 kB/s. -n 500 : Set │
│ │ │ limit to 500 B/s. -n 2M │
│ │ │ : Set limit to 1 M/s. │
│ │ │ -n 1G : Set limit to 1 │
│ │ │ G/s. │
└───────────────┴─────────┴──────────────────────────┘
Suricatta command line parameters
Example: swupdate -u "-t default -u localhost:8080 -i 1B7"
Note that different suricatta modules may have different parameters. The below listed
options are for SWUpdate's hawkBit support.
Mandatory arguments are marked with '*':
┌────────────────────────┬───────────┬──────────────────────────┐
│Parameter │ Type │ Description │
├────────────────────────┼───────────┼──────────────────────────┤
│-t <tenant> │ string │ * Set hawkBit tenant ID │
│ │ │ for this device. │
├────────────────────────┼───────────┼──────────────────────────┤
│-u <url> │ string │ * Host and port of the │
│ │ │ hawkBit instance, e.g., │
│ │ │ localhost:8080 │
├────────────────────────┼───────────┼──────────────────────────┤
│-i <id> │ integer │ * The device ID to │
│ │ │ communicate to hawkBit. │
├────────────────────────┼───────────┼──────────────────────────┤
│-c <confirm> │ integer │ Confirm update status to │
│ │ │ server: 1=AGAIN, │
│ │ │ 2=SUCCESS, 3=FAILED │
├────────────────────────┼───────────┼──────────────────────────┤
│-x │ │ Do not abort on flawed │
│ │ • │ server certificates. │
├────────────────────────┼───────────┼──────────────────────────┤
│-p <polldelay> │ integer │ Delay in seconds between │
│ │ │ two hawkBit poll │
│ │ │ operations (default: │
│ │ │ 45s). │
├────────────────────────┼───────────┼──────────────────────────┤
│-r <retry> │ integer │ Resume and retry │
│ │ │ interrupted downloads │
│ │ │ (default: 5 tries). │
├────────────────────────┼───────────┼──────────────────────────┤
│-w <retrywait> │ integer │ Time to wait prior to │
│ │ │ retry and resume a │
│ │ │ download (default: 5s). │
├────────────────────────┼───────────┼──────────────────────────┤
│-y <proxy> │ string │ Use proxy. Either give │
│ │ │ proxy URL, else │
│ │ │ {http,all}_proxy env is │
│ │ │ tried. │
├────────────────────────┼───────────┼──────────────────────────┤
│-k <targettoken> │ string │ Set target token. │
├────────────────────────┼───────────┼──────────────────────────┤
│-g <gatewaytoken> │ string │ Set gateway token. │
├────────────────────────┼───────────┼──────────────────────────┤
│-f <interface> │ string │ Set the network │
│ │ │ interface to connect to │
│ │ │ hawkBit. │
├────────────────────────┼───────────┼──────────────────────────┤
│-e │ │ Daemon enabled at │
│ │ • │ startup (default). │
├────────────────────────┼───────────┼──────────────────────────┤
│-d │ │ Daemon disabled at │
│ │ • │ startup. │
├────────────────────────┼───────────┼──────────────────────────┤
│--disable-token-for-dwl │ │ Do not send │
│ │ • │ authentication header │
│ │ │ when downloading SWU. │
├────────────────────────┼───────────┼──────────────────────────┤
│--cache-file │ string │ This allows one to │
│ │ │ resume an update after a │
│ │ │ power cut. If the SWU is │
│ │ │ saved in a file, │
│ │ │ SWUpdate can reuse the │
│ │ │ file and download just │
│ │ │ the remaining part of │
│ │ │ the SWU. │
├────────────────────────┼───────────┼──────────────────────────┤
│-m <seconds> │ integer │ Delay in seconds between │
│ │ │ re-trying to send │
│ │ │ initial feedback │
│ │ │ specified with "-c" │
│ │ │ option. Default value is │
│ │ │ 10 seconds. If Suricatta │
│ │ │ is started with initial │
│ │ │ state of STATE_WAIT ("-c │
│ │ │ 6"), this value is │
│ │ │ ignored. │
├────────────────────────┼───────────┼──────────────────────────┤
│-s <seconds> │ integer │ Connection timeout to │
│ │ │ use in seconds. If user │
│ │ │ doesn't set this option, │
│ │ │ default libcurl │
│ │ │ connection timeout value │
│ │ │ of 300 seconds is used. │
│ │ │ NOTE: it is not possible │
│ │ │ for Suricatta to respond │
│ │ │ to external program API │
│ │ │ requests during this │
│ │ │ period - adapt this │
│ │ │ value to your use case! │
├────────────────────────┼───────────┼──────────────────────────┤
│-a <name> <value> │ strings │ Custom HTTP header with │
│ │ │ given name and value to │
│ │ │ be sent with every HTTP │
│ │ │ request made. │
├────────────────────────┼───────────┼──────────────────────────┤
│-n <value> │ string │ Maximum download speed │
│ │ │ to be used. Value be │
│ │ │ specified in kB/s, B/s, │
│ │ │ MB/s or GB/s. Examples: │
│ │ │ -n 100k : Set limit to │
│ │ │ 100 kB/s. -n 500 : Set │
│ │ │ limit to 500 B/s. -n 2M │
│ │ │ : Set limit to 1 M/s. │
│ │ │ -n 1G : Set limit to 1 │
│ │ │ G/s. │
└────────────────────────┴───────────┴──────────────────────────┘
Webserver command line parameters
Example: swupdate -w "-r /www -p 8080"
Mandatory arguments are marked with '*':
┌──────────────────────────┬─────────┬──────────────────────────┐
│Parameter │ Type │ Description │
├──────────────────────────┼─────────┼──────────────────────────┤
│-r <document root> │ string │ * Path where the web app │
│ │ │ is stored. │
├──────────────────────────┼─────────┼──────────────────────────┤
│-p <port> │ integer │ * TCP port to be │
│ │ │ listened if not set, │
│ │ │ 8080 is used │
├──────────────────────────┼─────────┼──────────────────────────┤
│-s <ssl> │ │ * Enable SSL support. │
│ │ │ Note: it must be │
│ │ │ configured with │
│ │ │ CONFIG_MONGOOSESSL │
├──────────────────────────┼─────────┼──────────────────────────┤
│--ssl-cert <cert> │ string │ Path to the certificate │
│ │ │ to present to clients │
├──────────────────────────┼─────────┼──────────────────────────┤
│-K <key> │ string │ Path to key │
│ │ │ corresponding to ssl │
│ │ │ certificate │
├──────────────────────────┼─────────┼──────────────────────────┤
│-t <timeout> │ integer │ Timeout to consider a │
│ │ │ connection lost if │
│ │ │ clients stops to send │
│ │ │ data. If hit, an update │
│ │ │ is aborted. Default=0 │
│ │ │ (unlimited) │
├──────────────────────────┼─────────┼──────────────────────────┤
│--auth-domain <string> │ string │ Set authentication │
│ │ │ domain Default: none │
├──────────────────────────┼─────────┼──────────────────────────┤
│ │ string │ Set authentication file │
│ --global-auth-file │ │ if any Default: none │
│ <string> │ │ │
└──────────────────────────┴─────────┴──────────────────────────┘
systemd Integration
SWUpdate has optional systemd support via the compile-time configuration switch
CONFIG_SYSTEMD. If enabled, SWUpdate signals systemd about start-up completion and can
make optional use of systemd's socket-based activation feature.
A sample systemd service unit file /etc/systemd/system/swupdate.service may look like the
following starting SWUpdate in suricatta daemon mode:
[Unit]
Description=SWUpdate daemon
Documentation=https://github.com/sbabic/swupdate
Documentation=https://sbabic.github.io/swupdate
[Service]
Type=notify
RuntimeDirectory=swupdate
RuntimeDirectoryPreserve=yes
ExecStart=/usr/bin/swupdate -u '-t default -u http://localhost -i 25'
[Install]
WantedBy=multi-user.target
Started via systemctl start swupdate.service, SWUpdate (re)creates its sockets on startup.
For using socket-based activation, an accompanying systemd socket unit file
/etc/systemd/system/swupdate.socket is required:
[Unit]
Description=SWUpdate socket listener
Documentation=https://github.com/sbabic/swupdate
Documentation=https://sbabic.github.io/swupdate
[Socket]
ListenStream=/run/swupdate/sockinstctrl
ListenStream=/run/swupdate/swupdateprog
[Install]
WantedBy=sockets.target
On swupdate.socket being started, systemd creates the socket files and hands them over to
SWUpdate when it starts. So, for example, when talking to /run/swupdate/swupdateprog,
systemd starts swupdate.service and hands-over the socket files. The socket files are also
handed over on a "regular" start of SWUpdate via systemctl start swupdate.service.
Note, that all dependent services need to access the swupdate sockets via the paths
specified in the swupdate.socket systemd unit.
Changes in boot-loader code
The SWUpdate consists of kernel and a root filesystem (image) that must be started by the
boot-loader. In case using U-Boot, the following mechanism can be implemented:
• U-Boot checks if a sw update is required (check gpio, serial console, etc.).
• the script "altbootcmd" sets the rules to start SWUpdate
• in case SWUpdate is required, U-boot run the script "altbootcmd"
Is it safe to change U-Boot environment ? Well, it is, but U-Boot must be configured
correctly. U-Boot supports two copies of the environment to be power-off safe during an
environment update. The board's configuration file must have defined
CONFIG_ENV_OFFSET_REDUND or CONFIG_ENV_ADDR_REDUND. Check in U-Boot documentation for
these constants and how to use them.
There are a further enhancement that can be optionally integrated into U-boot to make the
system safer. The most important I will suggest is to add support for boot counter in
U-boot (documentation is in U-Boot docs). This allows U-Boot to track for attempts to
successfully run the application, and if the boot counter is greater as a limit, can start
automatically SWUpdate to replace a corrupt software.
GRUB by default does not support double copies of environment as in case of U-Boot. This
means that there is possibility that environment block get's corrupted when power-off
occurs during environment update. To minimize the risk, we are not modifying original
environment block. Variables are written into temporary file and after successful
operation rename instruction is called.
Image File Format
SWUpdate uses cpio as image file format because it is a simple, well-established, and
streamable format. More specifically, the New ASCII format (header magic number 070701)
and the New CRC format (header magic number 070702) are supported. Both formats are
essentially equivalent with the New CRC format additionally having set the cpio header
field check to the least-significant 32 bits of the sum of all (unsigned) data bytes. This
checksum is verified by SWUpdate. If this verification fails, SWUpdate yields an error
like the following:
Checksum WRONG ! Computed 0xfa11ed00, it should be 0xffffffff
Note that there's artifact sha256 verification available (see CONFIG_HASH_VERIFY) which is
recommended over relying on cpio's checksum facility.
For both cpio formats, the New ASCII as well as the New CRC format, the cpio file size is
limited to 32 Bit, i.e., 4 GB.
Building a single image
cpio is used as container format because of its simplicity and its ability to be streamed.
The meta information file sw-description (default, see CONFIG_SETSWDESCRIPTION) describing
the images in the container must be the first file in the cpio archive. The images follow
it, in any order.
To produce an image, a script like the following can be used:
CONTAINER_VER="1.0"
PRODUCT_NAME="my-software"
FILES="sw-description image1.ubifs \
image2.gz.u-boot uImage.bin myfile sdcard.img"
for i in $FILES;do
echo $i;done | cpio -ov -H crc > ${PRODUCT_NAME}_${CONTAINER_VER}.swu
Alternatively, swugenerator may be used to generate the image.
The generated image can be checked by running the following command:
swupdate -c -i my-software_1.0.swu
Support of compound image
A single image can be built automatically inside Yocto. meta-swupdate extends the classes
with the swupdate class. A recipe should inherit it and add an own sw-description file to
generate the image.
Alternatively, swugenerator may be used to generate compound images outside Yocto.
UPDATE STRATEGY EXAMPLES
SWUpdate is a building block and it allows one to design and implementing its own update
strategy. Even if many projects have common ways for updating, it is possible to high
customize the update for each project. The most common strategies (single-copy and
dual-copy) were already described at the beginning of this documentation and of course are
well supported in SWUpdate.
Single copy - running as standalone image
See Single copy - running as standalone image.
Double copy with fall-back
See Double copy with fall-back.
Combine double-copy with rescue system
This provides a recovery procedure to cover update failure in severe cases when software
is damaged. In case none of the copy can be started, the bootloader will start the rescue
system (possibly stored on another storage as the main system) to try to rescue the board.
[image]
The rescue system can be updated as well during a standard update.
Split system update with application update
Updating a whole image is quite straightforward, but this means to transfer bigger amount
of data if just a few files are updated. It is possible to split the update in several
smaller parts to reduce the transfer size. This requires a special care to take care of
compatibility between system and application, that can be solved with customized Lua
scripts in the sw-description file. SWUpdate supports versioning for each artefact, and
anyone can add own rules to verify compatibility between components. [image]
Configuration update
Thought to update the software, SWUpdate can be used to install configuration data as
well. Build system can create configuration SWU with files / data for the configuration
of the system. There is no requirements what these SWU should contains - it is duty of
the integrator to build them and make them suitable for his own project. Again,
configuration data can be updated as separate process using one of the above scenarios.
SWUPDATE: SYNTAX AND TAGS WITH THE DEFAULT PARSER
Introduction
SWUpdate uses the library "libconfig" as default parser for the image description.
However, it is possible to extend SWUpdate and add an own parser, based on a different
syntax and language as the one supported by libconfig. In the examples directory there is
the code for a parser written in Lua, with the description in XML.
Using the default parser, sw-description follows the syntax rules described in the
libconfig manual. Please take a look at ‐
http://www.hyperrealm.com/libconfig/libconfig_manual.html for an explanation of basic
types. The whole description must be contained in the sw-description file itself: using
of the #include directive is not allowed by SWUpdate.
The following example explains the implemented tags:
software =
{
version = "0.1.0";
description = "Firmware update for XXXXX Project";
hardware-compatibility: [ "1.0", "1.2", "1.3"];
/* partitions tag is used to resize UBI partitions */
partitions: ( /* UBI Volumes */
{
name = "rootfs";
device = "mtd4";
size = 104896512; /* in bytes */
},
{
name = "data";
device = "mtd5";
size = 50448384; /* in bytes */
}
);
images: (
{
filename = "rootfs.ubifs";
volume = "rootfs";
},
{
filename = "swupdate.ext3.gz.u-boot";
volume = "fs_recovery";
},
{
filename = "sdcard.ext3.gz";
device = "/dev/mmcblk0p1";
compressed = "zlib";
},
{
filename = "bootlogo.bmp";
volume = "splash";
},
{
filename = "uImage.bin";
volume = "kernel";
},
{
filename = "fpga.txt";
type = "fpga";
},
{
filename = "bootloader-env";
type = "bootloader";
}
);
files: (
{
filename = "README";
path = "/README";
device = "/dev/mmcblk0p1";
filesystem = "vfat"
}
);
scripts: (
{
filename = "erase_at_end";
type = "lua";
},
{
filename = "display_info";
type = "lua";
}
);
bootenv: (
{
name = "vram";
value = "4M";
},
{
name = "addfb";
value = "setenv bootargs ${bootargs} omapfb.vram=1:2M,2:2M,3:2M omapdss.def_disp=lcd"
}
);
}
The first tag is "software". The whole description is contained in this tag. It is
possible to group settings per device by using Board specific settings.
Handling configuration differences
The concept can be extended to deliver a single image containing the release for multiple
devices. Each device has its own kernel, dtb, and root filesystem, or they can share some
parts.
Currently this is managed (and already used in a real project) by writing an own parser,
that checks which images must be installed after recognizing which is the device where
software is running.
Because the external parser can be written in Lua and it is completely customizable,
everybody can set his own rules. For this specific example, the sw-description is written
in XML format, with tags identifying the images for each device. To run it, the liblxp
library is needed.
<?xml version="1.0" encoding="UTF-8"?>
<software version="1.0">
<name>Update Image</name>
<version>1.0.0</version>
<description>Firmware for XXXXX Project</description>
<images>
<image device="firstdevice" version="0.9">
<stream name="dev1-uImage" type="ubivol" volume="kernel" />
<stream name="dev1.dtb" type="ubivol" volume="dtb" />
<stream name="dev1-rootfs.ubifs" type="ubivol" volume="rootfs"/>
<stream name="dev1-uboot-env" type="uboot" />
<stream name="raw_vfat" type="raw" dest="/dev/mmcblk0p4" />
<stream name="sdcard.lua" type="lua" />
</image>
<image device="seconddevice" version="0.9">
<stream name="dev2-uImage" type="ubivol" volume="kernel" />
<stream name="dev2.dtb" rev="0.9" type="ubivol" volume="dtb" />
<stream name="dev2-rootfs.ubifs" type="ubivol" volume="rootfs"/>
</image>
</images>
</software>
The parser for this is in the /examples directory. By identifying which is the running
device, the parser return a table containing the images that must be installed and their
associated handlers. By reading the delivered image, SWUpdate will ignore all images that
are not in the list processed by the parser. In this way, it is possible to have a single
delivered image for the update of multiple devices.
Multiple devices are supported by the default parser, too.
software =
{
version = "0.1.0";
target-1 = {
images: (
{
...
}
);
};
target-2 = {
images: (
{
...
}
);
};
}
In this way, it is possible to have a single image providing software for each device you
have.
By default, the hardware information is extracted from /etc/hwrevision file. The file
should contain a single line in the following format:
<boardname> <revision>
Where:
• <revision> will be used for matching with hardware compatibility list
• <boardname> can be used for grouping board specific settings
Software collections
Software collections and operation modes can be used to implement a dual copy strategy.
The simplest case is to define two installation locations for the firmware image and call
SWUpdate selecting the appropriate image.
software =
{
version = "0.1.0";
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
};
}
In this way it is possible to specify that copy-1 gets installed to /dev/mtd4, while
copy-2 to /dev/mtd5. By properly selecting the installation locations, SWUpdate will
update the firmware in the other slot.
The method of image selection is out of the scope of SWUpdate and user is responsible for
calling SWUpdate passing proper settings.
Priority finding the elements in the file
SWUpdate search for entries in the sw-description file according to the following
priority:
1. Try <boardname>.<selection>.<mode>.<entry>
2. Try <selection>.<mode>.<entry>
3. Try <boardname>.<entry>
4. Try <entry>
Take an example. The following sw-description describes the release for a set of boards.
software =
{
version = "0.1.0";
myboard = {
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
}
}
stable = {
copy-1: {
images: (
{
device = "/dev/mtd6"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd7"
...
}
);
}
}
}
On myboard, SWUpdate searches and finds myboard.stable.copy1(2). When running on different
boards, SWUpdate does not find an entry corresponding to the boardname and it falls back
to the version without boardname. This allows to realize the same release for different
boards having a completely different hardware. myboard could have an eMMC and an ext4
filesystem, while another device can have raw flash and install an UBI filesystem.
Nevertheless, they are both just a different format of the same release and they could be
described together in sw-description. It is important to understand the priorities how
SWUpdate scans for entries during the parsing.
Using links
sw-description can become very complex. Let's think to have just one board, but in
multiple hw revision and they differ in Hardware. Some of them can be grouped together,
some of them require a dedicated section. A way (but not the only one !) could be to add
mode and selects the section with -e stable,<rev number>.
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.3", "3.0", "3.1"];
rev-1.0: {
images: (
...
);
scripts: (
...
);
}
rev-1.2: {
hardware-compatibility: ["1.2"];
images: (
...
);
scripts: (
...
);
}
rev-2.0: {
hardware-compatibility: ["2.0"];
images: (
...
);
scripts: (
...
);
}
rev-1.3: {
hardware-compatibility: ["1.3"];
images: (
...
);
scripts: (
...
);
}
rev-3.0:
{
hardware-compatibility: ["3.0"];
images: (
...
);
scripts: (
...
);
}
rev-3.1:
{
hardware-compatibility: ["3.1"];
images: (
...
);
scripts: (
...
);
}
}
}
}
If each of them requires an own section, it is the way to do. Anyway, it is more probable
than revisions can be grouped together, for example board with the same major revision
number could have the same installation instructions. This leads in the example to 3
groups for rev1.X, rev2.X, and rev3.X. Links allow one to group section together. When a
"ref" is found when SWUpdate searches for a group (images, files, script, bootenv), it
replaces the current path in the tree with the value of the string. In this way, the
example above can be written in this way:
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.3", "3.0", "3.1"];
rev-1x: {
images: (
...
);
scripts: (
...
);
}
rev1.0 = {
ref = "#./rev-1x";
}
rev1.2 = {
ref = "#./rev-1x";
}
rev1.3 = {
ref = "#./rev-1x";
}
rev-2x: {
images: (
...
);
scripts: (
...
);
}
rev2.0 = {
ref = "#./rev-2x";
}
rev-3x: {
images: (
...
);
scripts: (
...
);
}
rev3.0 = {
ref = "#./rev-3x";
}
rev3.1 = {
ref = "#./rev-3x";
}
}
}
}
The link can be absolute or relative. The keyword "ref" is used to indicate a link. If
this is found, SWUpdate will traverse the tree and replaces the current path with the
values find in the string pointed by "ref". There are simple rules for a link:
• it must start with the character '#'
• "." points to the current level in the tree, that means the parent of "ref"
• ".." points to the parent level in the tree
• "/" is used as filed separator in the link
A relative path has a number of leading "../" to move the current cursor to the parent
leaf of the tree. In the following example, rev40 sets a link to a "common" section,
where images is found. This is sets via a link, too, to a section in the parent node. The
path software.myboard.stable.common.images is then replaced by
software.myboard.stable.trythis
software =
{
version = {
ref = "#./commonversion";
}
hardware-compatibility = ["rev10", "rev11", "rev20"];
commonversion = "0.7-linked";
pc:{
stable:{
common:{
images =
{
ref = "#./../trythis";
}
};
trythis:(
{
filename = "rootfs1.ext4";
device = "/dev/mmcblk0p8";
type = "raw";
} ,
{
filename = "rootfs5.ext4";
device = "/dev/mmcblk0p7";
type = "raw";
}
);
pdm3rev10:
{
images:(
{
filename = "rootfs.ext3"; device = "/dev/mmcblk0p2";}
);
uboot:(
{ name = "bootpart";
value = "0:2";}
);
};
pdm3rev11 =
{
ref = "#./pdm3rev10";
}
pdm3rev20 =
{
ref = "#./pdm3rev10";
}
pdm3rev40 =
{
ref = "#./common";
}
};
};
}
Each entry in sw-description can be redirect by a link as in the above example for the
"version" attribute.
hardware-compatibility
hardware-compatibility: [ "major.minor", "major.minor", ... ]
This entry lists the hardware revisions that are compatible with this software image.
Example:
hardware-compatibility: [ "1.0", "1.2", "1.3"];
This defines that the software is compatible with HW-Revisions 1.0, 1.2, and 1.3, but not
with 1.1 or any other version not explicitly listed here. In the above example,
compatibility is checked by means of string comparison. If the software is compatible with
a large number of hardware revisions, it may get cumbersome to enumerate all compatible
versions. To allow more compact specifications, regular expressions (POSIX extended) can
be used by adding a prefix #RE: to the entry. Rewriting the above example would yield:
hardware-compatibility: [ "#RE:^1\.[023]$" ];
It is in the responsibility of the respective project to find the revision of the board on
which SWUpdate is running. No assumptions are made about how the revision can be obtained
(GPIOs, EEPROM,..) and each project is free to select the most appropriate way. In the end
the result must be written to the file /etc/hwrevision (or in another file if specified as
configuration option) before SWUpdate is started.
partitions : UBI layout
This tag allows one to change the layout of UBI volumes. Please take care that MTDs are
not touched and they are configured by the Device Tree or in another way directly in
kernel.
partitions: (
{
name = <volume name>;
size = <size in bytes>;
device = <MTD device>;
}
);
All fields are mandatory. SWUpdate searches for a volume of the given name and if
necessary adjusts size or type (see below). If no volume with the given name is found, a
new volume is created on the UBI device attached to the MTD device given by device. device
can be specified by number (e.g. "mtd4") or by name (the name of the MTD device, e.g.
"ubi_partition"). The UBI device is attached automatically.
The default behavior of swupdate is to create a dynamic UBI volume. To create a static
volume, add a line data = "static"; to the respective partition entry.
If a size of 0 is given, the volume will be deleted if it exists. This can be used to
remove orphan volumes possibly created by older software versions which are not required
anymore.
images
The tag "images" collects the image that are installed to the system. The syntax is:
images: (
{
filename[mandatory] = <Name in CPIO Archive>;
volume[optional] = <destination volume>;
device[optional] = <destination volume>;
mtdname[optional] = <destination mtd name>;
type[optional] = <handler>;
/* optionally, the image can be copied at a specific offset */
offset[optional] = <offset>;
/* optionally, the image can be compressed if it is in raw mode */
compressed;
},
/* Next Image */
.....
);
volume is only used to install the image in a UBI volume. volume and device cannot be used
at the same time. If device is set, the raw handler is automatically selected.
The following example is to update a UBI volume:
{
filename = "core-image-base.ubifs";
volume = "rootfs";
}
To update an image in raw mode, the syntax is:
{
filename = "core-image-base.ext3";
device = "/dev/mmcblk0p1";
}
To flash an image at a specific offset, the syntax is:
{
filename = "u-boot.bin";
device = "/dev/mmcblk0p1";
offset = "16K";
}
The offset handles the following multiplicative suffixes: K=1024 and M=1024*1024.
However, writing to flash in raw mode must be managed in a special way. Flashes must be
erased before copying, and writing into NAND must take care of bad blocks and ECC errors.
For these reasons, the handler "flash" must be selected:
For example, to copy the kernel into the MTD7 of a NAND flash:
{
filename = "uImage";
device = "mtd7";
type = "flash";
}
The filename is mandatory. It is the Name of the file extracted by the stream. volume is
only mandatory in case of UBI volumes. It should be not used in other cases.
Alternatively, for the handler “flash”, the mtdname can be specified, instead of the
device name:
{
filename = "uImage";
mtdname = "kernel";
type = "flash";
}
Files
It is possible to copy single files instead of images. This is not the preferred way, but
it can be used for debugging or special purposes.
files: (
{
filename = <Name in CPIO Archive>;
path = <path in filesystem>;
device[optional] = <device node >;
filesystem[optional] = <filesystem for mount>;
properties[optional] = {create-destination = "true";}
}
);
Entries in "files" section are managed as single files. The attributes "filename" and
"path" are mandatory. Attributes "device" and "filesystem" are optional; they tell
SWUpdate to mount device (of the given filesystem type, e.g. "ext4") before copying
"filename" to "path". Without "device" and "filesystem", the "filename" will be copied to
"path" in the current rootfs.
As a general rule, swupdate doesn't copy out a file if the destination path doesn't
exists. This behavior could be changed using the special property "create-destination".
As another general rule, the raw file handler installs the file directly to the specified
path. If the target file already exists and the raw file handler is interrupted, the
existing file may be replaced by an empty or partially written file. A use case can exist
where having an empty or corrupted file is worse than the existing file. For this reason,
the raw file handler supports an "atomic-install" property. Setting the property to "true"
installs the file to the specified path with ".tmp" appended to the filename. Once the
contents of the file have been written and the buffer is flushed, the ".tmp" file is
renamed to the target file. This minimizes chances that an empty or corrupted file is
created by an interrupted raw file handler.
Scripts
Scripts runs in the order they are put into the sw-description file. The result of a
script is valuated by SWUpdate, that stops the update with an error if the result is <> 0.
They are copied into a temporary directory before execution and their name must be unique
inside the same cpio archive.
If no type is given, SWUpdate default to "lua". Please note that running a shell script
opens a set of different security issues, check also chapter "Best practice".
Lua
scripts: (
{
filename = <Name in CPIO Archive>;
type = "lua";
}
);
Lua scripts are run using the internal interpreter.
They must have at least one of the following functions:
function preinst()
SWUpdate scans for all scripts and check for a preinst function. It is called before
installing the images.
function postinst()
SWUpdate scans for all scripts and check for a postinst function. It is called after
installing the images.
function postfailure()
Only in case an update fails, SWUpdate scans for all scripts and check for a postfailure
function. This could be useful in case it is necessary to restore a previous state, for
example, in case the application was stop, it should run again.
shellscript
SWUpdate will run the binary shell "/bin/sh" to execute the script.
scripts: (
{
filename = <Name in CPIO Archive>;
type = "shellscript";
}
);
Shell scripts are called by forking the process and running the shell as /bin/sh.
SWUpdate scans for all scripts and calls them before and after installing the images.
SWUpdate passes 'preinst', 'postinst' or 'postfailure' as first argument to the script.
If the data attribute is defined, its value is passed as the last argument(s) to the
script.
preinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "preinstall";
}
);
preinstall are shell scripts and called via system command. SWUpdate scans for all
scripts and calls them before installing the images. If the data attribute is defined,
its value is passed as the last argument(s) to the script.
Note that cannot be ensured that preinstall scripts run before an artifact is installed in
streaming mode. In fact, if streaming is activated, the artifact must be installed as soon
as it is received from network because there is no temporary copy. Because there is no fix
order in the SWU, an artifact can be packed before any script in the SWU. The right way is
to write an "embedded-script" in Lua inside sw-description: because it becomes part of
sw-description, it runs when sw-description is parsed and before any handler runs, even
before a partition handler.
postinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "postinstall";
}
);
postinstall are shell scripts and called via system command. SWUpdate scans for all
scripts and calls them after installing the images. If the data attribute is defined, its
value is passed as the last argument(s) to the script.
Update Transaction and Status Marker
By default, SWUpdate sets the bootloader environment variable "recovery_status" to
"in_progress" prior to an update operation and either unsets it or sets it to "failed"
after the update operation. This is an interface for SWUpdate-external tooling: If there
is no "recovery_status" variable in the bootloader's environment, the update operation has
been successful. Else, if there is a "recovery_status" variable with the value "failed",
the update operation has not been successful.
While this is in general essential behavior for firmware updates, it needn't be for less
critical update operations. Hence, whether or not the update transaction marker is set by
SWUpdate can be controlled by the boolean switch "bootloader_transaction_marker" which is
global per sw-description file. It defaults to true. The following example snippet
disables the update transaction marker:
software =
{
version = "0.1.0";
bootloader_transaction_marker = false;
...
It is also possible to disable setting of the transaction marker entirely (and
independently of the setting in sw-description) by starting SWUpdate with the -M option.
The same applies to setting the update state in the bootloader via its environment
variable "ustate" (default) to STATE_INSTALLED=1 or STATE_FAILED=3 after an installation.
This behavior can be turned off globally via the -m option to SWUpdate or per
sw-description via the boolean switch "bootloader_state_marker".
reboot flag
It is possible to signal that a reboot for a specific update is not required. This
information is evaluated by SWUpdate just to inform a backend about the transaction
result. If a postinstall script (command line parameter -p) is passed at the startup to
perform a reboot, it will be executed anyway because SWUpdate cannot know the nature of
this script.
SWUpdate sends this information to the progress interface and it is duty of the listeners
to interpret the information. The attribute is a boolean:
reboot = false;
Attribute belongs to the general section, where also version belongs. It is not required
to activate the flag with reboot = true because it is the default behavior, so just
disabling makes sense.
The tool swupdate-progress interprets the flag: if it was started with reboot support (-r
parameter), it checks if a "no-reboot" message is received and disables to reboot the
device for this specific update. When the transaction completes, the reboot feature is
activated again in case a new update will require to reboot the device. This allows to
have on the fly updates, where not the whole software is updated and a reboot is not
required.
bootloader
There are two ways to update the bootloader (currently U-Boot, GRUB, and EFI Boot Guard)
environment. First way is to add a file with the list of variables to be changed and
setting "bootloader" as type of the image. This informs SWUpdate to call the bootloader
handler to manage the file (requires enabling bootloader handler in configuration). There
is one bootloader handler for all supported bootloaders. The appropriate bootloader must
be chosen from the bootloader selection menu in menuconfig.
images: (
{
filename = "bootloader-env";
type = "bootloader";
}
)
The format of the file is described in U-boot documentation. Each line is in the format
<name of variable>=<value>
if value is missing, the variable is unset.
The format is compatible with U-Boot "env import" command. It is possible to produce the
file from target as result of "env export".
Comments are allowed in the file to improve readability, see this example:
# Default variables
bootslot=0
board_name=myboard
baudrate=115200
## Board Revision dependent
board_revision=1.0
The second way is to define in a group setting the variables that must be changed:
bootenv: (
{
name = <Variable name>;
value = <Variable value>;
}
)
SWUpdate will internally generate a script that will be passed to the bootloader handler
for adjusting the environment.
For backward compatibility with previously built .swu images, the "uboot" group name is
still supported as an alias. However, its usage is deprecated.
SWUpdate persistent variables
Not all updates require to inform the bootloader about the update, and in many cases a
reboot is not required. There are also cases where changing bootloader's environment is
unwanted due to restriction for security. SWUpdate needs then some information after new
software is running to understand if everything is fine or some actions like a fallback
are needed. SWUpdate can store such as information in variables (like shell variables),
that can be stored persistently. The library libubootenv provide a way for saving such
kind as database in a power-cut safe mode. It uses the algorythm originally implemented
in the U-Boot bootloader. It is then guaranteed that the system will always have a valid
instance of the environment. The library supports multiple environment databases at the
same time, identifies with namespaces. SWUpdate should be configured to set the namespace
used for own variables. This is done by setting the attribute namespace-vars in the
runtime configuration file (swupdate.cfg). See also example/configuration/swupdate.cfg for
details.
The format is the same used with bootloader for single variable:
vars: (
{
name = <Variable name>;
value = <Variable value>;
}
)
SWUpdate will set these variables all at once like the bootloader variables. These
environment is stored just before writing the bootloader environment, that is always the
last step in an update.
Board specific settings
Each setting can be placed under a custom tag matching the board name. This mechanism can
be used to override particular setting in board specific fashion.
Assuming that the hardware information file /etc/hwrevision contains the following entry:
my-board 0.1.0
and the following description:
software =
{
version = "0.1.0";
my-board = {
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
}
SWUpdate will set bootpart to 0:2 in bootloader's environment for this board. For all
other boards, bootpart will be set to 0:1. Board specific settings take precedence over
default scoped settings.
Software collections and operation modes
Software collections and operations modes extend the description file syntax to provide an
overlay grouping all previous configuration tags. The mechanism is similar to Board
specific settings and can be used for implementing a dual copy strategy or delivering both
stable and unstable images within a single update file.
The mechanism uses a custom user-defined tags placed within software scope. The tag names
must not be any of: version, hardware-compatibility, uboot, bootenv, files, scripts,
partitions, images
An example description file:
software =
{
version = "0.1";
hardware-compatibility = [ "revA" ];
/* differentiate running image modes/sets */
stable:
{
main:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p2";
}
);
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
alt:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p1";
}
);
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
};
};
}
The configuration describes a single software collection named stable. Two distinct image
locations are specified for this collection: /dev/mmcblk0p1 and /dev/mmcblk0p2 for main
mode and alt mode respectively.
This feature can be used to implement a dual copy strategy by specifying the collection
and mode explicitly.
Versioning schemas in SWUpdate
SWUpdate can perform version comparisons for the whole Software by checking the version
attribute in the common part of sw-description and / or for single artifacts. SWUpdate
supports two different version schemas, and they must be followed if version comparison is
requested.
Numbering schema (default)
SWUpdate supports a version based on the schema:
<major>.<minor>.<revision>.<build>
where each field is a plain number (no alphanumeric) in the range 0..65535. User can add
further fields using the dot separator, but they are not considered for version
comparison. SWUpdate will check if a version number is set according to this rule and fall
back to semantic version upon failure. The version is converted to a 64 bit number (each
field is 16 bit) and compared against the running version of the same artifact.
Please consider that, because additional fields are descriptive only, for the comparison
they are not considered. This example contains version numbers that are interpreted as the
same version number by SWUpdate:
1.2.3.4
1.2.3.4.5
1.2.3.4.5.6
But the following is different:
1.2.3.4-alpha
And it is treated as semantic version.
Semantic version
SWUpdate supports semantic version. See official documentation for more details.
Checking version of installed software
SWUpdate can optionally verify if a sub-image is already installed and, if the version to
be installed is exactly the same, it can skip to install it. This is very useful in case
some high risky image should be installed or to speed up the upgrade process. One case is
if the bootloader needs to be updated. In most time, there is no need to upgrade the
bootloader, but practice showed that there are some cases where an upgrade is strictly
required - the project manager should take the risk. However, it is nicer to have always
the bootloader image as part of the .swu file, allowing to get the whole distro for the
device in a single file, but the device should install it just when needed.
SWUpdate searches for a file (/etc/sw-versions is the default location) containing all
versions of the installed images. This must be generated before running SWUpdate. The
file must contain pairs with the name of image and version, as:
<name of component> <version>
In sw-description, the optional attributes "name", "version", and "install-if-different"
provide the connection. Name and version are then compared with the data in the versions
file. install-if-different is a boolean that enables the check for this image. It is then
possible to check the version just for a subset of the images to be installed.
If used with "install-if-different", then version can be any string. For example:
bootloader 2015.01-rc3-00456-gd4978d
kernel 3.17.0-00215-g2e876af
There is also an attribute "install-if-higher" that checks if the version of the new
software is higher than the version of the installed software. If it's false, the new
software isn't installed. The goal is to avoid installing an older version of software.
In this case, version can be any of 2 formats. Either the version consists of up to 4
numbers in the range 0..65535 separated by a dot, e.g. <major>.<minor>.<rev>.<build>, or
it is a semantic version.
bootloader 2018.03.01
kernel 3.17.0-pre1+g2e876af
rfs 0.17-foo3.bar5+2020.07.01
app 1.7
It is advised not to mix version formats! Semantic versions only support 3 numbers (major,
minor, patch) and the fourth number will be silently dropped if present.
Embedded Script
It is possible to embed a script inside sw-description. This is useful in a lot of
conditions where some parameters are known just by the target at runtime. The script is
global to all sections, but it can contain several functions that can be specific for each
entry in the sw-description file.
These attributes are used for an embedded-script:
embedded-script = "<Lua code>"
It must be taken into account that the parser has already run and usage of double quotes
can interfere with the parser. For this reason, each double quote in the script must be
escaped.
That means a simple Lua code as:
print ("Test")
must be changed to:
print (\"Test\")
If not, the parser thinks to have the closure of the script and this generates an error.
See the examples directory for examples how to use it. Any entry in files or images can
trigger one function in the script. The "hook" attribute tells the parser to load the
script and to search for the function pointed to by the hook attribute. For example:
files: (
{
filename = "examples.tar";
type = "archive";
path = "/tmp/test";
hook = "set_version";
preserve-attributes = true;
}
);
After the entry is parsed, the parser runs the Lua function pointed to by hook. If Lua is
not activated, the parser raises an error because a sw-description with an embedded script
must be parsed, but the interpreter is not available.
Each Lua function receives as parameter a table with the setup for the current entry. A
hook in Lua is in the format:
function lua_hook(image)
image is a table where the keys are the list of available attributes. If an attribute
contains a "-", it is replaced with "_", because "-" cannot be used in Lua. This means,
for example, that:
install-if-different ==> install_if_different
installed-directly ==> installed_directly
Attributes can be changed in the Lua script and values are taken over on return. The Lua
function must return 2 values:
• a boolean, to indicate whether the parsing was correct
• the image table or nil to indicate that the image should be skipped
Example:
function set_version(image)
print (\"RECOVERY_STATUS.RUN: \".. swupdate.RECOVERY_STATUS.RUN)
for k,l in pairs(image) do
swupdate.trace(\"image[\" .. tostring(k) .. \"] = \" .. tostring(l))
end
image.version = \"1.0\"
image.install_if_different = true
return true, image
end
The example sets a version for the installed image. Generally, this is detected at runtime
reading from the target.
Attribute reference
There are 4 main sections inside sw-description:
• images: entries are images and SWUpdate has no knowledge about them.
• files: entries are files, and SWUpdate needs a filesystem for them. This is generally
used to expand from a tar-ball or to update single files.
• scripts: all entries are treated as executables, and they will be run twice (as pre- and
post- install scripts).
• bootenv: entries are pair with bootloader environment variable name and its value.
Attributes in sw-description
┌─────────────────────┬────────────────────┬─────────────────────┬─────────────────────┐
│Name │ Type │ Applies to │ Description │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│filename │ string │ images files │ filename as found │
│ │ │ scripts │ in the cpio archive │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│volume │ string │ images │ Just if type = │
│ │ │ │ "ubivol". UBI │
│ │ │ │ volume where image │
│ │ │ │ must be installed. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│ubipartition │ string │ images │ Just if type = │
│ │ │ │ "ubivol". Volume to │
│ │ │ │ be created or │
│ │ │ │ adjusted with a new │
│ │ │ │ size │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│device │ string │ images files │ devicenode as found │
│ │ │ │ in /dev or a │
│ │ │ │ symlink to it. Can │
│ │ │ │ be specified as │
│ │ │ │ absolute path or a │
│ │ │ │ name in /dev folder │
│ │ │ │ For example if │
│ │ │ │ /dev/mtd-dtb is a │
│ │ │ │ link to /dev/mtd3 │
│ │ │ │ "mtd3", "mtd-dtb", │
│ │ │ │ "/dev/mtd3" and │
│ │ │ │ "/dev/mtd-dtb" are │
│ │ │ │ valid names. Usage │
│ │ │ │ depends on handler. │
│ │ │ │ For files, it │
│ │ │ │ indicates on which │
│ │ │ │ device the │
│ │ │ │ "filesystem" must │
│ │ │ │ be mounted. If not │
│ │ │ │ specified, the │
│ │ │ │ current rootfs will │
│ │ │ │ be used. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│filesystem │ string │ files │ indicates the │
│ │ │ │ filesystem type │
│ │ │ │ where the file must │
│ │ │ │ be installed. Only │
│ │ │ │ used if "device" │
│ │ │ │ attribute is set. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│path │ string │ files │ For files: │
│ │ │ │ indicates the path │
│ │ │ │ (absolute) where │
│ │ │ │ the file must be │
│ │ │ │ installed. If │
│ │ │ │ "device" and │
│ │ │ │ "filesystem" are │
│ │ │ │ set, SWUpdate will │
│ │ │ │ install the file │
│ │ │ │ after mounting │
│ │ │ │ "device" with │
│ │ │ │ "filesystem" type. │
│ │ │ │ (path is always │
│ │ │ │ relative to the │
│ │ │ │ mount point.) │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│preserve-attributes │ bool │ files │ flag to control │
│ │ │ │ whether the │
│ │ │ │ following │
│ │ │ │ attributes will be │
│ │ │ │ preserved when │
│ │ │ │ files are unpacked │
│ │ │ │ from an archive │
│ │ │ │ (assuming │
│ │ │ │ destination │
│ │ │ │ filesystem supports │
│ │ │ │ them, of course): │
│ │ │ │ timestamp, uid/gid │
│ │ │ │ (numeric), perms, │
│ │ │ │ file attributes, │
│ │ │ │ extended attributes │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│type │ string │ images files │ string identifier │
│ │ │ scripts │ for the handler, as │
│ │ │ │ it is set by the │
│ │ │ │ handler when it │
│ │ │ │ registers itself. │
│ │ │ │ Example: "ubivol", │
│ │ │ │ "raw", "rawfile", │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│compressed │ string │ images files │ string to indicate │
│ │ │ │ the "filename" is │
│ │ │ │ compressed and must │
│ │ │ │ be decompressed │
│ │ │ │ before being │
│ │ │ │ installed. the │
│ │ │ │ value denotes the │
│ │ │ │ compression type. │
│ │ │ │ currently supported │
│ │ │ │ values are "zlib" │
│ │ │ │ and "zstd". │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│compressed │ bool (dep recated) │ images files │ Deprecated. Use the │
│ │ │ │ string form. true │
│ │ │ │ is equal to │
│ │ │ │ 'compressed = │
│ │ │ │ "zlib"'. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│installed-directly │ bool │ images │ flag to indicate │
│ │ │ │ that image is │
│ │ │ │ streamed into the │
│ │ │ │ target without any │
│ │ │ │ temporary copy. Not │
│ │ │ │ all handlers │
│ │ │ │ support streaming. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│name │ string │ bootenv │ name of the │
│ │ │ │ bootloader variable │
│ │ │ │ to be set. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│value │ string │ bootenv │ value to be │
│ │ │ │ assigned to the │
│ │ │ │ bootloader variable │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│name │ string │ images files │ name that │
│ │ │ │ identifies the │
│ │ │ │ sw-component it can │
│ │ │ │ be any string and │
│ │ │ │ it is compared with │
│ │ │ │ the entries in │
│ │ │ │ sw-versions │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│version │ string │ images files │ version for the │
│ │ │ │ sw-component it can │
│ │ │ │ be any string and │
│ │ │ │ it is compared with │
│ │ │ │ the entries in │
│ │ │ │ sw-versions │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│description │ string │ │ user-friendly │
│ │ │ │ description of the │
│ │ │ │ swupdate archive │
│ │ │ │ (any string) │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│reboot │ bool │ │ allows to disable │
│ │ │ │ reboot for the │
│ │ │ │ current running │
│ │ │ │ update │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│install-if-different │ bool │ images files │ flag if set, name │
│ │ │ │ and version are │
│ │ │ │ compared with the │
│ │ │ │ entries in │
│ │ │ │ sw-versions │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│install-if-higher │ bool │ images files │ flag if set, name │
│ │ │ │ and version are │
│ │ │ │ compared with the │
│ │ │ │ entries in │
│ │ │ │ sw-versions │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│encrypted │ bool │ images files │ flag if set, file │
│ │ │ scripts │ is encrypted and │
│ │ │ │ must be decrypted │
│ │ │ │ before installing. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│ivt │ string │ images files │ IVT in case of │
│ │ │ scripts │ encrypted artefact │
│ │ │ │ It has no value if │
│ │ │ │ "encrypted" is not │
│ │ │ │ set. Each artefact │
│ │ │ │ can have an own IVT │
│ │ │ │ to avoid attacker │
│ │ │ │ can guess the the │
│ │ │ │ key. It is an │
│ │ │ │ ASCII string of 32 │
│ │ │ │ chars │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│data │ string │ images files │ This is used to │
│ │ │ scripts │ pass arbitrary data │
│ │ │ │ to a handler. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│sha256 │ string │ images files │ sha256 hash of │
│ │ │ scripts │ image, file or │
│ │ │ │ script. Used for │
│ │ │ │ verification of │
│ │ │ │ signed images. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│embedded-script │ string │ │ Lua code that is │
│ │ │ │ embedded in the │
│ │ │ │ sw-description │
│ │ │ │ file. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│offset │ string │ images │ Optional │
│ │ │ │ destination offset │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│hook │ string │ images files │ The name of the │
│ │ │ │ function (Lua) to │
│ │ │ │ be called when the │
│ │ │ │ entry is parsed. │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│mtdname │ string │ images │ name of the MTD to │
│ │ │ │ update. Used only │
│ │ │ │ by the flash │
│ │ │ │ handler to identify │
│ │ │ │ the the mtd to │
│ │ │ │ update, instead of │
│ │ │ │ specifying the │
│ │ │ │ devicenode │
├─────────────────────┼────────────────────┼─────────────────────┼─────────────────────┤
│size │ int64 │ images files │ size of the file as │
│ │ │ scripts │ it is expected in │
│ │ │ │ the SWU. If set and │
│ │ │ │ the cpio size does │
│ │ │ │ not match for some │
│ │ │ │ reason the update │
│ │ │ │ will fail with an │
│ │ │ │ error. │
└─────────────────────┴────────────────────┴─────────────────────┴─────────────────────┘
UPDATE IMAGES FROM VERIFIED SOURCE
It is becoming very important that a device must not only be safely updated, but also that
it can verify if the delivered image is coming from a known source and it was not
corrupted introducing some malware.
To achieve this goal, SWUpdate must verify the incoming images. There are several ways to
do this. Should the compound image be signed ? Or some parts of it ?
Advantages and disadvantages are described in the following chapter.
Signing the compound image
It looks quite straightforward if the whole compound image is signed. However, this has
some heavy drawbacks. It is not possible to know if the image is verified until the whole
image is loaded. This means that verification can be done after installing the single
images instead of doing it before touching the device. This leads to have some uninstall
procedure if part of a not verified image is already installed, procedures that cannot be
safe in case of power off letting some unwanted piece of software on the device.
Signing the sub-images
If each sub-image is signed, the verification is done before calling the corresponding
hardware. Only signed images can be installed. Anyway, this remains unbound with the
description of the release in sw-description. Even if sw-description is signed, an
attacker can mix signed images together generating a new compound image that can be
installed as well, because all sub-images are verified.
Combining signing sw-description with hash verification
To avoid the described drawbacks, SWUpdate combines signed sw-description with the
verification of hashes for each single image. This means that only sw-description
generated by a verified source can be accepted by the installer. sw-description contains
hashes for each sub-image to verify that each delivered sub-image really belongs to the
release.
Choice of algorithm
The algorithm chosen to sign and verify the sw-description file can be selected via
menuconfig. Currently, the following mechanisms are implemented:
• RSA Public / private key. The private key belongs to the build system, while the public
key must be installed on the target.
• CMS using certificates
• GPG key signing
For RSA and CMS algorithms, key or certificate is passed to SWUpdate with the -k
parameter.
Tool to generate keys / certificates
For RSA and CMS signing, the openssl tool is used to generate the keys. This is part of
the OpenSSL project. A complete documentation can be found at the openSSL Website.
For GPG, gpg can be used to generate the keys and to sign the images. A complete
documentation can be found at the GnuPG Website.
Usage with RSA PKCS#1.5 or RSA PSS
Generating private and public key
First, the private key must be created:
openssl genrsa -aes256 -out priv.pem
This asks for a passphrase. It is possible to retrieve the passphrase from a file - of
course, this must be protected against intrusion.
openssl genrsa -aes256 -passout file:passout -out priv.pem
The private key is used to export the public key with:
openssl rsa -in priv.pem -out public.pem -outform PEM -pubout
"public.pem" contains the key in a format suitable for swupdate. The file can be passed to
swupdate at the command line with the -k parameter.
How to sign with RSA
Signing the image with rsa-pkcs#1.5 is very simple:
openssl dgst -sha256 -sign priv.pem sw-description > sw-description.sig
Signing the image with rsa-pss is also very simple:
openssl dgst -sha256 -sign priv.pem -sigopt rsa_padding_mode:pss \
-sigopt rsa_pss_saltlen:-2 sw-description > sw-description.sig
Usage with certificates and CMS
Generating self-signed certificates
openssl req -x509 -newkey rsa:4096 -nodes -keyout mycert.key.pem \
-out mycert.cert.pem -subj "/O=SWUpdate /CN=target"
Check the documentation for more information about parameters. The "mycert.key.pem"
contains the private key and it is used for signing. It is not delivered on the target.
The target must have "mycert.cert.pem" installed - this is used by SWUpdate for
verification.
Using PKI issued certificates
It is also possible to use PKI issued code signing certificates. However, SWUpdate uses
OpenSSL library for handling CMS signatures and the library requires the following
attributes to be set on the signing certificate:
keyUsage=digitalSignature
extendedKeyUsage=emailProtection
It is also possible to completely disable signing certificate key usage checking if this
requirement cannot be satisfied. This is controlled by
CONFIG_CMS_IGNORE_CERTIFICATE_PURPOSE configuration option.
How to sign with CMS
Signing the image is simple as in the previous case:
openssl cms -sign -in sw-description -out sw-description.sig -signer mycert.cert.pem \
-inkey mycert.key.pem -outform DER -nosmimecap -binary
Usage with GNU PG
Generating a new keypair
First, a primary keypair needs to be generated
gpg --gen-key
The generated keys can be listed as follows
gpg -k
Check the documentation for more information about parameters.
How to sign with gpg
Signing the image is very simple:
gpg --batch --output sw-description.sig
--detach-sig sw-description
For an alternative GnuPG home directory, and if there are multiple keypairs, the following
can be used to specify. In this example, the GnuPG home directory is in GPG_HOMEDIR, while
the signing key is found in GPG_KEY.
gpg --batch --homedir "${GPG_HOMEDIR}" --default-key "${GPG_KEY}" --output sw-description.sig
--detach-sig sw-description
Building a signed SWU image
There are two files, sw-description and its signature sw-description.sig. The signature
file must always directly follow the description file.
Each image inside sw-description must have the attribute "sha256", with the SHA256 sum of
the image. If an image does not have the sha256 attribute, the whole compound image
results as not verified and SWUpdate stops with an error before starting to install.
A simple script to create a signed image can be:
#!/bin/bash
MODE="RSA-PKCS-1.5"
PRODUCT_NAME="myproduct"
CONTAINER_VER="1.0"
IMAGES="rootfs kernel"
FILES="sw-description sw-description.sig $IMAGES"
#if you use RSA
if [ x"$MODE" == "xRSA-PKCS-1.5" ]; then
openssl dgst -sha256 -sign priv.pem sw-description > sw-description.sig
elif if [ x"$MODE" == "xRSA-PSS" ]; then
openssl dgst -sha256 -sign priv.pem -sigopt rsa_padding_mode:pss \
-sigopt rsa_pss_saltlen:-2 sw-description > sw-description.sig
elif if [ x"$MODE" == "xGPG" ]; then
gpg --batch --homedir "${GPG_HOME_DIR}" --default-key "${GPG_KEY}" \
--output sw-description.sig --detach-sig sw-description
else
openssl cms -sign -in sw-description -out sw-description.sig -signer mycert.cert.pem \
-inkey mycert.key.pem -outform DER -nosmimecap -binary
fi
for i in $FILES;do
echo $i;done | cpio -ov -H crc > ${PRODUCT_NAME}_${CONTAINER_VER}.swu
Example for sw-description with signed image
The example applies to a Beaglebone, installing Yocto images:
software =
{
version = "0.1.0";
hardware-compatibility: [ "revC"];
images: (
{
filename = "core-image-full-cmdline-beaglebone.ext3";
device = "/dev/mmcblk0p2";
type = "raw";
sha256 = "43cdedde429d1ee379a7d91e3e7c4b0b9ff952543a91a55bb2221e5c72cb342b";
}
);
scripts: (
{
filename = "test.lua";
type = "lua";
sha256 = "f53e0b271af4c2896f56a6adffa79a1ffa3e373c9ac96e00c4cfc577b9bea5f1";
}
);
}
Running SWUpdate with signed images
Verification is activated by setting CONFIG_SIGNED_IMAGES in SWUpdate's configuration. If
activated, SWUpdate will always check the compound image. For security reasons, it is not
possible to disable the check at runtime.
For RSA and CMS signing, the -k parameter (public key file) is mandatory and the program
stops if the public key is not passed.
For GPG signing, CONFIG_SIGALG_GPG needs to be enabled. The GPG key will need to be
imported to the device's GnuPG home directory. To do this, the key will need to be
exported:
gpg --export <keyid> --output <public key>
You can then copy it onto the device and import it into your public keyring:
gpg --import <public key>
To verify that the key has been imported successfully:
gpg --list-keys
SWUpdate will need need to be configured with the following parameters:
GnuPG Home directory: gpg-home-dir in swupdate.cfg
GPGME Protocol: gpgme-protocol in swupdate.cfg: openpgp or cms
SYMMETRICALLY ENCRYPTED UPDATE IMAGES
SWUpdate allows one to symmetrically encrypt update images using the AES block cipher in
CBC mode. The following shows encryption with 256 bit key length but you may use other key
lengths as well.
Building an Encrypted SWU Image
First, create a key; for aes-256-cbc we need 32 bytes of key and 16 bytes for an
initialisation vector (IV). A complete documentation can be found at the OpenSSL Website.
openssl rand -hex 32
# key, for example 390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040
openssl rand -hex 16
# IV, for example d5d601bacfe13100b149177318ebc7a4
Then, encrypt an image using this information via
openssl enc -aes-256-cbc -in <INFILE> -out <OUTFILE> -K <KEY> -iv <IV>
where <INFILE> is the unencrypted source image file and <OUTFILE> is the encrypted output
image file to be referenced in sw-description. <KEY> is the hex value part of the 2nd
line of output from the key generation command above and <IV> is the hex value part of the
3rd line.
Then, create a key file to be supplied to SWUpdate via the -K switch by putting the key
and initialization vector hex values on one line separated by whitespace, e.g., for above
example values
390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040 d5d601bacfe13100b149177318ebc7a4
Previous versions of SWUpdate allowed for a salt as third word in key file, that was never
actually used for aes and has been removed.
You should change the IV with every encryption, see CWE-329. The ivt sw-description
attribute overrides the key file's IV for one specific image.
Encryption of UBI volumes
Due to a limit in the Linux kernel API for UBI volumes, the size reserved to be written on
disk should be declared before actually writing anything.
See the property "decrypted-size" in UBI Volume Handler's documentation.
Example sw-description with Encrypted Image
The following example is a (minimal) sw-description for installing a Yocto image onto a
Beaglebone. Pay attention to the encrypted = true; setting.
software =
{
version = "0.0.1";
images: ( {
filename = "core-image-full-cmdline-beaglebone.ext3.enc";
device = "/dev/mmcblk0p3";
encrypted = true;
ivt = "65D793B87B6724BB27954C7664F15FF3";
}
);
}
Running SWUpdate with Encrypted Images
Symmetric encryption support is activated by setting the ENCRYPTED_IMAGES option in
SWUpdate's configuration. Use the -K parameter to provide the symmetric key file generated
above to SWUpdate.
Decrypting with a PKCS#11 token
PKCS#11 support is activated by setting the PKCS11 option in SWUpdate's configuration. The
key file has to have a PKCS#11 URL instead of the key then, containing at least the
elements of this example:
pkcs11:slot-id=42;id=%CA%FE%BA%BE?pin-value=1234&module-path=/usr/lib/libsofthsm2.so 65D793B87B6724BB27954C7664F15FF3
The encryption key can be imported to the PKCS#11 token by using pkcs11-tool as follow:
echo -n "390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040" | xxd -p -r > swupdate-aes-key.bin
pkcs11-tool --module /usr/lib/libsofthsm2.so --slot 0x42 --login --write-object swupdate-aes-key.bin --id CAFEBABE --label swupdate-aes-key --type secrkey --key-type AES:32
HANDLERS
Overview
It is quite difficult to foresee all possible installation cases. Instead of trying to
find all use cases, SWUpdate let the developer free to add his own installer (that is, a
new handler), that must be responsible to install an image of a certain type. An image is
marked to be of a defined type to be installed with a specific handler.
The parser make the connection between 'image type' and 'handler'. It fills a table
containing the list of images to be installed with the required handler to execute the
installation. Each image can have a different installer.
Supplied handlers
In mainline there are the handlers for the most common cases. They include:
• flash devices in raw mode (both NOR and NAND)
• UBI volumes
• UBI volumes partitioner
• raw flashes handler (NAND, NOR, SPI-NOR, CFI interface)
• disk partitioner
• raw devices, such as a SD Card partition
• bootloader (U-Boot, GRUB, EFI Boot Guard) environment
• Lua scripts handler
• shell scripts handler
• rdiff handler
• readback handler
• archive (zo, tarballs) handler
• remote handler
• microcontroller update handler
For example, if an image is marked to be updated into a UBI volume, the parser must fill a
supplied table setting "ubi" as required handler, and filling the other fields required
for this handler: name of volume, size, and so on.
Creating own handlers
SWUpdate can be extended with new handlers. The user needs to register his own handler
with the core and he must provide the callback that SWUpdate uses when an image required
to be installed with the new handler.
The prototype for the callback is:
int my_handler(struct img_type *img,
void __attribute__ ((__unused__)) *data)
The most important parameter is the pointer to a struct img_type. It describes a single
image and inform the handler where the image must be installed. The file descriptor of the
incoming stream set to the start of the image to be installed is also part of the
structure.
The structure img_type contains the file descriptor of the stream pointing to the first
byte of the image to be installed. The handler must read the whole image, and when it
returns back SWUpdate can go on with the next image in the stream.
The data parameter is usually a pointer that was registered with the handler. For script
handlers it is instead a pointer to a struct script_handler_data which contains a
script_fn enum value, indicating the current installation phase, and the registered data
pointer.
SWUpdate provides a general function to extract data from the stream and copy to somewhere
else:
int copyfile(int fdin, int fdout, int nbytes, unsigned long *offs,
int skip_file, int compressed, uint32_t *checksum, unsigned char *hash);
fdin is the input stream, that is img->fdin from the callback. The hash, in case of signed
images, is simply passed to copyfile() to perform the check, exactly as the checksum
parameter. copyfile() will return an error if checksum or hash do not match. The handler
does not need to bother with them. How the handler manages the copied data, is specific
to the handler itself. See supplied handlers code for a better understanding.
The handler's developer registers his own handler with a call to:
__attribute__((constructor))
void my_handler_init(void)
{
register_handler("mytype", my_handler, my_mask, data);
}
SWUpdate uses the gcc constructors, and all supplied handlers are registered when SWUpdate
is initialized.
register_handler has the syntax:
register_handler(my_image_type, my_handler, my_mask, data);
Where:
• my_image_type : string identifying the own new image type.
• my_handler : pointer to the installer to be registered.
• my_mask : HANDLER_MASK enum value(s) specifying what input type(s) my_handler can
process.
• data : an optional pointer to an own structure, that SWUpdate saves in the handlers'
list and pass to the handler when it will be executed.
UBI Volume Handler
The UBI volume handler will update UBI volumes without changing the layout on the storage.
Therefore, volumes must be created/adjusted beforehand. This can be done using the
partitions tag (see partitions : UBI layout).
The UBI volume handler will search for volumes in all MTD devices (unless blacklisted, see
UBIBLACKLIST) to find the volume into which the image shall be installed. For this reason,
volume names must be unique within the system. Two volumes with the same name are not
supported and will lead to unpredictable results (SWUpdate will install the image to the
first volume with that name it finds, which may not be right one!).
When updating volumes, it is guaranteed that erase counters are preserved and not lost.
The behavior of updating is identical to that of the ubiupdatevol(1) tool from mtd-utils.
In fact, the same library from mtd-utils (libubi) is reused by SWUpdate.
atomic volume renaming
The UBI volume handler has basic support for carrying out atomic volume renames by
defining the replaces property, which must contain a valid UBI volume name. After
successfully updating the image to volume, an atomic swap of the names of volume and
replaces is done. Consider the following example
{
filename ="u-boot.img";
volume ="u-boot_r";
properties: {
replaces = "u-boot";
}
}
After u-boot.img is successfully installed into the volume "u-boot_r", the volume
"u-boot_r" is renamed to "u-boot" and "u-boot" is renamed to "u-boot_r".
This mechanism allows one to implement a simple double copy update approach without the
need of shared state with the bootloader. For example, the U-Boot SPL can be configured to
always load U-Boot from the volume u-boot without the need to access the environment. The
volume replace functionality will ensure that this volume name always points to the
currently valid volume.
However, please note the following limitations:
• Currently the rename takes place after each image was installed successfully. Hence, it
only makes sense to use this feature for images that are independent of the other
installed images. A typical example is the bootloader. This behavior may be modified in
the future to only carry out one atomic rename after all images were installed
successfully.
• Atomic renames are only possible and permitted for volumes residing on the same UBI
device.
There is a handler ubiswap that allow one to do an atomic swap for several ubi volume
after all the images were flashed. This handler is a script for the point of view of
swudate, so the node that provide it the data should be added in the section scripts.
scripts: (
{
type = "ubiswap";
properties: {
swap-0 = [ "boot" , " boot_r" ];
swap-1 = [ "kernel" , "kernel_r" ];
swap-2 = [ "rootfs" , "rootfs_r" ];
},
},
);
WARNING: if you use the property replaces on an ubi volume that is also used with the
handler ubiswap, this ubi volume will be swapped twice. It's probably not what you want
...
volume auto resize
The UBI volume handler has support to auto resize before flashing an image with the
property auto-resize. When this property is set on an image, the ubi volume is resized to
fit exactly the image.
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
auto-resize = "true";
}
}
WARNING: when this property is used, the device must be defined.
volume always remove
The UBI volume handler has support to always remove ubi volume before flashing with the
property always-remove. When this property is set on an image, the ubi volume is always
removed. This property should be used with property auto-resize.
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
always-remove = "true";
auto-resize = "true";
}
}
size properties
Due to a limit in the Linux kernel API for UBI volumes, the size reserved to be written on
disk should be declared before actually writing anything. Unfortunately, the size of an
encrypted or compressed image is not known until the decryption or decompression finished.
This prevents correct declaration of the file size to be written on disk.
For this reason UBI images can declare the special properties "decrypted-size" or
"decompressed-size" like this:
images: ( {
filename = "rootfs.ubifs.enc";
volume = "rootfs";
encrypted = true;
properties: {
decrypted-size = "104857600";
}
},
{
filename = "homefs.ubifs.gz";
volume = "homefs";
compressed = "zlib";
properties: {
decompressed-size = "420000000";
}
}
);
The real size of the image should be calculated and written to the sw-description before
assembling the cpio archive. In this example, 104857600 is the size of the rootfs after
the decryption: the encrypted size is by the way larger. The decompressed size is of the
homefs is 420000000.
The sizes are bytes in decimal notation.
Lua Handlers
In addition to the handlers written in C, it is possible to extend SWUpdate with handlers
written in Lua that get loaded at SWUpdate startup. The Lua handler source code file may
either be embedded into the SWUpdate binary via the CONFIG_EMBEDDED_LUA_HANDLER config
option or has to be installed on the target system in Lua's search path as
swupdate_handlers.lua so that it can be loaded by the embedded Lua interpreter at
run-time.
In analogy to C handlers, the prototype for a Lua handler is
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
...
end
where image is a Lua table (with attributes according to sw-description's attribute
reference) that describes a single artifact to be processed by the handler (also see the
Lua Handler Interface Specification in handlers/swupdate.lua).
Note that dashes in the attributes' names are replaced with underscores for the Lua domain
to make them idiomatic, e.g., installed-directly becomes installed_directly in the Lua
domain.
For a script handler written in Lua, the prototype is
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @param scriptfn string Type, one of `preinst` or `postinst`
--- @return number # 0 on success, 1 on error
function lua_handler(image, scriptfn)
...
end
where scriptfn is either "preinst" or "postinst".
To register a Lua handler, the swupdate module provides the swupdate.register_handler()
method that takes the handler's name, the Lua handler function to be registered under that
name, and, optionally, the types of artifacts for which the handler may be called. If the
latter is not given, the Lua handler is registered for all types of artifacts. The
following call registers the above function lua_handler as my_handler which may be called
for images:
swupdate.register_handler("my_handler", lua_handler, swupdate.HANDLER_MASK.IMAGE_HANDLER)
A Lua handler may call C handlers ("chaining") via the swupdate.call_handler() method. The
callable and registered C handlers are available (as keys) in the table swupdate.handler.
The following Lua code is an example of a simple handler chain-calling the rawfile C
handler:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
if not swupdate.handler["rawfile"] then
swupdate.error("rawfile handler not available")
return 1
end
image.path = "/tmp/destination.path"
local err, msg = swupdate.call_handler("rawfile", image)
if err ~= 0 then
swupdate.error(string.format("Error chaining handlers: %s", msg))
return 1
end
return 0
end
Note that when chaining handlers and calling a C handler for a different type of artifact
than the Lua handler is registered for, the image table's values must satisfy the called C
handler's expectations: Consider the above Lua handler being registered for "images"
(swupdate.HANDLER_MASK.IMAGE_HANDLER) via the swupdate.register_handler() call shown
above. As per the sw-description's attribute reference, the "images" artifact type doesn't
have the path attribute but the "file" artifact type does. So, for calling the rawfile
handler, image.path has to be set prior to chain-calling the rawfile handler, as done in
the example above. Usually, however, no such adaptation is necessary if the Lua handler is
registered for handling the type of artifact that image represents.
In addition to calling C handlers, the image table passed as parameter to a Lua handler
has a image:copy2file() method that implements the common use case of writing the input
stream's data to a file, which is passed as this method's argument. On success,
image:copy2file() returns 0 or -1 plus an error message on failure. The following Lua code
is an example of a simple handler calling image:copy2file():
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
local err, msg = image:copy2file("/tmp/destination.path")
if err ~= 0 then
swupdate.error(string.format("Error calling copy2file: %s", msg))
return 1
end
return 0
end
Beyond using image:copy2file() or chain-calling C handlers, the image table passed as
parameter to a Lua handler has a image:read(<callback()>) method that reads from the input
stream and calls the Lua callback function <callback()> for every chunk read, passing this
chunk as parameter. On success, 0 is returned by image:read(). On error, -1 plus an error
message is returned. The following Lua code is an example of a simple handler printing the
artifact's content:
--- Lua Handler.
--
--- @param image img_type Lua equivalent of `struct img_type`
--- @return number # 0 on success, 1 on error
function lua_handler(image)
err, msg = image:read(function(data) print(data) end)
if err ~= 0 then
swupdate.error(string.format("Error reading image: %s", msg))
return 1
end
return 0
end
Using the image:read() method, an artifact's contents may be (post-)processed in and
leveraging the power of Lua without relying on preexisting C handlers for the purpose
intended.
Just as C handlers, a Lua handler must consume the artifact described in its image
parameter so that SWUpdate can continue with the next artifact in the stream after the Lua
handler returns. Chaining handlers, calling image:copy2file(), or using image:read()
satisfies this requirement.
The swupdate Lua module interface specification that details what functionality is made
available to Lua handlers by SWUpdate's corelib/lua_interface.c is found in
handlers/swupdate.lua. It serves as reference, for mocking purposes, and type checking
thanks to the EmmyLua-inspired annotations.
Note that although the dynamic nature of Lua handlers would technically allow one to embed
them into a to be processed .swu image, this is not implemented as it carries some
security implications since the behavior of SWUpdate is changed dynamically.
Shell script handler
This handler allows to run a shell script that is packed into the SWU. Please note that
running a shell script opens a set of different security issues. Shell scripts are
supported due to their large acceptance, but you should prefer Lua Scripts.
SWUpdate will run the binary shell "/bin/sh" to execute the script.
Lua script handler
A Lua Script handler runs a script in Lua language. There are two possible ways to run the
script:
• local: the script runs in own (isolated) Lua state that is created for the script.
The script has access only to function defined inside the script or functions
provided by external libraries, like the internal swupdate library called via
"require(swupdate)".
• global: SWUpdate create a Lua state at the beginning of an Update and this is valid
until the update is terminated. In this case, the script has access to any function
and structure that was defined during the update. For example, a function can be
defined inside sw-description, and the script can call it.
As default, each script runs in isolated / local Lua state. If the property "global-state"
is set, then the common LUa state used for each Update transaction is taken.
Scripts ran in isolated context in previous versions. SWUpdate allocates a new Lua state,
and import the basic libraries before loading the script. A script is then isolated, but
it cannot access to function already loaded, or it is not possible to reuse functions from
2 or more scripts.
With the introduction of a per installation Lua state, Lua scripts can call functions
already defined in previous scripts, or defined in sw-description. Because when a script
is loaded, existing functions with the same name are overwritten, it was decided that
functions in scripts must be unique, that is each function should be declared just once
during an installation process.
This means that for global state, sw-description should contain the name of the function
for each step (pre- , postinstall or postfailure) that should be called: the names
preinst, postinst and postfailure are still valid in case the script runs with isolated
state.
This allows also to load a script without executing if no functions are defined, and
functions in the script can be called by later scripts.
Note that the handler will load the script in case of global state just once during the
"preinstall" call. Later, it is assumed that functions will be already available.
Example:
scripts: (
{
filename = "testscript.lua";
type = "lua";
properties: {
global-state = "true";
preinstall = "pretest1";
}
},
{
filename = "test2script.lua";
type = "lua";
properties: {
global-state = "true";
postinstall = "posttest2";
postfailure = "failure";
}
}
Two scripts are defined. Both are using the global Lua state. Functions in test2script
can find and run functions defined in testscript.lua, because both are belonging to the
same context. When preinstall scripts are called, only the function "pretest1" from the
first script is called, because no function name is defined for this step with the
following scripts.
Remote handler
Remote handlers are thought for binding legacy installers without having the necessity to
rewrite them in Lua. The remote handler forward the image to be installed to another
process, waiting for an acknowledge to be sure that the image is installed correctly. The
remote handler makes use of the zeromq library - this is to simplify the IPC with Unix
Domain Socket. The remote handler is quite general, describing in sw-description with the
"data" attribute how to communicate with the external process. The remote handler always
acts as client, and try a connect() using the socket identified by the "data" attribute.
For example, a possible setup using a remote handler could be:
images: (
{
filename = "myimage"";
type = "remote";
data = "test_remote";
}
)
The connection is instantiated using the socket test_remote (according to the "data"
field's value) in the directory pointed to by the environment variable TMPDIR with /tmp as
fall-back if TMPDIR is not set. If connect() fails, the remote handler signals that the
update is not successful. Each zeromq message from SWUpdate is a multi-part message split
into two frames:
• first frame contains a string with a command.
• second frame contains data and can be of 0 bytes.
There are currently just two possible commands: INIT and DATA. After a successful connect,
SWUpdate sends the initialization string in the format:
INIT:<size of image to be installed>
The external installer is informed about the size of the image to be installed, and it can
assign resources if it needs. It will answer with the string ACK or NACK. The first NACK
received by SWUpdate will interrupt the update. After sending the INIT command, the remote
handler will send a sequence of DATA commands, where the second frame in message will
contain chunks of the image to be installed. It is duty of the external process to take
care of the amount of data transferred and to release resources when the last chunk is
received. For each DATA message, the external process answers with a ACK or NACK message.
SWU forwarder
The SWU forwarder handler can be used to update other systems where SWUpdate is running.
It can be used in case of master / slaves systems, where the master is connected to the
network and the "slaves" are hidden to the external world. The master is then the only
interface to the world. A general SWU can contain embedded SWU images as single artifacts,
and the SWU handler will forward it to the devices listed in the description of the
artifact. The handler can have a single "url" properties entry with an array of urls.
Each url is the address of a secondary board where SWUpdate is running with webserver
activated. The SWU handler expects to talk with SWUpdate's embedded webserver. This helps
to update systems where an old version of SWUpdate is running, because the embedded
webserver is a common feature present in all versions. The handler will send the embedded
SWU to all URLs at the same time, and setting installed-directly is supported by this
handler. [image]
The following example shows how to set a SWU as artifact and enables the SWU forwarder:
images: (
{
filename = "image.swu";
type = "swuforward";
properties: {
url = ["http://192.168.178.41:8080", "http://192.168.178.42:8080"];
};
});
The SWU forwarder can be used as generic uploader to an URL. This is requires to enable
Lua support. The back channel should still run via Websocket, if the connected server
should communicate a progress status. The handler allows to link an own Lua code that is
able to parse the incoming data, and report an error. The properties field should contain
the name of the Lua function that should be called to parse the answer from the remote
system. Note that the data passed to Lua is converted to a string (and null terminated).
images: (
{
filename = "image.swu";
type = "swuforward";
properties: {
url = ["http://192.168.178.41", "http://192.168.178.42];
parser-function = "parse_answer";
};
});
The parser function should be already loaded. The Lua function receives as input a string
with the data received via the Websocket back channel. The function will read the data and
return a single scalar value of swupdate_status, that is one of
swupdate.RECOVERY_STATUS.*. Example:
function parse_answer(s)
if string.match (s, 'SUCCESS') then
return swupdate.RECOVERY_STATUS.SUCCESS
end
if string.match (s, 'FAILURE') then
return swupdate.RECOVERY_STATUS.FAILURE
end
return swupdate.RECOVERY_STATUS.RUN
end";
The handler evaluates the return value, and stops when one of SUCCESS or FAILURE is
returned.
If no parser-function is passed, the handler will run the internal parser used to connect
remote SWUpdate systems, and expects to see a JSON message sent by the Mongoose Webserver.
rdiff handler
The rdiff handler adds support for applying binary delta patches generated by librsync's
rdiff tool.
Naturally, the smaller the difference between the diff's source and target, the more
effective is using this handler rather than shipping the full target, e.g., via the image
handler. Hence, the most prominent use case for the rdiff handler is when having a
read-only root filesystem and applying a small update like security fixes or feature
additions. If the sweet spot is crossed, an rdiff patch may even exceed the full target's
size due to necessary patch metadata. Also note that in order to be most effective, an
image to be processed with rdiff should be built deterministic (see ‐
reproducible-builds.org).
The rdiff algorithm requires no resources whatsoever on the device as the patch is fully
computed in the backend. Consequently, the backend has to have knowledge of the current
software running on the device in order to compute a sensible patch. Alike, the patch has
to be applied on the device to an unmodified source as used in the backend for patch
computation. This property is in particular useful for resource-constrained devices as
there's no need for the device to, e.g., aid in the difference computation.
First, create the signature of the original (base) file via rdiff signature <basefile>
<signaturefile>. Then, create the delta file (i.e., patch) from the original base file to
the target file via rdiff delta <signaturefile> <targetfile> <deltafile>. The <deltafile>
is the artifact to be applied via this handler on the device. Essentially, it mimics
running rdiff patch <basefile> <deltafile> <targetfile> on the device. Naturally for
patches, the very same <basefile> has to be used for creating as well as for applying the
patch to.
This handler registers itself for handling files and images. An exemplary sw-description
fragment for the files section is
files: (
{
type = "rdiff_file"
filename = "file.rdiff.delta";
path = "/usr/bin/file";
}
);
Note that the file referenced to by path serves as <basefile> and gets replaced by a
temporary file serving as <targetfile> while the rdiff patch processing.
An exemplary sw-description fragment for the images section is
images: (
{
type = "rdiff_image";
filename = "image.rdiff.delta";
device = "/dev/mmcblk0p2";
properties: {
rdiffbase = ["/dev/mmcblk0p1"];
};
}
);
Here, the property rdiffbase qualifies the <basefile> while the device attribute
designates the <targetfile>. Note that there's no support for the optional offset
attribute in the rdiff_image handler as there's currently no apparent use case for it and
skipping over unchanged content is handled well by the rdiff algorithm.
ucfw handler
This handler allows one to update the firmware on a microcontroller connected to the main
controller via UART. Parameters for setup are passed via sw-description file. Its
behavior can be extended to be more general. The protocol is ASCII based. There is a
sequence to be done to put the microcontroller in programming mode, after that the handler
sends the data and waits for an ACK from the microcontroller.
The programming of the firmware shall be:
1. Enter firmware update mode (bootloader)
1. Set "reset line" to logical "low"
2. Set "update line" to logical "low"
3. Set "reset line" to logical "high"
2. Send programming message
$PROG;<<CS>><CR><LF>
to the microcontroller. (microcontroller will remain in programming state)
3. microcontroller confirms with
$READY;<<CS>><CR><LF>
4. Data transmissions package based from mainboard to microcontroller package definition:
• within a package the records are sent one after another without the end of line
marker <CR><LF>
• the package is completed with <CR><LF>
5. The microcontroller requests the next package with $READY;<<CS>><CR><LF>
6. Repeat step 4 and 5 until the complete firmware is transmitted.
7. The keypad confirms the firmware completion with $COMPLETED;<<CS>><CR><LF>
8.
Leave firmware update mode
1. Set "Update line" to logical "high"
2. Perform a reset over the "reset line"
<<CS>> : checksum. The checksum is calculated as the two's complement of the modulo-256
sum over all bytes of the message string except for the start marker "$". The handler
expects to get in the properties the setup for the reset and prog gpios. They should be in
this format:
properties = {
reset = "<gpiodevice>:<gpionumber>:<activelow>";
prog = "<gpiodevice>:<gpionumber>:<activelow>";
}
Example:
images: (
{
filename = "microcontroller-image";
type = "ucfw";
device = "/dev/ttymxc5";
properties: {
reset = "/dev/gpiochip0:38:false";
prog = "/dev/gpiochip0:39:false";
};
}
);
SSBL Handler
This implements a way to switch two software sets using a duplicated structure saved on
the flash (currently, only NOR flash is supported). Each of the two structures contains
address and size of the image to be loaded by a first loader. A field contain the "age",
and it is incremented after each switch to show which is the active set.
Structure of SSBL Admin
┌───────────────────────────────┬─────────────┐
│SSBL Magic Number (29 bit)Name │ Age (3 bit) │
├───────────────────────────────┼─────────────┤
│Image Address Offset │ │
├───────────────────────────────┼─────────────┤
│Image Size │ │
└───────────────────────────────┴─────────────┘
The handler implements a post install script. First, it checks for consistency the two
structures and find the active reading the first 32 bit value with a magic number and the
age. It increments the age and saves the new structure in the inactive copy. After a
reboot, the loader will check it and switch the software set.
scripts: (
{
type = "ssblswitch";
properties: {
device = ["mtdX", "mtdY"];
offset = ["0", "0"];
imageoffs = ["0x780000", "0xA40000"];
imagesize = ["0x800000", "0x800000"];
}
}
Properties in sw-description are all mandatory. They define where the SSBL Administration
data are stored for both sets. Each properties is an array of two entries, containing
values for each of the two SSBL administration.
Properties for SSBL handler
┌────────────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├────────────┼────────┼──────────────────────────┤
│device │ string │ MTD device where the │
│ │ │ SSBL Admin Header is │
│ │ │ stored │
├────────────┼────────┼──────────────────────────┤
│offset │ hex │ Offset of SSBL header │
│ │ │ inside the MTD device │
├────────────┼────────┼──────────────────────────┤
│imageoffset │ hex │ Offset of the image to │
│ │ │ be loaded by a │
│ │ │ bootloader when this │
│ │ │ SSBL is set. │
├────────────┼────────┼──────────────────────────┤
│imagesize │ hex │ Size of the image to be │
│ │ │ loaded by a bootloader │
│ │ │ when this SSBL is set. │
└────────────┴────────┴──────────────────────────┘
Readback Handler
To verify that an image was written properly, this readback handler calculates the sha256
hash of a partition (or part of it) and compares it against a given hash value.
The following example explains how to use this handler:
scripts: (
{
device = "/dev/mmcblk2p1";
type = "readback";
properties: {
sha256 = "e7afc9bd98afd4eb7d8325196d21f1ecc0c8864d6342bfc6b6b6c84eac86eb42";
size = "184728576";
offset = "0";
};
}
);
Properties size and offset are optional, all the other properties are mandatory.
Properties for readback handler
┌───────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├───────┼────────┼──────────────────────────┤
│device │ string │ The partition which │
│ │ │ shall be verified. │
├───────┼────────┼──────────────────────────┤
│type │ string │ Identifier for the │
│ │ │ handler. │
├───────┼────────┼──────────────────────────┤
│sha256 │ string │ Expected sha256 hash of │
│ │ │ the partition. │
├───────┼────────┼──────────────────────────┤
│size │ string │ Data size (in bytes) to │
│ │ │ be verified. If 0 or │
│ │ │ not set, the handler │
│ │ │ will get the partition │
│ │ │ size from the device. │
├───────┼────────┼──────────────────────────┤
│offset │ string │ Offset (in bytes) to the │
│ │ │ start of the partition. │
│ │ │ If not set, default │
│ │ │ value 0 will be used. │
└───────┴────────┴──────────────────────────┘
Copy handler
The copy handler copies one source to a destination. It is a script handler, and no
artifact in the SWU is associated with the handler. It can be used to copy configuration
data, or parts that should be taken by the current installation. It requires the
mandatory property (copyfrom), while device contains the destination path. The handler
performs a byte copy, and it does not matter which is the source - it can be a file or a
partition. An optional type field can set if the handler is active as pre or postinstall
script. If not set, the handler is called twice.
Attributes for copy handler
┌───────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├───────┼────────┼──────────────────────────┤
│device │ string │ If set, it is the │
│ │ │ destination. │
├───────┼────────┼──────────────────────────┤
│type │ string │ One of "preinstall" or │
│ │ │ "postinstall" │
└───────┴────────┴──────────────────────────┘
Properties for copy handler
┌────────────────────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├────────────────────┼────────┼──────────────────────────┤
│size │ string │ Data size (in bytes) to │
│ │ │ be copied. If 0 or not │
│ │ │ set, the handler will │
│ │ │ try to find the size │
│ │ │ from the device. │
├────────────────────┼────────┼──────────────────────────┤
│chain │ string │ Handler to be called to │
│ │ │ install the data read │
│ │ │ from the "copyfrom" │
│ │ │ source. │
├────────────────────┼────────┼──────────────────────────┤
│recursive │ string │ Recursive copy if │
│ │ │ copyfrom is a directory │
│ │ │ ("true" or "false") │
├────────────────────┼────────┼──────────────────────────┤
│create- destination │ string │ Create the destination │
│ │ │ path if it does not │
│ │ │ exist ("true" or │
│ │ │ "false") │
└────────────────────┴────────┴──────────────────────────┘
scripts : (
{
device = "/dev/mmcblk2p1";
type = "copy";
properties : {
copyfrom = "/dev/mmcblk2p2";
type = "postinstall";
chain = "raw";
}
}
Bootloader handler
The bootloader handler allows to set bootloader's environment with a file. The file shold
have the format:
# Comments are allowed using the hash char
varname=value
Empty lines are skipped. This simplifies the update of the whole environment instead of
setting each variable inside the "bootenv" section in sw-description. The property
nooverride allows to skip variables that are already set in sw-description. If not set,
variables set in bootenv are overwritten.
images: (
{
filename = "uEnv.txt";
type = "bootloader";
properties: {
nooverride = "true";
}
}
);
bootenv: (
{
name = "bootenv01key";
value = "SOME VALUE";
});
In the example above, bootenv01key is not overwritten by a value in uEnv.txt because the
flag "nooverride" is set.
Archive handler
The archive handler extracts an archive to a destination path. It supports whatever
format libarchive has been compiled to support, for example even if swupdate itself has no
direct support for xz it can be possible to extract tar.xz files with it.
The attribute preserve-attributes must be set to preserve timestamps. uid/gid (numeric),
permissions (except +x, always preserved) and extended attributes.
The property create-destination can be set to the string true to have swupdate create the
destination path before extraction.
files: (
{
filename = "examples.tar.zst";
type = "archive";
path = "/extract/here";
preserve-attributes = true;
installed-directly = true;
properties: {
create-destination = "true";
}
}
);
Disk partitioner
This handler creates or modifies partitions using the library libfdisk. Handler must be
put into the partitions section of sw-description. Setup for each partition is put into
the properties field of sw-description. After writing the partition table it may create a
file system on selected partitions. (Available only if CONFIG_DISKFORMAT is set.)
Properties for diskpart handler
┌────────────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├────────────┼────────┼──────────────────────────┤
│labeltype │ string │ "gpt" or "dos" │
├────────────┼────────┼──────────────────────────┤
│nolock │ string │ "true" or "false" │
│ │ │ (default=false) This is │
│ │ │ like a force. If it is │
│ │ │ set, a lock failure will │
│ │ │ be ignored(lock will │
│ │ │ still be attempted). │
├────────────┼────────┼──────────────────────────┤
│noinuse │ string │ "true" or "false" │
│ │ │ (default=false) If set, │
│ │ │ it does not require the │
│ │ │ device to be not in use │
│ │ │ (mounted, etc.) │
├────────────┼────────┼──────────────────────────┤
│partition-X │ array │ Array of values │
│ │ │ belonging to the │
│ │ │ partition number X │
└────────────┴────────┴──────────────────────────┘
For each partition, an array of couples key=value must be given. The following keys are
supported:
Setup for a disk partition
┌─────────┬─────────┬──────────────────────────┐
│Name │ Type │ Description │
├─────────┼─────────┼──────────────────────────┤
│size │ string │ Size of partition. K, M │
│ │ │ and G can be used for │
│ │ │ Kilobytes, Megabytes and │
│ │ │ Gigabytes. │
├─────────┼─────────┼──────────────────────────┤
│start │ integer │ First sector for the │
│ │ │ partition │
├─────────┼─────────┼──────────────────────────┤
│name │ string │ Name of the partition │
├─────────┼─────────┼──────────────────────────┤
│type │ string │ Type of partition, it │
│ │ │ has two different │
│ │ │ meanings. It is the hex │
│ │ │ code for DOS (MBR) │
│ │ │ partition table or it is │
│ │ │ the string identifier in │
│ │ │ case of GPT. │
├─────────┼─────────┼──────────────────────────┤
│dostype │ string │ Type of DOS (MBR) │
│ │ │ partition entry when │
│ │ │ using a table with a │
│ │ │ "gpt" labeltype. Using │
│ │ │ this option will create │
│ │ │ a hybrid MBR table. It │
│ │ │ is the hex code for DOS │
│ │ │ (MBR) partition table. │
│ │ │ This would typically be │
│ │ │ used when one wants to │
│ │ │ use a GPT formatted disk │
│ │ │ with a board that │
│ │ │ requires a dos table │
│ │ │ entry for initial │
│ │ │ bootstrapping. Note: A │
│ │ │ maximum of 3 partitions │
│ │ │ can have a dostype │
│ │ │ specified, this limit │
│ │ │ only applies to dos │
│ │ │ table entries and does │
│ │ │ not affect partitions │
│ │ │ without a dostype │
│ │ │ specified. │
├─────────┼─────────┼──────────────────────────┤
│fstype │ string │ Optional filesystem type │
│ │ │ to be created on the │
│ │ │ partition. If no fstype │
│ │ │ key is given, no file │
│ │ │ will be created on the │
│ │ │ corresponding partition. │
│ │ │ vfat / ext2 / ext3 /ext4 │
│ │ │ / btrfs file system is │
│ │ │ supported │
├─────────┼─────────┼──────────────────────────┤
│partuuid │ string │ The partition UUID (GPT │
│ │ │ only). If omitted, a │
│ │ │ UUID will be generated │
│ │ │ automatically. │
├─────────┼─────────┼──────────────────────────┤
│flag │ string │ The following flags are │
│ │ │ supported: Dos Partition │
│ │ │ : "boot" set bootflag │
├─────────┼─────────┼──────────────────────────┤
│force │ string │ If set to "true", │
│ │ │ existing file-system │
│ │ │ shall be overwritten │
│ │ │ uncoditionally. Default │
│ │ │ value is "false". │
└─────────┴─────────┴──────────────────────────┘
GPT example:
partitions: (
{
type = "diskpart";
device = "/dev/sde";
properties: {
labeltype = "gpt";
partition-1 = [ "size=64M", "start=2048",
"name=bigrootfs", "type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B"];
partition-2 = ["size=256M", "start=133120",
"name=ldata", "type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7",
"fstype=vfat"];
partition-3 = ["size=512M", "start=657408",
"name=log", "fstype =ext4", 63DAF-8483-4772-8E79-3D69D8477DE4"];
partition-4 = ["size=4G", "start=1705984",
"name=system", "type=0FC63DAF-8483-4772-8E79-3D69D8477DE4"];
partition-5 = ["size=512M", "start=10094592",
"name=part5", "type=0FC63DAF-8483-4772-8E79-3D69D8477DE4"];
}
}
MBR Example:
partitions: (
{
type = "diskpart";
device = "/dev/sde";
properties: {
labeltype = "dos";
partition-1 = [ "size=64M", "start=2048", "name=bigrootfs", "type=0x83"];
partition-2 = ["size=256M", "start=133120", "name=ldata", "type=0x83"];
partition-3 = ["size=256M", "start=657408", "name=log", "type=0x83"];
partition-4 = ["size=6G", "start=1181696", "name=system", "type=0x5"];
partition-5 = ["size=512M", "start=1183744", "name=part5", "type=0x83"];
partition-6 = ["size=512M", "start=2234368", "name=part6", "type=0x83"];
partition-7 = ["size=16M", "start=3284992", "name=part7", "type=0x6",
"fstype=vfat"];
}
}
Toggleboot Handler
This handler is a script handler. It turns on the bootflag for one of a disk partition if
the partition table is DOS. It reports an error if the table is GPT.
script: (
{
type = "toggleboot";
device = "/dev/sde";
properties: {
partition = "1";
}
}
gpt partition installer
There is a handler gptpart that allows writing an image into a gpt partition selected by
the name. This handler do not modify the gpt partition (type, size, ...), it just writes
the image in the GPT partition.
images: (
{
filename = "u-boot.bin";
type = "gptpart";
device = "/dev/vdb";
volume = "u-boot-1";
offset = "1024";
},
{
filename = "kernel.bin";
type = "gptpart";
device = "/dev/vdb";
volume = "kernel-1";
},
);
gpt partition swap
There is a handler gptswap that allow to swap gpt partitions after all the images were
flashed. This handler only swaps the name of the partition. It coud be useful for a dual
bank strategy. This handler is a script for the point of view of swupdate, so the node
that provides it should be added in the section scripts.
Simple example:
scripts: (
{
type = "gptswap";
device = "/dev/vdb";
properties =
{
swap-0 = [ "u-boot-0" , "u-boot-1" ];
swap-1 = [ "kernel-0" , "kernel-1" ];
};
},
);
Diskformat Handler
This handler checks if the device already has a file system of the specified type.
(Available only if CONFIG_DISKFORMAT is set.) If the file system does not yet exist, it
will be created. In case an existing file system shall be overwritten, this can be
achieved by setting the property force to true.
partitions: (
{
type = "diskformat";
device = "/dev/loop0p1";
properties: {
fstype = "vfat";
force = "true";
}
})
Unique UUID Handler
This handler checks if the device already has a filesystems with a provide UUID. This is
helpful in case the bootloader chooses the device to boot from the UUID and not from the
partition number. One use case is with the GRUB bootloader when GRUB_DISABLE_LINUX_UUID
is not set, as usual on Linux Distro as Debian or Ubuntu.
The handler iterates all UUIDs given in sw-description and raises error if one of them is
found on the device. It is a partition handler and it runs before any image is installed.
partitions: (
{
type = "uniqueuuid";
properties: {
fs-uuid = ["21f16cae-612f-4bc6-8ef5-e68cc9dc4380",
"18e12df1-d8e1-4283-8727-37727eb4261d"];
}
});
BTRFS Partition Handler
This handler is activated if support for BTRFS is on. It allows to created and delete
subvolumes during an update.
partitions: (
{
type = "btrfs";
device = "/dev/loop0p1";
properties: {
command = < one of create" or "delete" >
path = <path for the subvolume>;
mount = "true" or missing;
create-destination = "true" or missing;
}
})
If mount is set, SWUpdate will mount the device and the path is appended to the mountpoint
used with mount. If device is already mounted, path is the absolute path.
BTRFS Snapshot Handler
The handler allows to install a BTRFS snapshot created with the "btrfs send" command.
SWUpdate is using the external "btrfs" utility, that must be installed on the target, and
"btrfs receive" is executed by sending the stream to the command.
All generic features are avaiulable, that means that an srtifact can be streamed by using
the "installed-directly" attribute.
images: (
{
filename = "btrfs-snapshot";
type = "btrfs-receive";
device = <optional, device with BTRFS fs, must be set if tomount is "true">;
properties: {
path = <mandatory, path where to install subvolume>;
btrfs-cmd = <optional, path to btrfs command>;
tomount = <boolean, optional, "true" / "false" >;
}
})
If tomount is set, SWUpdate will temporary mount "device" as BTRFS filesystem and will try
to install the snapshot to path. btrfs-cmd is optional, fallback is /usr/bin/btrfs.
Generic Executor handler
The BTRFS snapshot handler requires to stream an artifact after normal handling
(decompression, decryption, etc.) to the external command "btrfs" without any temporary
copy. The same infrastucture can be used to stream any artifact to any arbitrary external
command that accepts the stream as stdin. This is done with the "executor" handler.
{
filename = "test";
type = "executor";
properties: {
cmd = <mandatory, command to be executed in a pipe>;
}
}
If create-destination is set, SWUpdate will create the destination path of the subvolume
before creating it.
Delta Update Handler
The handler processes a ZCHUNK header and finds which chunks should be downloaded after
generating the corresponding header of the running artifact to be updated. The handler
uses just a couple of attributes from the main setup, and gets more information from the
properties. The attributes are then passed to a secondary handler that will install the
artefact after the delta handler has assembled it. The handler requires ZST because this
is the compression format for Zchunk.
The SWU must just contain the ZCK's header, while the ZCK file is put as it is on the
server. The utilities in Zchunk project are used to build the zck file.
zck -u -h sha256 <artifact>
This will generates a file <arifact>.zck. To extract the header, use the zck_read_header
utility:
HSIZE=`zck_read_header -v <artifact>.zck | grep "Header size" | cut -d':' -f2`
dd if=<artifact>.zck of=<artifact>.header bs=1 count=$((HSIZE))
The resulting header file must be packed inside the SWU.
Properties for delta update handler
┌─────────────┬────────┬──────────────────────────┐
│Name │ Type │ Description │
├─────────────┼────────┼──────────────────────────┤
│url │ string │ This is the URL from │
│ │ │ where the handler will │
│ │ │ download the missing │
│ │ │ chunks. The server must │
│ │ │ support byte range │
│ │ │ header. │
├─────────────┼────────┼──────────────────────────┤
│source │ string │ name of the device or │
│ │ │ file to be used for the │
│ │ │ comparison. │
├─────────────┼────────┼──────────────────────────┤
│chain │ string │ this is the name (type) │
│ │ │ of the handler that is │
│ │ │ called after │
│ │ │ reassembling the │
│ │ │ artifact. │
├─────────────┼────────┼──────────────────────────┤
│max-ranges │ string │ Max number of ranges │
│ │ │ that a server can │
│ │ │ accept. Default value │
│ │ │ (150) should be ok for │
│ │ │ most servers. │
├─────────────┼────────┼──────────────────────────┤
│zckloglevel │ string │ this sets the log level │
│ │ │ of the zcklib. Logs are │
│ │ │ intercepted by SWupdate │
│ │ │ and appear in SWUpdate's │
│ │ │ log. Value is one of │
│ │ │ debug,info │
│ │ │ warn,error,none │
├─────────────┼────────┼──────────────────────────┤
│debug-chunks │ string │ "true", default is not │
│ │ │ set. This activates │
│ │ │ more verbose debugging │
│ │ │ output and the list of │
│ │ │ all chunks is printed, │
│ │ │ and it reports if a │
│ │ │ chunk is downloaded or │
│ │ │ copied from the source. │
├─────────────┼────────┼──────────────────────────┤
│source-size │ string │ This limits the index of │
│ │ │ the source It is helpful │
│ │ │ in case of filesystem in │
│ │ │ much bigger partition. │
│ │ │ It has the value for the │
│ │ │ size or it can be set to │
│ │ │ "detect" and the handler │
│ │ │ will try to find the │
│ │ │ effective size of fs. │
└─────────────┴────────┴──────────────────────────┘
Example:
{
filename = "software.header";
type = "delta";
device = "/dev/mmcblk0p2";
properties: {
url = "http://examples.com/software.zck";
chain = "raw";
source = "/dev/mmcblk0p3";
zckloglevel = "error";
/* debug-chunks = "true"; */
};
}
Memory issue with zchunk
SWUpdate will create the header from the current version, often from a block partition. As
default, Zchunk creates a temporary file with all chunks in /tmp, that is at the end
concatenated to the header and written to the destination file. This means that an amount
of memory equal to the partition (SWUpdate does not compress the chunks) is required. This
was solved with later version of Zchunk - check inside zchunk code if ZCK_NO_WRITE is
supported.
Docker handlers
To improve containers update, a docker set of handlers implements part of the API to
communicate with the docker daemon. Podman (another container solution) has a
compatibility layer for docker REST API and could be used as well, but SWUpdate is
currently not checking if a podman daemon must be started.
Goal of these handlers is just to provice the few API to update images and start
containers - it does not implement the full API.
Docker Load Image
This handler allow to load an image without copying temporarily and push it to the docker
daemon. It implements the /images/load entry point., and it is implemented as "image"
handler. The file should be in a format accepted by docker.
images: (
{
filename = "docker-image.tar"
type = "docker_imageload";
installed-directly = true;
});
The handler checks return value (JSON message) from the daemon, and returns with success
if the image is added.
In case the file must be decompressed, SWUpdate requires the size of the decompressed
image to be passed to the daemon:
images: (
{
filename = "alpine.tar.gz";
type = "docker_imageload";
installed-directly = true;
compressed = "zlib";
properties: {
decompressed-size = "5069312";
};
});
Docker Remove Image
It is implemented as script (post install). Example:
scripts: ( {
type = "docker_imagedelete";
properties: {
name = "alpine:3.4";
};
});
Docker Delete Unused Images
It is implemented as script (post install). Example:
scripts: ( {
type = "docker_imageprune";
});
Docker: container create
It is implemented as post-install script. The script itself is the json file passed to the
daemon to configure and set the container. The container is just created, not started.
For example, having this hello-world.json file:
{
"Image": "hello-world",
"HostConfig": {
"RestartPolicy": {
"Name": "always"
},
"AutoRemove": false
}
}
Creating the container can be done in sw-description with:
scripts: ( {
filename = "hello-world.json";
type = "docker_containercreate";
properties: {
name = "helloworld"; /* Name of container */
}
});
Docker Remove Container
It is implemented as script (post install). Example:
scripts: ( {
type = "docker_containerdelete";
properties: {
name = "helloworld";
};
});
Docker : Start / Stop containers
Examples:
scripts: ( {
type = "docker_containerstart";
properties: {
name = "helloworld";
};
});
scripts: ( {
type = "docker_containerstop";
properties: {
name = "helloworld";
};
});
MONGOOSE DAEMON MODE
Introduction
Mongoose is a daemon mode of SWUpdate that provides a web server, web interface and web
application.
The web application in web-app uses the Node.js package manager and gulp as build tool. It
depends on Bootstrap 4, Font Awesome 5 and Dropzone.js.
Startup
After having configured and compiled SWUpdate with enabled mongoose web server:
./swupdate --help
lists the mandatory and optional arguments to be provided to mongoose. As an example,
./swupdate -l 5 -w '-r ./examples/www/v2 -p 8080' -p 'reboot'
runs SWUpdate in mongoose daemon mode with log-level TRACE and a web server at ‐
http://localhost:8080.
Example
The ready to use example of the web application in the examples/www/v2 directory uses a
Public Domain background.jpg image from pixabay with is released under the Creative
Commons CC0 license. The used favicon.png and logo.png images are made from the SWUpdate
logo and therefore subject to the GNU General Public License version 2. You must comply to
this license or replace the images with your own files.
Customize
You could customize the web application inside the web-app directory. Beside the replace
of the favicon.png, logo.png and background.jpg images inside the images directory you
could customize the Bootstrap colors and settings inside the scss/bootstrap.scss style
sheet. The style sheet changes need a rebuild of the web application source code.
Develop
The development requires Node.js version 18 or greater and a prebuilt SWUpdate project
with enabled mongoose web server and web application interface version 2 support.
1. Enter the web application directory:
cd ./web-app
2. Install the dependencies:
npm install
3. Build the web application:
npm run build
4. Start the web application:
../swupdate -w '-r ./dist -p 8080' -p 'echo reboot'
5. Test the web application:
http://localhost:8080/
6. Pack the web application (optional):
npm run package -- --output swupdate-www.tar.gz
Contribute
Please run the linter before any commit
npm run lint
SURICATTA DAEMON MODE
Introduction
Suricatta is -- like mongoose -- a daemon mode of SWUpdate, hence the name suricatta
(engl. meerkat) as it belongs to the mongoose family.
Suricatta regularly polls a remote server for updates, downloads, and installs them.
Thereafter, it reboots the system and reports the update status to the server, based on an
update state variable currently stored in bootloader's environment ensuring persistent
storage across reboots. Some U-Boot script logics or U-Boot's bootcount feature may be
utilized to alter this update state variable, e.g., by setting it to reflect failure in
case booting the newly flashed root file system has failed and a switchback had to be
performed.
Suricatta is designed to be extensible in terms of the servers supported as described in
Section The Suricatta Interface. Currently, support for the hawkBit server is implemented
via the hawkBit Direct Device Integration API alongside a simple general purpose HTTP
server. The support for suricatta modules written in Lua is not a particular server
support implementation but rather an option for writing such in Lua instead of C.
Running suricatta
After having configured and compiled SWUpdate with enabled suricatta support for hawkBit,
./swupdate --help
lists the mandatory and optional arguments to be provided to suricatta when using hawkBit
as server. As an example,
./swupdate -l 5 -u '-t default -u http://10.0.0.2:8080 -i 25'
runs SWUpdate in suricatta daemon mode with log-level TRACE, polling a hawkBit instance at
http://10.0.0.2:8080 with tenant default and device ID 25.
If multiple server support is compiled in, the -S / --server option or a server entry in
the configuration file's [suricatta] section selects the one to use at run-time. For
convenience, when having support for just one server compiled-in, this is chosen
automatically.
Note that on startup when having installed an update, suricatta tries to report the update
status to its upstream server, e.g., hawkBit, prior to entering the main loop awaiting
further updates. If this initial report fails, e.g., because of a not (yet) configured
network or a currently unavailable hawkBit server, SWUpdate may exit with an according
error code. This behavior allows one to, for example, try several upstream servers
sequentially. If suricatta should keep retrying until the update status is reported to
its upstream server irrespective of the error conditions, this has to be realized
externally in terms of restarting SWUpdate on exit.
After an update has been performed, an agent listening on the progress interface may
execute post-update actions, e.g., a reboot, on receiving DONE. Additionally, a
post-update command specified in the configuration file or given by the -p command line
option can be executed.
Note that at least a restart of SWUpdate has to be performed as post-update action since
only then suricatta tries to report the update status to its upstream server. Otherwise,
succinct update actions announced by the upstream server are skipped with an according
message until a restart of SWUpdate has happened in order to not install the same update
again.
The Suricatta Interface
Support for servers other than hawkBit or the general purpose HTTP server can be realized
by implementing the "interfaces" described in include/channel.h and
include/suricatta/server.h, the latter either in C or in Lua. The channel interface
abstracts a particular connection to the server, e.g., HTTP-based in case of hawkBit. The
server interface defines the logics to poll and install updates. See
corelib/channel_curl.c / include/channel_curl.h and suricatta/server_hawkbit.{c,h} for an
example implementation in C targeted towards hawkBit.
include/channel.h describes the functionality a channel has to implement:
typedef struct channel channel_t;
struct channel {
...
};
channel_t *channel_new(void);
which sets up and returns a channel_t struct with pointers to functions for opening,
closing, fetching, and sending data over the channel.
include/suricatta/server.h describes the functionality a server has to implement:
typedef struct {
server_op_res_t has_pending_action(int *action_id);
server_op_res_t install_update(void);
server_op_res_t send_target_data(void);
unsigned int get_polling_interval(void);
server_op_res_t start(const char *cfgfname, int argc, char *argv[]);
server_op_res_t stop(void);
server_op_res_t ipc(int fd);
void (*help)(void);
} server_t;
These functions constituting a particular suricatta server implementation have to be
registered for being selectable at run-time by calling register_server() (see
include/suricatta/server.h) with a name and a server_t struct pointer implemented in a
__attribute__((constructor)) marked function, see suricatta/server_hawkbit.c as example.
The type server_op_res_t is defined in include/suricatta/suricatta.h. It represents the
valid function return codes for a server's implementation.
In addition to implementing the particular channel and server, the suricatta/Kconfig file
has to be adapted to include a new option so that the new implementation becomes
selectable in SWUpdate's configuration. In the simplest case, adding an option like the
following one for hawkBit into the menu "Server" section is sufficient.
config SURICATTA_HAWKBIT
bool "hawkBit support"
default y
select JSON
help
Support for hawkBit server.
https://projects.eclipse.org/projects/iot.hawkbit
Having included the new server implementation into the configuration, edit
suricatta/Makefile to specify the implementation's linkage into the SWUpdate binary, e.g.,
for the hawkBit example implementation, the following lines add server_hawkbit.o to the
resulting SWUpdate binary if SURICATTA_HAWKBIT was selected while configuring SWUpdate.
ifneq ($(CONFIG_SURICATTA_HAWKBIT),)
obj-$(CONFIG_SURICATTA) += server_hawkbit.o
endif
Support for wfx
The wfx server is supported by the Lua Suricatta module suricatta/server_wfx.lua (cf.
Section Support for Suricatta Modules in Lua). Specifically, it implements a binding to
the Device Artifact Update (DAU) workflow family.
If enabled via CONFIG_SURICATTA_WFX, the wfx Lua Suricatta module is embedded into the
SWUpdate binary so that no extra deployment steps are required. Note that this is purely a
convenience shortcut for the installation of a Lua Suricatta module as described in
Support for Suricatta Modules in Lua.
Job Definition
As being a general purpose workflow executor, wfx doesn't impose a particular job
definition nor schema, except that it's in JSON format. Instead, the job definition is a
contract between the operator creating jobs, each possibly following a different workflow,
and the client(s) executing those jobs in lock-step with the wfx.
The wfx Lua Suricatta module understands job definitions as in the following example (see
job.definition.json_schema in suricatta/server_wfx.lua):
{
"version": "1.0",
"type": ["firmware", "dummy"],
"artifacts": [
{
"name": "Example Device Firmware Artifact",
"version": "1.1",
"uri": "http://localhost:8080/download/example_artifact.swu"
}
]
}
The type list field allows to label update jobs. Labels are sent :-concatenated to the
progress interface on Update Activation. The only predefined label firmware instructs the
wfx Lua Suricatta module to record an installed update to the bootloader environment (see
Bootloader Interface). Within the artifacts list, only the uri field is strictly required
for each artifact while the fields name and version are used for informational purposes,
if provided. Further fields, including top-level fields, are ignored on purpose and may
be freely used, e.g., to enrich the job definition with metadata for update management.
Since wfx is not concerned with the job definition except for conveying it to the client
(i.e. SWUpdate), it can be adapted to specific needs by feeding a different job definition
into the wfx on job creation and adapting the verification and job handling methods in the
wfx Lua Suricatta module's job.definition = {...} Table.
Workflows
The two Device Artifact Update (DAU) workflows wfx.workflow.dau.direct and ‐
wfx.workflow.dau.phased are supported by the wfx Lua Suricatta module. Hint: Use wfx's
wfx-viewer command line tool to visualize the YAML workflows in SVG or PlantUML format.
For each transition in a workflow for which the eligible field contains CLIENT, e.g.,
transitions:
- from: <FROM_STATE>
to: <TO_STATE>
eligible: CLIENT
there has to be a matching transition execution function defined in the wfx Lua Suricatta
module. It executes the client actions to go from state <FROM_STATE> to state <TO_STATE>
and finally sends the new job status to the wfx, updating it:
job.workflow.dispatch:set(
"<FROM_STATE>",
"<TO_STATE>",
--- @param self job.workflow.transition
--- @param job job
--- @return transition.result
function(self, job)
if not job.status
:set({
state = self.to.name, -- resolves to `<TO_STATE>`
message = ("[%s] <TO_STATE> reached"):format(self.to.name),
progress = 100,
})
:send() then
-- Do not try to execute further transitions, yield to wfx.
return transition.result.FAIL_YIELD
end
return transition.result.COMPLETED
end
)
See suricatta/server_wfx.lua for examples of such transition execution functions.
New or adapted workflows are supported by appropriately defining/modifying the transition
execution functions in suricatta/server_wfx.lua -- or taking it as inspiration and
creating a new wfx Lua Suricatta module as described in Support for Suricatta Modules in
Lua.
Update Activation
The Device Artifact Update (DAU) workflows offer a dedicated activation step in the update
steps sequence to decouple artifact installation and activation times so to not, e.g.,
upon a power-cut, prematurely test-boot into the new firmware after installation until the
activation is actually due.
When the activation step is executed, the wfx Lua Suricatta module sends a progress
message (see Getting information on running update) upon which a progress client executes
or schedules activation measures. For example, the following JSON is sent as the progress
message's .info field on activation of the Job Definition example given above:
{
"state": "ACTIVATING",
"progress": 100,
"message": "firmware:dummy"
}
The progress message's .status is PROGRESS, see tools/swupdate-progress.c for details on
how a progress client can be implemented.
Note: The activation message may be sent multiple times if the update activation is
pending, namely on each wfx poll operation and if a new update job is enqueued while the
current one is not activated.
Because of the (predefined) firmware label present, the progress client should initiate or
schedule a reboot of the device in order to test-boot the new firmware. Also because of
the firmware label present, the wfx Lua Suricatta module records the installed update to
the bootloader environment. If this label was missing, no such recording would've been
made which is suitable for, e.g., configuration or application updates.
In order for the this mechanism to work, SWUpdate must not record the update to the
bootloader environment after it has installed it or, in case of configuration or
application updates, must not touch the bootloader environment at all (see the Sections
Update Transaction and Status Marker and bootloader in SWUpdate: syntax and tags with the
default parser).
Hence, for firmware updates requiring a test-boot into the new firmware, the following
properties should be set in the .swu file's sw-description:
software =
{
bootloader_transaction_marker = true;
bootloader_state_marker = false;
...
For configuration or application updates, the following properties apply:
software =
{
bootloader_transaction_marker = false;
bootloader_state_marker = false;
...
Support for general purpose HTTP server
This is a very simple backend that uses standard HTTP response codes to signal if an
update is available. There are closed source backends implementing this interface, but
because the interface is very simple interface, this server type is also suitable for
implementing an own backend server. For inspiration, there's a simple (mock) server
implementation available in examples/suricatta/server_general.py.
The API consists of a GET with Query parameters to inform the server about the installed
version. The query string has the format:
http(s)://<base URL>?param1=val1¶m2=value2...
As examples for parameters, the device can send its serial number, MAC address and the
running version of the software. It is duty of the backend to interpret this - SWUpdate
just takes them from the "identify" section of the configuration file and encodes the URL.
The server answers with the following return codes:
┌──────────┬─────────────┬──────────────────────────┐
│HTTP Code │ Text │ Description │
├──────────┼─────────────┼──────────────────────────┤
│302 │ Found │ A new software is │
│ │ │ available at URL in the │
│ │ │ Location header │
├──────────┼─────────────┼──────────────────────────┤
│400 │ Bad Request │ Some query parameters │
│ │ │ are missing or in wrong │
│ │ │ format │
├──────────┼─────────────┼──────────────────────────┤
│403 │ Forbidden │ Client certificate not │
│ │ │ valid │
├──────────┼─────────────┼──────────────────────────┤
│404 │ Not found │ No update is available │
│ │ │ for this device │
├──────────┼─────────────┼──────────────────────────┤
│503 │ Unavailable │ An update is available │
│ │ │ but server can't handle │
│ │ │ another update process │
│ │ │ now. │
└──────────┴─────────────┴──────────────────────────┘
Server's answer can contain the following headers:
┌──────────────┬───────┬──────────────────────────┐
│Header's name │ Codes │ Description │
├──────────────┼───────┼──────────────────────────┤
│Retry-after │ 503 │ Contains a number which │
│ │ │ tells the device how │
│ │ │ long to wait until ask │
│ │ │ the next time for │
│ │ │ updates. (Seconds) │
├──────────────┼───────┼──────────────────────────┤
│Content-MD5 │ 302 │ Contains the checksum of │
│ │ │ the update file which is │
│ │ │ available under the url │
│ │ │ of location header │
├──────────────┼───────┼──────────────────────────┤
│Location │ 302 │ URL where the update │
│ │ │ file can be downloaded. │
└──────────────┴───────┴──────────────────────────┘
The device can send logging data to the server. Any information is transmitted in a HTTP
PUT request with the data as plain string in the message body. The Content-Type Header
need to be set to text/plain.
The URL for the logging can be set as separate URL in the configuration file or via
--logurl command line parameter:
The device sends data in a CSV format (Comma Separated Values). The format is:
value1,value2,...
The format can be specified in the configuration file. A format For each event can be set.
The supported events are:
┌────────┬──────────────────────────────────┐
│Event │ Description │
├────────┼──────────────────────────────────┤
│check │ dummy. It could send an event │
│ │ each time the server is polled. │
├────────┼──────────────────────────────────┤
│started │ A new software is found and │
│ │ SWUpdate starts to install it │
├────────┼──────────────────────────────────┤
│success │ A new software was successfully │
│ │ installed │
├────────┼──────────────────────────────────┤
│fail │ Failure by installing the new │
│ │ software │
└────────┴──────────────────────────────────┘
The general server has an own section inside the configuration file. As example:
gservice =
{
url = ....;
logurl = ;
logevent : (
{event = "check"; format="#2,date,fw,hw,sp"},
{event = "started"; format="#12,date,fw,hw,sp"},
{event = "success"; format="#13,date,fw,hw,sp"},
{event = "fail"; format="#14,date,fw,hw,sp"}
);
}
date is a special field and it is interpreted as localtime in RFC 2822 format. Each Comma
Separated field is looked up inside the identify section in the configuration file, and if
a match is found the substitution occurs. In case of no match, the field is sent as it is.
For example, if the identify section has the following values:
identify : (
{ name = "sp"; value = "333"; },
{ name = "hw"; value = "ipse"; },
{ name = "fw"; value = "1.0"; }
);
with the events set as above, the formatted text in case of "success" will be:
Formatted log: #13,Mon, 17 Sep 2018 10:55:18 CEST,1.0,ipse,333
Support for Suricatta Modules in Lua
The server_lua.c C-to-Lua bridge enables writing suricatta modules in Lua. It provides the
infrastructure in terms of the interface to SWUpdate "core" to the Lua realm, enabling the
"business logic" such as handling update flows and communicating with backend server APIs
to be modeled in Lua. To the Lua realm, the server_lua.c C-to-Lua bridge provides the same
functionality as the other suricatta modules written in C have, realizing a separation of
means and control. Effectively, it lifts the interface outlined in Section The Suricatta
Interface to the Lua realm.
As an example server implementation, see examples/suricatta/server_general.py for a simple
(mock) server of a backend that's modeled after the "General Purpose HTTP Server" (cf.
Section Support for general purpose HTTP server). The matching Lua suricatta module is
found in examples/suricatta/swupdate_suricatta.lua. Place it in Lua's path so that a
require("swupdate_suricatta") can load it or embed it into the SWUpdate binary by enabling
CONFIG_EMBEDDED_SURICATTA_LUA and setting CONFIG_EMBEDDED_SURICATTA_LUA_SOURCE
accordingly.
The interface specification in terms of a Lua (suricatta) module is found in
suricatta/suricatta.lua.
suricatta
The suricatta table is the module's main table housing the exposed functions and
definitions via the sub-tables described below. In addition, the main functions
suricatta.install() and suricatta.download() as well as the convenience functions
suricatta.getversion(), suricatta.sleep(), and suricatta.get_tmpdir() are exposed:
The function suricatta.install(install_channel) installs an update artifact from a remote
server or a local file. The install_channel table parameter designates the channel to be
used for accessing the artifact plus channel options diverging from the defaults set at
channel creation time. For example, an install_channel table may look like this:
{ channel = chn, url = "https://artifacts.io/update.swu" }
where chn is the return value of a call to channel.open(). The other table attributes,
like url in this example, are channel options diverging from or omitted while channel
creation time, see suricatta.channel. For installing a local file, an install_channel
table may look like this:
{ channel = chn, url = "file:///path/to/file.swu" }
The function suricatta.download(download_channel, localpath) just downloads an update
artifact. The parameter download_channel is as for suricatta.install(). The parameter
localpath designates the output path for the artifact. The suricatta.get_tmpdir() function
(see below) is in particular useful for this case to supply a temporary download location
as localpath. A just downloaded artifact may be installed later using suricata.install()
with an appropriate file:// URL, realizing a deferred installation.
Both, suricatta.install() and suricatta.download() return true, or, in case of error, nil,
a suricatta.status value, and a table with messages in case of errors, else an empty
table.
The function suricatta.getversion() returns a table with SWUpdate's version and patchlevel
fields. This information can be used to determine API (in-)compatibility of the Lua
suricatta module with the SWUpdate version running it.
The function suricatta.sleep(seconds) is a wrapper around SLEEP(3) for, e.g., implementing
a REST API call retry mechanism after a number of given seconds have elapsed.
The function suricatta.get_tmpdir() returns the path to SWUpdate's temporary working
directory where, e.g., the suricatta.download() function may place the downloaded
artifacts.
suricatta.status
The suricatta.status table exposes the server_op_res_t enum values defined in
include/util.h to the Lua realm.
suricatta.notify
The suricatta.notify table provides the usual logging functions to the Lua suricatta
module matching their uppercase-named pendants available in the C realm.
One notable exception is suricatta.notify.progress(message) which dispatches the message
to the progress interface (see Getting information on running update). Custom progress
client implementations listening and acting on custom progress messages can be realized
using this function.
All notify functions return nil.
suricatta.pstate
The suricatta.pstate table provides a binding to SWUpdate's (persistent) state handling
functions defined in include/state.h, however, limited to the bootloader environment
variable STATE_KEY defined by CONFIG_UPDATE_STATE_BOOTLOADER and defaulting to ustate. In
addition, it captures the update_state_t enum values.
The function suricatta.pstate.save(state) requires one of suricatta.pstate's "enum" values
as parameter and returns true, or, in case of error, nil. The function
suricatta.pstate.get() returns one of suricatta.pstate's "enum" values or, in case of
error, STATE_ERROR.
suricatta.server
The suricatta.server table provides the sole function
suricatta.server.register(function_p, purpose). It registers a Lua function "pointed" to
by function_p for the purpose purpose which is defined by suricatta.server's "enum"
values. Those enum values correspond to the functions defined in the interface outlined in
the Section on The Suricatta Interface.
In addition to these functions, the two callback functions CALLBACK_PROGRESS and
CALLBACK_CHECK_CANCEL can be registered optionally: The former can be used to upload
progress information to the server while the latter serves as dwlwrdata function (see
include/channel_curl.h) to decide on whether an installation should be aborted while the
download phase.
For details on the (callback) functions and their signatures, see the interface
specification suricatta/suricatta.lua and the documented example Lua suricatta module
found in examples/suricatta/swupdate_suricatta.lua.
The suricatta.server.register() function returns true, or, in case of error, nil.
suricatta.channel
The suricatta.channel table captures channel handling for suricatta Lua modules. The
single function suricatta.channel.open(options) creates and opens a channel to a server.
Its single parameter options is a table specifying the channel's default options such as
proxy, retries, usessl, strictssl, or headers_to_send. For convenience, options that may
change per request such as url, content-type, or headers_to_send may be set as defaults on
channel creation time while being selectively overruled on a per request basis. The
channel options currently supported to be set are listed in the suricatta.channel.options
table. In essence, the options parameter table is the Lua table equivalent of
include/channel_curl.h's channel_data_t.
The suricatta.channel.open(options) function returns a channel table which is either
passed to the suricatta.install() and suricatta.download() functions or used directly for
communication with a server. More specifically, it has the three functions
• get(options) for retrieving information from the server,
• put(options) for sending information to the server, and
• close() for closing the channel.
The get() and put() functions' single parameter options is a per-request channel option
table as described above.
The functions get() and put() return true, or, in case of error, nil, a suricatta.status
value, and an operation result table. The latter contains the fields:
• http_response_code carrying the HTTP error code,
• format as one of suricatta.channel.content's options,
• raw_reply if options contained format = suricatta.channel.content.RAW,
• json_reply if options contained format = suricatta.channel.content.JSON, and
• the HTTP headers received in the received_headers table, if any.
The suricatta.channel.content "enum" table defines the "format", i.e., the response body
content type and whether to parse it or not:
• NONE means the response body is discarded.
• RAW means the raw server's reply is available as raw_reply.
• JSON means the server's JSON reply is parsed into a Lua table and available as
json_reply.
The suricatta.channel.method "enum" table defines the HTTP method to use for a request
issued with the put(options) function, i.e., POST, PATCH, or PUT as specified in the
options parameter table via the method attribute. In addition to the HTTP method, the
request body's content is set with the request_body attribute in the options parameter
table.
As a contrived example, consider the following call to a channel's put() function
...
local res, _, data = channel.put({
url = string.format("%s/%s", base_url, device_id),
content_type = "application/json",
method = suricatta.channel.method.PATCH,
format = suricatta.channel.content.NONE,
request_body = "{ ... }"
})
...
that issues a HTTP PATCH to some URL with a JSON content without having interest in the
response body.
More examples of how to use a channel can be found in the example suricatta Lua module
examples/suricatta/swupdate_suricatta.lua.
suricatta.bootloader
The suricatta.bootloader table exposes SWUpdate's bootloader environment modification
functions to suricatta Lua modules.
The enum-like table suricatta.bootloader.bootloaders holds the bootloaders SWUpdate
supports, i.e.
suricatta.bootloader.bootloaders = {
EBG = "ebg",
NONE = "none",
GRUB = "grub",
UBOOT = "uboot",
},
The function suricatta.bootloader.get() returns the currently selected bootloader in terms
of a suricatta.bootloader.bootloaders field value.
The function suricatta.bootloader.is(name) takes one of suricatta.bootloader.bootloaders's
field values as name and returns true if it is the currently selected bootloader, false
otherwise.
The functions in the suricatta.bootloader.env table interact with the currently selected
bootloader's environment:
The function suricatta.bootloader.env.get(variable) retrieves the value associated to
variable from the bootloader's environment.
The function suricatta.bootloader.env.set(variable, value) sets the bootloader
environment's key variable to value.
The function suricatta.bootloader.env.unset(variable) deletes the bootloader environment's
key variable.
The function suricatta.bootloader.env.apply(filename) applies all key=value lines of a
local file filename to the currently selected bootloader's environment.
CONFIG FOR HAWKBIT UNDER SSL/TLS USING PRIVATE CA / SUB CA
A user-contributed recipe based on hawkBit (0.2.0-SNAPSHOT) + swupdate (v2018.03)
Purpose
Use HTTPS on a hawkBit server to avoid server spoofing. Anonymous client connections are
authorized.
Recipe
1. On the PKI:
• Create a pkcs#12 (.p12) file, rolling server key, server, private CA, sub CA certs
into a single file.
• Use a password on the server key you won't be ashamed of.
• Also create a single .pem file for the private CA + sub-CA
2. On the hawkBit host:
• hawkBit uses the Java KeyStore to access credentials, but a JKS is not designed
apparently to hold CA certs, which is a problem for private CAs. The workaround is to
make it gulp an entire pkcs#12 file.
• It looks like a JKS like this cannot have a password different from the one
protecting the .p12. Keytool also seems to have a little tendency to destruct the
.jks if you change your mind and want to change the password... Basically do
everything you need with openssl and use only keytool for generating the .jks file.
The following command imports a .p12 into a "pkcs12 Java keystore", keeping the same
password:
keytool -importkeystore -srckeystore hb-pass.p12 -srcstoretype pkcs12 \
-destkeystore hb-pass.jks -deststoretype pkcs12 \
-alias 1 -deststorepass <password_of_p12>
Then you need to adapt application.properties of the hawkBit server to make use of the
keystore. There are extra requirements to make hawkBit send artifacts via HTTPS.
This is the relevant part of <hawkBit
dir>/hawkbit-runtime/hawkbit-update-server/src/main/resources/application.properties:
# HTTPS mode working w/ swupdate
# See also https://docs.spring.io/spring-boot/docs/1.4.7.RELEASE/reference/html/howto-embedded-servlet-containers.html#howto-configure-ssl
# https://github.com/eclipse/hawkbit/issues/618
#
# Need to run as root to use port 443
server.hostname=hb.domain
server.port=8443
#
# Overriding some of hawkbit-artifactdl-defaults.properties is required
hawkbit.artifact.url.protocols.download-http.protocol=https
hawkbit.artifact.url.protocols.download-http.port=8443
#
# Upgrades http:8443 to https:8443
# Would redirect + upgrade http:80 to https:443
security.require-ssl=true
server.use-forward-headers=true
#
# Server cert+key w/ private CA + subCA
# See also https://stackoverflow.com/questions/906402/how-to-import-an-existing-x509-certificate-and-private-key-in-java-keystore-to-u
# http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html (2008, still relevant!?)
#
# File .jks is a .p12 imported via keytool. Only one password supported, set from openssl.
server.ssl.key-store=hb-pass.jks
server.ssl.key-password=password
server.ssl.key-store-password=password-yes_the_same_one
...
3. On the swupdate client host(s):
• The client needs the private CA certificate(s) to authenticate the server.
• There is a setting in swupdate to specify the path to a single CA cert, not a
directory. Beyond that, libcurl looks into /etc/ssl/certs. So we're using a compound
"CA chain" .pem file to hold both private CA and sub-CA in our preferred location.
This is the relevant part of /etc/swupdate/swupdate.conf:
...
suricatta :
{
tenant = "default";
id = "machineID";
url = "https://hb.domain:8443";
nocheckcert = false;
cafile = "/etc/swupdate/priv-cachain.pem"; /* CA + sub CA in one file */
/* sslkey = anon client: do not set; */
/* sslcert = anon client: do not set; */
...
SWUPDATE: API FOR EXTERNAL PROGRAMS
Overview
SWUpdate contains an integrated web-server to allow remote updating. However, which
protocols are involved during an update is project specific and differs significantly.
Some projects can decide to use FTP to load an image from an external server, or using
even a proprietary protocol. The integrated web-server uses this interface.
SWUpdate has a simple interface to let external programs to communicate with the
installer. Clients can start an upgrade and stream an image to the installer, querying
then for the status and the final result. The API is at the moment very simple, but it can
easy be extended in the future if new use cases will arise.
API Description
The communication runs via Unix Domain Socket (UDS). The socket path is determined by
• the CONFIG_SOCKET_CTRL_PATH compile-time configuration,
• $RUNTIME_DIRECTORY/sockinstctrl if $RUNTIME_DIRECTORY is set,
• $TMPDIR/sockinstctrl if $TMPDIR is set, or
• /tmp/sockinstctrl,
in this order, and taken over from systemd or created by SWUpdate. This socket should,
however, not be used directly but instead by the Client Library explained below.
The exchanged packets are described in network_ipc.h
typedef struct {
int magic;
int type;
msgdata data;
} ipc_message;
Where the fields have the meaning:
• magic : a magic number as simple proof of the packet
• type : one of REQ_INSTALL, ACK, NACK, GET_STATUS, POST_UPDATE, SWUPDATE_SUBPROCESS,
SET_AES_KEY
• msgdata : a buffer used by the client to send the image or by SWUpdate to report back
notifications and status.
The client sends a REQ_INSTALL packet and waits for an answer. SWUpdate sends back ACK or
NACK, if for example an update is already in progress.
After the ACK, the client sends the whole image as a stream. SWUpdate expects that all
bytes after the ACK are part of the image to be installed. SWUpdate recognizes the size
of the image from the CPIO header. Any error lets SWUpdate to leave the update state, and
further packets will be ignored until a new REQ_INSTALL will be received. [image]
It is recommended to use the client library to communicate with SWUpdate. On the lower
level with direct socket communication, it cannot be guaranteed that the structures will
remain compatible in the future. The client library was affected by this issue, too, and
it is changed to accept an opaque interface that will survive API changes. Compatibility
layers could be added on-demand in the future due to API changes.
Client Library
Functions to start an update
A library simplifies the usage of the IPC making available a way to start asynchronously
an update.
The library consists of one function and several call-backs.
int swupdate_async_start(writedata wr_func, getstatus status_func,
terminated end_func, void *req, ssize_t size)
typedef int (*writedata)(char **buf, int *size);
typedef int (*getstatus)(ipc_message *msg);
typedef int (*terminated)(RECOVERY_STATUS status);
swupdate_async_start creates a new thread and start the communication with SWUpdate,
triggering for a new update. The wr_func is called to get the image to be installed. It
is responsibility of the callback to provide the buffer and the size of the chunk of data.
The getstatus call-back is called after the stream was downloaded to check how upgrade is
going on. It can be omitted if only the result is required.
The terminated call-back is called when SWUpdate has finished with the result of the
upgrade.
Example about using this library is in the examples/client directory.
The req structure is casted to void to ensure API compatibility. Am user should
instantiate it as struct swupdate_request. This contains fields that can control the
update process:
struct swupdate_request {
unsigned int apiversion;
sourcetype source;
int dry_run;
size_t len;
char info[512];
char software_set[256];
char running_mode[256];
};
A user should first call swupdate_prepare_req()
void swupdate_prepare_req(struct swupdate_request *req);
This fills the request structure with default values. After that, the user can fill the
other fields as:
• sourcetype : one of SOURCE_UNKNOWN, SOURCE_WEBSERVER, SOURCE_SURICATTA,
SOURCE_DOWNLOADER, SOURCE_LOCAL
• dry_run : one of RUN_DEFAULT (set from command line), RUN_DRYRUN, RUN_INSTALL.
• info, len : a variable length data that can be forwarded to the progress interface.
The installer in SWUpdate does not evaluate it.
• software_set and running_mode : this allows one to set the selection for the update,
it should be accepted by swupdate (-q, --accepted-select option when swupdate is
launched).
Functions to set AES keys
The key for decryption can be set with command line parameter (see -K), but it is possible
to set it via IPC. In this way, each update could have a different key.
int swupdate_set_aes(char *key, char *ivt)
The key is for AES-256. The length for key and ivt are then defined by the algorithm amd
they are passed as ASCII string, so the length must be 64 bytes for key and 32 bytes for
IVT.
Functions to control SWUpdate
int ipc_send_cmd(ipc_message *msg);
ipc_send_cmd is used to send a command to a SWUpdate subprocess (as suricatta). The
function is synchron, that means it clocks until the subprocess has answered with ACK or
NACK. This function sets type to SWUPDATE_SUBPROCESS. The caller must then set the other
fields in message according to the destination. The msgdata field is a structure as:
struct {
sourcetype source; /* Who triggered the update */
int cmd; /* Optional encoded command */
int timeout; /* timeout in seconds if an aswer is expected */
unsigned int len; /* Len of data valid in buf */
char buf[2048]; /*
* Buffer that each source can fill
* with additional information
*/
}
The caller fills source with the subprocess that accepts the command. Values of cmd are in
network_ipc.h.
Messages for suricatta
suricatta accepts messages in JSON format. The message must be formatted in the buf field
of the message data.
Setting the polling time
{ "polling" : <value in seconds, range 0..X>}
Setting it to 0 has the special meaning that the polling time is retrieved from the
Backend (if this is supported by the server).
Enable / disable Suricatta daemon
{ "enable" : true }
{ "enable" : false }
Set custom device attributes for Suricatta (for Hawkbit implementation)
{ "identify" : [
{
"name" : "customizableAttributeOne",
"value" : "valueOne"
},
{
"name" : "customizableAttributeTwo",
"value" : "valueTwo"
}
]}
New attributes can be added at runtime, and existing attributes can be modified in the
same way. Changes will be reflected on the server in the next poll iteration.
Trigger a check on the server
This is useful in case the device is mostly offline, and when it is online, it should
check immediately if an update exists and run it. In fact, after enabling the suricatta
daemon, the update follows the usual states, and the daemon waits for a polling time
before loading the new software. This command forces an update (if available) without
changing the polling time.
{ "trigger" : true }
Activate an already installed Software
After a software was installed, the new software boots and if everything runs fine, an
acknowledge should be sent to the hawkBit server. If this feature is used, for example to
let the end user decide if the new software is accepted, the parameters used by the
installation should be stored during the update process.
{ "id" : <action id>,
"finished" : "success", "failure", "none",
"execution" : ["closed", "proceeding", canceled", "rejected", "resumed"]
"details" : [ ]
}
Get hawkBit Server Status
To provide the hawkBit server status to other processes, it can be requested by sending an
empty message with message type CMD_GET_STATUS.
The response is a JSON object containing the hawkBit server status <status>. <status> is
a number representing the value of the channel_op_res_t enum from channel_op_res.h. As the
hawkBit server is polled, its status can only be updated when it has been polled.
Therefore the response also contains the time <time>, when the hawkBit server has been
polled the last time. It is provided as ISO 8601 date and time string.
(2021-10-14T13:42:37.000+00)
{ "server" : {
"status" : <status>
"time" : <time>
}
}
An example application can be found under tools/swupdate-gethawkbitstatus.c
API to the integrated Webserver
The integrated Webserver provides REST resources to push a SWU package and to get inform
about the update process. This API is based on HTTP standards. There are to kind of
interface:
• Install API to push a SWU and to restart the device after update.
• A WebSocket interface to send the status of the update process.
Install API
POST /upload
This initiates an update: the initiator sends the request and start to stream the SWU in
the same way as described in API Description. The POST /upload request contains:
• the SWU file in its body
• the following header field
Content-Type: multipart/form-data; boundary...
For example to stream the SWU file and start the update procedure using curl:
curl -F file=@update_file.swu http://host:8080/upload
Restart API
POST /restart
If configured (see post update command), this request will restart the device.
WebSocket API
The integrated Webserver exposes a WebSocket API. The WebSocket protocol specification
defines ws (WebSocket) and wss (WebSocket Secure) as two new uniform resource identifier
(URI) schemes that are used for unencrypted and encrypted connections, respectively and
both of them are supported by SWUpdate. A WebSocket provides full-duplex communication
but it is used in SWUpdate to send events to an external host after each change in the
update process. The Webserver sends JSON formatted responses as results of internal
events.
The response contains the field type, that defines which event is sent.
Event Type
┌────────┬──────────────────────────────────┐
│type │ Description of event │
├────────┼──────────────────────────────────┤
│status │ Event sent when SWUpdate's │
│ │ internal state changes │
├────────┼──────────────────────────────────┤
│source │ Event to inform from which │
│ │ interface an update is received │
├────────┼──────────────────────────────────┤
│info │ Event with custom message to be │
│ │ passed to an external process │
├────────┼──────────────────────────────────┤
│message │ Event that contains the error │
│ │ message in case of error │
├────────┼──────────────────────────────────┤
│step │ Event to inform about the │
│ │ running update │
└────────┴──────────────────────────────────┘
Status Change Event
This event is sent when the internal SWUpdate status change. Following status are
supported:
IDLE
START
RUN
SUCCESS
FAILURE
DONE
Example:
{
"type": "status",
"status": "SUCCESS"
}
Source Event
This event informs from which interface a SWU is loaded.
{
"type": "source",
"source": "WEBSERVER"
}
The field source can have one of the following values:
UNKNOWN
WEBSERVER
SURICATTA
DOWNLOADER
LOCAL
Info Event
This event forwards all internal logs sent with level=INFO.
{
"type": "info",
"source": < text message >
}
Message Event
This event contains the error message in case of failure.
Fields for message event
┌───────┬──────────────────────────────────┐
│name │ Description │
├───────┼──────────────────────────────────┤
│status │ "message" │
├───────┼──────────────────────────────────┤
│level │ "3" in case of error, "6" as │
│ │ info │
├───────┼──────────────────────────────────┤
│text │ Message associated to the event │
└───────┴──────────────────────────────────┘
Example:
{
"type": "message",
"level": "3",
"text" : "[ERROR] : SWUPDATE failed [0] ERROR core/cpio_utils.c : ",
}
Step event
This event contains which is the current step running and which percentage of this step is
currently installed.
Fields for step event
┌────────┬──────────────────────────────────┐
│name │ Description │
├────────┼──────────────────────────────────┤
│number │ total number of steps N for this │
│ │ update │
├────────┼──────────────────────────────────┤
│step │ running step in range [1..N] │
├────────┼──────────────────────────────────┤
│name │ filename of artefact to be │
│ │ installed │
├────────┼──────────────────────────────────┤
│percent │ percentage of the running step │
└────────┴──────────────────────────────────┘
Example:
{
"type": "step",
"number": "7",
"step": "2",
"name": "rootfs.ext4.gz",
"percent": "18"
}
GETTING INFORMATION ON RUNNING UPDATE
It is often required to inform the operator about the status of the running update and not
just to return if the update was successful or not. For example, if the target has a
display or a remote interface, it can be forwarded which is reached percentage of the
update to let estimate how much the update will still run. SWUpdate has an interface for
this ("progress API"). An external process can register itself with SWUpdate, and it will
receive notifications when something in the update was changed. This is different from the
IPC API, because the last one is mainly used to transfer the SWU image, and it is only
possible to poll the interface to know if the update is still running.
API Description
An external process registers itself to SWUpdate with a connect() request to the domain
socket "/tmp/swupdateprog" as per default configuration of SWUpdate. There is no
information to send, and SWUpdate simply inserts the new connection into the list of
processes to be informed. SWUpdate will send a frame back after any change in the update
process with the following data (see include/progress_ipc.h):
struct progress_msg {
unsigned int magic; /* Magic Number */
unsigned int status; /* Update Status (Running, Failure) */
unsigned int dwl_percent; /* % downloaded data */
unsigned long long dwl_bytes; /* total of bytes to be downloaded */
unsigned int nsteps; /* No. total of steps */
unsigned int cur_step; /* Current step index */
unsigned int cur_percent; /* % in current step */
char cur_image[256]; /* Name of image to be installed */
char hnd_name[64]; /* Name of running handler */
sourcetype source; /* Interface that triggered the update */
unsigned int infolen; /* Len of data valid in info */
char info[2048]; /* additional information about install */
};
The single fields have the following meaning:
• magic is not yet used, it could be added for simply verification of the frame.
• status is one of the values in swupdate_status.h (START, RUN, SUCCESS, FAILURE,
DOWNLOAD, DONE).
• dwl_percent is the percentage of downloaded data when status = DOWNLOAD.
• dwl_bytes is the total number of bytes that are to be downloaded.
• nsteps is the total number of installers (handlers) to be run.
• cur_step is the index of the running handler. cur_step is in the range 1..nsteps
• cur_percent is the percentage of work done inside the current handler. This is useful
when updating a slow interface, such as a slow flash, and signals which is the
percentage of image already copied into the destination.
• cur_image is the name of the image in sw-description that is currently being
installed.
• hnd_name reports the name of the running handler.
• source is the interface that triggered the update.
• infolen length of data in the following info field.
• info additional information about installation.
As an example for a progress client, tools/swupdate-progress.c prints the status on the
console and drives "psplash" to draw a progress bar on a display.
LANGUAGE BINDINGS
Overview
In general, SWUpdate is agnostic to a particular language it is operated from, thanks to
SWUpdate's socket-based control and progress APIs for external programs. As long as the
language of choice has proper socket (library) support, SWUpdate can be operated with it.
However, for convenience, a Lua language binding in terms of a shared library, currently
lua_swupdate.so.0.1, is provided.
Lua Language Binding
The Lua language binding is realized in terms of the lua_swupdate module that defines
three bindings, namely for the control interface, the progress interface, and a
convenience function yielding a table holding all local network interfaces including their
IP addresses and submasks.
The lua_swupdate Lua module interface specification that details what functionality is
made available by bindings/lua_swupdate.c is found in bindings/lua_swupdate.lua. It serves
as reference, for mocking purposes, and type checking thanks to the EmmyLua-inspired
annotations.
Note that, depending on the filesystem location of the Lua binding's shared library, Lua's
package.cpath may have to be adapted by setting the environment variable LUA_CPATH,
modifying package.cpath prior to a require('lua_swupdate'), or , as last resort, using
package.loadlib() instead of require('lua_swupdate').
Control Interface
The lua_swupdate module's control interface binding conveniently makes SWUpdate's
socket-based control API available to pure Lua.
The binding is captured in the swupdate_control object that is returned by a call to
swupdate.control(). This object offers the three methods connect(), write(<chunkdata>),
and close():
The connect() method initializes the connection to SWUpdate's control socket, sends
REQ_INSTALL, and waits for ACK or NACK, returning the socket connection file descriptor,
mostly for information purposes, or, in case of an error, nil plus an error message.
The artifact's data can then be sent to SWUpdate via the write(<chunkdata>) method,
returning true, or, in case of errors, nil plus an error message.
Finally, the close() method closes the connection to SWUpdate's control socket after which
it waits for SWUpdate to complete the update transaction and executes the post-install
command, if given.
The following example snippet illustrates how to use the control interface binding:
local artifact = io.open("/some/path/to/artifact.swu", "rb" )
swupdate = require('lua_swupdate')
local ctrl = swupdate.control()
if not ctrl:connect() then
-- Deliberately neglecting error message.
io.stderr:write("Error connecting to SWUpdate control socket.\n")
return
end
while true do
local chunk = artifact:read(1024)
if not chunk then break end
if not ctrl:write(chunk) then
-- Deliberately neglecting error message.
io.stderr:write("Error writing to SWUpdate control socket.\n")
break
end
end
local res, msg = ctrl:close()
if not res then
io.stderr:write(string.format("Error finalizing update: %s\n", msg))
end
Progress Interface
The lua_swupdate module's progress interface binding conveniently makes SWUpdate's
socket-based progress API available to pure Lua.
The binding is captured in the swupdate_progress object that is returned by a call to
swupdate.progress(). This object offers the three methods connect(), receive(), and
close():
The connect() method connects to SWUpdate's progress socket, waiting until the connection
has been established. Note that it is only really required to explicitly call connect() to
reestablish a broken connection as the swupdate_progress object's instantiation already
initiates the connection.
The receive() method returns a table representation of the struct progress_msg described
in the progress interface's API description.
The close() method deliberately closes the connection to SWUpdate's progress socket.
IPv4 Interface
For convenience, the lua_swupdate module provides the ipv4() method returning a table
holding the local network interfaces as the table's keys and their space-separated IP
addresses plus subnet masks as respective values.
BOOTLOADER INTERFACE
Overview
SWUpdate has bindings to various bootloaders in order to store persistent state
information across reboots. Currently, the following bootloaders are supported:
• A fake bootloader called "Environment in RAM",
• EFI Boot Guard,
• U-Boot, and
• GRUB.
The actual (sub)set of bootloaders supported is a compile-time choice. At run-time, the
compile-time set default bootloader interface implementation is used unless overruled to
use another bootloader interface implementation via the -B command line switch or a
configuration file (via the bootloader setting in the globals section, see
examples/configuration/swupdate.cfg).
Note that the run-time support for some bootloaders, currently U-Boot and EFI Boot Guard,
relies on loading the respective bootloader's environment modification shared library at
run-time. Hence, even if support for a particular bootloader is compiled-in, the according
shared library must be present and loadable on the target system at run-time for using
this bootloader interface implementation. This allows, e.g., distributions to ship a
generic SWUpdate package and downstream integrators to combine this generic package with
the appropriate bootloader by just providing its environment modification shared library.
Bootloader Interface Description
The bootloader interface implementations are located in bootloader/. Each bootloader has
to implement the interface functions as defined in include/bootloader.h, more precisely
char *env_get(const char *name);
int env_set(const char *name, const char *value);
int env_unset(const char *name);
int apply_list(const char *filename);
which retrieve a key's value from the bootloader environment, set a key to a value in the
bootloader environment, delete a key-value pair from the bootloader environment, and apply
the key=value pairs found in a file.
Then, each bootloader interface implementation has to register itself to SWUpdate at
run-time by calling the register_bootloader(const char *name, bootloader *bl) function
that takes the bootloader's name and a pointer to struct bootloader as in
include/bootloader.h which is filled with pointers to the respective above mentioned
interface functions. If the bootloader setup fails and hence it cannot be successfully
registered, e.g., because the required shared library for environment modification cannot
be loaded, NULL is to be returned as pointer to struct bootloader.
For example, assuming a bootloader named "trunk" and (static) interface functions
implementations do_env_{get,set,unset}() as well as do_apply_list() in a
bootloader/trunk.c file, the following snippet registers this bootloader to SWUpdate at
run-time:
static bootloader trunk = {
.env_get = &do_env_get,
.env_set = &do_env_set,
.env_unset = &do_env_unset,
.apply_list = &do_apply_list
};
__attribute__((constructor))
static void trunk_probe(void)
{
(void)register_bootloader(BOOTLOADER_TRUNK, &trunk);
}
with
#define BOOTLOADER_TRUNK "trunk"
added to include/bootloader.h as a single central "trunk" bootloader name definition
aiding in maintaining the uniqueness of bootloader names. This new "trunk" bootloader
should also be added to the Suricatta Lua Module interface specification's bootloader
Table suricatta.bootloader.bootloaders = { ... } in suricatta/suricatta.lua.
ATTENTION:
Take care to uniquely name the bootloader.
See, e.g., bootloader/{uboot,ebg}.c for examples of a bootloader using a shared
environment modification library and bootloader/{grub,none}.c for a simpler bootloader
support example.
Bootloader Build System Integration
A bootloader support implementation needs to be registered to the kconfig build system.
First, the bootloader support implementation, named "trunk" and implemented in
bootloader/trunk.c for example, needs to be added to bootloader/Kconfig in the Bootloader
Interfaces menu as follows:
...
menu "Bootloader"
menu "Bootloader Interfaces"
...
config BOOTLOADER_TRUNK
bool "TrUnK Bootloader"
help
Support for the TrUnK Bootloader
https://github.com/knurt/trunk
Then, in order to enable the compile-time selection of the "trunk" bootloader as default,
add a section to the Default Bootloader Interface choice submenu of the Bootloader menu as
follows:
choice
prompt "Default Bootloader Interface"
help
Default bootloader interface to use if not explicitly
overridden via configuration or command-line option
at run-time.
...
config BOOTLOADER_DEFAULT_TRUNK
bool "TrUnK"
depends on BOOTLOADER_TRUNK
help
Use TrUnK as default bootloader interface.
Finally, bootloader/Makefile needs to be adapted to build the "trunk" bootloader support
code, given BOOTLOADER_TRUNK was enabled:
obj-$(CONFIG_BOOTLOADER_TRUNK) += trunk.o
If the "trunk" bootloader, for example, requires loading a shared environment modification
library, then Makefile.flags needs to be adapted as well, e.g., as follows:
ifeq ($(CONFIG_BOOTLOADER_TUNK),y)
LDLIBS += dl
endif
META-SWUPDATE: BUILDING WITH YOCTO
Overview
The Yocto-Project is a community project under the umbrella of the Linux Foundation that
provides tools and template to create the own custom Linux-based software for embedded
systems.
Add-on features can be added using layers. meta-swupdate is the layer to cross-compile the
SWUpdate application and to generate the compound SWU images containing the release of the
product. It contains the required changes for mtd-utils and for generating Lua. Using
meta-SWUpdate is a straightforward process. As described in Yocto's documentation about
layers, you should include it in your bblayers.conf file to use it.
Add meta-SWUpdate as usual to your bblayers.conf. You have also to add meta-oe to the
list.
In meta-SWUpdate there is a recipe to generate an initrd with a rescue system with
SWUpdate. Use:
MACHINE=<your machine> bitbake swupdate-image
You will find the result in your tmp/deploy/<your machine> directory. How to install and
start an initrd is very target specific - please check in the documentation of your
bootloader.
What about libubootenv ?
This is a common issue when SWUpdate is built. SWUpdate depends on this library, that is
generated from the U-Boot's sources. This library allows one to safe modify the U-Boot
environment. It is not required if U-Boot is not used as bootloader. If SWUpdate cannot
be linked, you are using an old version of U-Boot (you need at least 2016.05). If this is
the case, you can add your own recipe for the package u-boot-fw-utils, adding the code for
the library.
It is important that the package u-boot-fw-utils is built with the same sources of the
bootloader and for the same machine. In fact, the target can have a default environment
linked together with U-Boot's code, and it is not (yet) stored into a storage. SWUpdate
should be aware of it, because it cannot read it: the default environment must be linked
as well to SWUpdate's code. This is done inside the libubootenv.
If you build for a different machine, SWUpdate will destroy the environment when it tries
to change it the first time. In fact, a wrong default environment is taken, and your board
won't boot again.
To avoid possible mismatch, a new library was developed to be hardware independent. A
strict match with the bootloader is not required anymore. The meta-swupdate layer contains
recipes to build the new library (libubootenv) and adjust SWUpdate to be linked against
it. To use it as replacement for u-boot-fw-utils:
• set PREFERRED_PROVIDER_u-boot-fw-utils = "libubootenv"
• add to SWUpdate config:
CONFIG_UBOOT=y
With this library, you can simply pass the default environment as file
(u-boot-initial-env). It is recommended for new project to switch to the new library to
become independent from the bootloader.
The swupdate class
meta-swupdate contains a class specific for SWUpdate. It helps to generate the SWU image
starting from images built inside the Yocto. It requires that all components, that means
the artifacts that are part of the SWU image, are present in the Yocto's deploy directory.
This class should be inherited by recipes generating the SWU. The class defines new
variables, all of them have the prefix SWUPDATE_ in the name.
• SWUPDATE_IMAGES : this is a list of the artifacts to be packaged together. The list
contains the name of images without any extension for MACHINE or filetype, that are
added automatically. Example :
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
• SWUPDATE_IMAGES_FSTYPES : extension of the artifact. Each artifact can have multiple
extension according to the IMAGE_FSTYPES variable. For example, an image can be
generated as tarball and as UBIFS for target. Setting the variable for each artifact
tells the class which file must be packed into the SWU image.
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
• SWUPDATE_IMAGES_NOAPPEND_MACHINE : flag to use drop the machine name from the artifact
file. Most images in deploy have the name of the Yocto's machine in the filename. The
class adds automatically the name of the MACHINE to the file, but some artifacts can be
deployed without it.
SWUPDATE_IMAGES_NOAPPEND_MACHINE[my-image] = "1"
• SWUPDATE_SIGNING : if set, the SWU is signed. There are 3 allowed values: RSA, CMS,
CUSTOM. This value determines used signing mechanism.
• SWUPDATE_SIGN_TOOL : instead of using openssl, use SWUPDATE_SIGN_TOOL to sign the image.
A typical use case is together with a hardware key. It is available if SWUPDATE_SIGNING
is set to CUSTOM
• SWUPDATE_PRIVATE_KEY : this is the file with the private key used to sign the image
using RSA mechanism. Is available if SWUPDATE_SIGNING is set to RSA.
• SWUPDATE_PASSWORD_FILE : an optional file containing the password for the private key.
It is available if SWUPDATE_SIGNING is set to RSA.
• SWUPDATE_CMS_KEY : this is the file with the private key used in signing process using
CMS mechanism. It is available if SWUPDATE_SIGNING is set to CMS.
• SWUPDATE_CMS_CERT : this is the file with the certificate used in signing process using
CMS method. It is available if SWUPDATE_SIGNING is set to CMS.
• SWUPDATE_AES_FILE : this is the file with the AES password to encrypt artifact. A new
fstype is supported by the class (type: enc). SWUPDATE_AES_FILE is generated as output
from openssl to create a new key with
openssl enc -aes-256-cbc -k <PASSPHRASE> -P -md sha1 -nosalt > $SWUPDATE_AES_FILE
To use it, it is enough to add IMAGE_FSTYPES += "enc" to the artifact. SWUpdate
supports decryption of compressed artifact, such as
IMAGE_FSTYPES += ".ext4.gz.enc"
Automatic sha256 in sw-description
The swupdate class takes care of computing and inserting sha256 hashes in the
sw-description file. The attribute sha256 must be set in case the image is signed. Each
artifact must have the attribute:
sha256 = "$swupdate_get_sha256(artifact-file-name)"
For example, to add sha256 to the standard Yocto core-image-full-cmdline:
sha256 = "$swupdate_get_sha256(core-image-full-cmdline-machine.ubifs)";
The name of the file must be the same as in deploy directory.
BitBake variable expansion in sw-description
To insert the value of a BitBake variable into the update file, pre- and postfix the
variable name with "@@". For example, to automatically set the version tag:
version = "@@DISTRO_VERSION@@";
Automatic versions in sw-description
By setting the version tag in the update file to $swupdate_get_pkgvar(<package-name>) it
is automatically replaced with PV from BitBake's package-data-file for the package
matching the name of the provided <package-name> tag. For example, to set the version tag
to PV of package u-boot:
version = "$swupdate_get_pkgvar(u-boot)";
To automatically insert the value of a variable from BitBake's package-data-file different
to PV (e.g. PKGV) you can append the variable name to the tag:
$swupdate_get_pkgvar(<package-name>@<package-data-variable>) For example, to set the
version tag to PKGV of package u-boot:
version = "$swupdate_get_pkgvar(u-boot@PKGV)";
Using checksum for version
It is possible to use the hash of an artifact as the version in order to use
"install-if-different". This allows versionless artifacts to be skipped if the artifact
in the update matches the currently installed artifact.
In order to use the hash as the version, the sha256 hash file placeholder described above
in Automatic sha256 in sw-description must be used for version.
Each artifact must have the attribute:
version = "@artifact-file-name"
The name of the file must be the same as in deploy directory.
Template for recipe using the class
DESCRIPTION = "Example recipe generating SWU image"
SECTION = ""
LICENSE = ""
# Add all local files to be added to the SWU
# sw-description must always be in the list.
# You can extend with scripts or whatever you need
SRC_URI = " \
file://sw-description \
"
# images to build before building swupdate image
IMAGE_DEPENDS = "core-image-full-cmdline virtual/kernel"
# images and files that will be included in the .swu image
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
# a deployable image can have multiple format, choose one
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
SWUPDATE_IMAGES_FSTYPES[uImage] = ".bin"
inherit swupdate
Simplified version for just image
In many cases there is a single image in the SWU. This is for example when just rootfs is
updated. The generic case described above required an additional recipe that must be
written and maintained. For this reason, a simplified version of the class is introduced
that allowed to build the SWU from the image recipe.
Users just need to import the swupdate-image class. This already sets some variables. A
sw-description must still be added into a files directory, that is automatically searched
by the class. User still needs to set SWUPDATE_IMAGE_FSTYPES[your image] to the fstype
that should be packed into the SWU - an error is raised if the flag is not set.
In the simple way, your recipe looks like
<your original recipe code>
SWUPDATE_IMAGES_FSTYPES[<name of your image>] = <fstype to be put into SWU>
inherit swupdate-image
What about grub ?
In order to use swupdate with grub, swupdate needs to be configured to use grub. Some of
the imporatant configurations are CONFIG_GRUBENV_PATH="/path/to/grubenv", where
"/path/to/grubenv" is thepath to grub environment. Example: "/boot/EFI/BOOT/grubenv".
The grubenv file should be created using grub-editenv tool, because it is a 1024-byte
file, therefore, any modification using nano or vim will only corrupt the file, and grub
will not be able to use it.
You can create a grubenv file using these commands for instance:
GRUBENV="/path/to/grubenv"
grub-editenv $GRUBENV create
grub-editenv $GRUBENV set rootfs=2
grub-editenv $GRUBENV set kernel=2
grub-editenv is a tool that is integrated to yocto.
When the grubenv file is created, grub should be configured to use it. This configuration
should be in the configuration file grub.cfg. Here is an example of grub.cfg that loads
the environment file before booting:
# Take a kernel and a rootfs by default in case grubenv is corrupted
rootfs=1
kernel=1
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
default=boot
# set timeout to zero to boot without timeout
timeout=0
# load grubenv the environment file that contains the value of rootfs and kernel variables
load_env -f "/path/to/grubenv"
# detect which memory contains 5 partitions
for i in 1 2 3 4 5; do if [ -d (hd${i},gpt5)/ ]; then drive=${i};fi ; done
# detect which rootfs should we boot with
if [ ${rootfs} = "1" ]; then rootfs_part=4 ; elif [ ${rootfs} = "2" ]; then rootfs_part=5 ; fi
# detect which kernel should we boot with
if [ ${kernel} = "1" ]; then kernel_part="(hd${drive},gpt2)" ; elif [ ${kernel} = "2" ]; then kernel_part="(hd${drive},gpt3)" ; fi
# The menuentry that is used to boot (more can be added if it is wanted)
menuentry 'boot'{
linux ${kernel_part}/bzImage root=/dev/mmcblk1p${rootfs_part} rw rootwait quiet console=ttyS2,115200 console=tty0 panic=5
}
The grub.cfg above is merely an example, and can be modified as the user wants to, as long
as it loads the environment variables,and it boots correctly using these environment
variables.
SWUPDATE BEST PRACTICE
This is intended as general rule to integrate SWUpdate into a custom project.
SWUpdate is an update agent and it is thought to be a framework. This means it is highly
configurable and SWUpdate should be configured to fit into a project, not vice versa.
SWUpdate makes just a few requirements on the system and it has no fixed update schema.
There is no restriction on how many partitions or which storage you are using. In some
more complex cases, the update depends on a lot of conditions, and SWUpdate can run
differently according to the mode a device is started in. Think about SWUpdate not being
a ready-to-use updater but a framework, and hence you should first write a meaningful:
Update Concept
Take your time and write first an update concept for your device. It is not wasted time.
You have to imagine conditions when an update is not working, and try to write down the
use cases when an update can fail and how the device can be restored. SWUpdate installs
new software, but a successful update means that the new software is started and runs
flawlessly. The interface with the bootloader (or the one that starts the software) must
be checked in details. A successful update means:
• SWUpdate runs successfully,
• the device reboots,
• the bootloader can start the new software, and
• the new software runs, makes some consistency checks, and declares that the transaction
(that is from old to new software) is terminated.
This means that some coordination between the bootloader and the update agent is
necessary. In most cases, this is done via persistent variables that are available to
both SWUpdate and the bootloader. SWUpdate has two built-in variables:
• recovery_status: this is set when SWUpdate starts to write to the device, and it is
unset after the installation completed with success or it is set to failed in case of
error. A bootloader can use it to check if an update was interrupted. This is a must in
case a single copy of the software is on the device.
• ustate: this triggers a state machine. SWUpdate sets it to 1 (INSTALL) after an update.
The bootloader can use it to check whether a new software must be tested. The
bootloader should implement a counter mechanism to check how many times it tried to
start a new software. When a threshold is reached, the bootloader should declare the new
software as buggy, and proceed with a fallback in case the old software is available.
A fallback is always initiated by the bootloader, because it knows if the new software is
running. It should toggle the copies and start the old software set. To communicate this
to user space and to SWUpdate, the bootloader sets the ustate variable to 3 (FAILED).
SWUpdate uses this information in case the result must be forwarded to an external server
(like a backend). There is a time window when a fallback can take place. In fact, after a
reboot and some attempts, the update transaction is declared successful or failed, and
later a new update can be executed. When a new update runs, the status of the stand-by
copy is unknown, because it could be the result of an interrupted update. Running an
incomplete software can lead to unpredictable results and must be strongly avoided. A
common pattern for a toggling in the bootloader is:
• ustate is not set or set to 0 (no update pending). The bootloader runs the configured
copy and won't ever toggle. In case of failure, a rescue system can be started if
available.
• ustate is set to 1 (INSTALLED): the new software is under test. The bootloader
initiates a fallback if the new software is not running and sets ustate to 3 (FAILED).
If the new software runs, it is the duty of user space (often SWUpdate or the init
script for SWUpdate) to reset ustate to 0. Note that resetting the variable is project
specific, and it could be set as last action after having sufficiently checked that the
new software is running. This includes performing in the application a database
migration, starting communicating with peers, whatever.
A possible diagram is shown in next picture - it is not supposed to work with any project,
but it gives an idea how fallback is working together with the bootloaders. [image]
Check in advance which security topics are relevant for your project. This includes:
• signed images (SWU is verified before installing), and then which crypto mechanism is
used (RSA keys, certificates, PKI)
• encryption for artifacts
• under which user and group SWUpdate and other components are allowed to run. Set user
id and group id if not root in swupdate.cfg.
• if any version can be installed or if you forbid a downgrade, and then be sure to pass
the range of versions you allow via --M, -R and --max-version.
• hardware-software compatibility check and how your device knows which hardware revision
is running.
Reduce dependencies to minimum
An update should be possible in any condition. Even if the system is degraded or in a bad
shape, if an update can work, the device can be functional again without returning it back
to the factory. SWUpdate is thought to be self contained: that means it does not make use
of external tools. If your system is degraded and filesystems get corrupted, there are
less chances to restore it if the update calls external tools. SWUpdate is started at boot
time and there are good chances it succeeds even if your system has some (software) flaws.
Be careful to make an update depending on your application or try to reduce the
dependencies. In fact, the application is updated often and an introduction of new bugs
can make the device no longer updatable. Take the dependencies under control, and if you
have any, be sure that the update is still working. You can fix any bugs if the update
works, but not anymore if the device cannot be updated.
Make a risk analysis
A more accurate analysis brings less surprises in the field. Think twice about what you
want to update, which components should be updated, and the risks of updating a single
point of failure. Very often, this means the bootloader. Compare risks and benefits: it
happens in many projects that having the possibility (with some risk) to update the
bootloader is better that returning the devices back to service. A cost / benefits
analysis should be part of the integration of the update agent.
SWUpdate builtin configuration
SWUpdate has a compile time configuration. The default configuration delivered with
meta-swupdate is not suitable for most projects. The easy way to check configuration in
Yocto is to run:
bitbake -c menuconfig swupdate
Outside Yocto, just run in SWUpdate's sources:
make menuconfig
Check security, bootloader, and which handlers should be installed. They depend strongly
on your project. If you build with OE, add a swupdate_%.bbappend to one of your layers,
and put the resulting configuration file as defconfig that can be fetched. Please review
the following configuration:
• Security settings
• Interfaces required (where the software is coming from). Disable the interface you do
not need.
• Handlers required for your project. Disable what you do not need, but consider if you
could need some of them in future. As example, you can safely disable ubivol if you do
not use raw NAND, but you can let archive enabled if you plan to install artifacts from
tarballs in future.
• It is highly recommended to enable Lua to extend runtime behavior.
SWUpdate startup
An easy way to start SWUpdate is provided only with meta-swupdate and Yocto. A generic
SystemV init script or a systemd unit for SWUpdate are executing a script swupdate.sh,
that is delivered together with the SWUpdate binaries. The script goes through
/etc/swupdate/conf.d/ and sources all found files. The integrator can use a set of
predefined variables to configure SWUpdate's command line parameters.
• SWUPDATE_WEBSERVER_ARGS : This string is passed if the webserver must be started. It
consists of the webserver specific parameters. If this variable is set, the script will
add -w to the list of parameters. Note: meta-swupdate contains a default configuration
for SWUPDATE_WEBSERVER_ARGS, that uses /www as document root for the Website and default
port 8080.
• SWUPDATE_SURICATTA_ARGS : Suricatta (backend) specific parameters. There is no default.
• SWUPDATE_ARGS : Parameters not belonging to Webserver or Suricatta.
Note that swupdate.sh sources the files in sorted order, so it is possible to override the
variables with a configuration file whose filename is loaded at the end. Preferred style
is to use SystemV like names, for example 10-webserver, 11-suricatta, and so on.
Write sw-description
sw-description is the central file that describes a new software release and how a release
must be installed. It should be a consequence of the update concept. There is not a
single right way. SWUpdate heavily uses 'selections' and links to extract just one part of
the whole sw-description, that can be used for different situations and different ways to
run the device. One use case for selections is to implement the dual-copy (often referred
to as A/B) mode: one selection contains instructions for one copy, the other for the
second copy. Which copy is the stand-by must be detected before running SWUpdate and
passed via the -e <selection,mode> switch. Other methods set up a link to the standby
storage (like /dev/standby) during boot. Or the standby device can be detected at runtime
with an embedded-script, as part of sw-description, with Lua code. Please note that for
the last case, SWUpdate is extended with functions exported to the Lua context that
simplify the detection. SWUpdate exports a getroot() function that returns type and value
for the device used as rootfs. See SWUpdate documentation for a complete list of functions
exported by SWUpdate that can be used in Lua. An embedded Lua script must just start with
require ('swupdate')
to make use of them.
Use OE variables as much as possible
meta-swupdate replaces a special construct in sw-description with the values of build
variables. The recognized construct in sw-description is delimited by @@, that is
@@VARIABLE-NAME@@. The exception (for compatibility reasons) is the automatic generation
of sha256. The syntax in that case is :
sha256 = "$swupdate_get_sha256(<name of artifact>)"
You can again use variable substitution for artifact names. Example:
sha256 = "$swupdate_get_sha256(@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@)";
Please note that each variable is double delimited (at the beginning and at the end) by
@@.
Deliver your scripts instead of relying on them being installed
You have the freedom to call any tools during an update. However, take care if you are
using some tools from the running rootfs / current software. This implies that the current
software is running flawlessly, as well as the tools you are calling. And this may not
always be the case.
Prefer Lua to shell scripts
Shell scripts are very popular, and they are often used even when they are not strictly
required. They can raise security issues. In fact, take as example a simple shell script.
Goal of rootkits is often the shell, because taking control of the shell means to control
the whole device. If the shell is compromised, the whole system is compromised. Running a
shell script means that SWUpdate should call "fork" followed by an "exec". This means also
that many resources are duplicated in the child process, and it could cause a further
problem if system is getting rid of resources. A better approach is to use Lua and to
deliver the scripts inside the SWU. In fact, the Lua interpreter is linked to SWUpdate and
runs in context of the SWUpdate process without forking a child process. Shell is not
involved at all. Of course, Lua scripts should be written to be self-contained, too, and
executing external tools should be done only if unavoidable.
Use installed-directly when possible
SWUpdate can be enabled for zero-copy (or streaming mode), that is the incoming SWU is
analyzed on the fly and it is installed by the associated handler without any temporary
copy. If this is not set, SWUpdate creates a temporary copy in $TMPDIR before passing it
to the handlers. Note that $TMPDIR generally points to a RAMDISK and storing files there
reduces the amount of memory available for the application. It makes sense to disable the
flag in case the artifact is a single point of failure. A typical example could be the
bootloader (not duplicated on the devices), and if the SWU is corrupted or the connection
gets broken, the board is left in a bricked state. It makes sense then to download the
whole artifact before installing.
Always enable sha256 verification
The SWU image is a CPIO archive with CRC (new ASCII format), but the check in CPIO is very
weak. Do not trust it, but enable sha256 for each artifact.
Always set the "type" attribute
SWUpdate sets some default handler if the type is not set. Do not use it, but set
explicitly the type (that is, which handler should install the artifact) in
sw-description.
Do not rely on install order
SWUpdate does not require that artifacts are put into the CPIO in a specific order. The
exception is sw-description, that must be the first file in a SWU. Avoid dependencies
inside the SWU, that is an artifact that can be installed only after another one was
installed before. If you really need it, for example if you want to install a file into a
filesystem provided as image, disable installed-directy for the file and enable it for the
filesystem image.
Do not drop atomicity !
SWUpdate guarantees atomicity as long as you don't do something that simply breaks it. As
example, think about the bootloader's environment. In an sw-description, there is a
specific section where the environment can be set, adding / modifying / deleting
variables. SWUpdate does not change single variables, but generates the resulting new
environment for the supported bootloader and this is written in one shot in a way (for
U-Boot / EFIBootguard, not for GRUB) that is power-cut safe. You can of course change the
environment in a postinstall script, like in the following way (for U-Boot):
fw_setenv var1 val1
fw_setenv var2 val2
fw_setenv var3 val3
fw_setenv var4 val4
fw_setenv var5 val5
If a power cut happens during two calls of fw_setenv, the environment is in an
intermediate state and this can brick the device.
Plan to have a rescue system
Even if you have a double-copy setup, something can go wrong. Plan to have a rescue system
(swupdate-image in meta-swupdate) and to install it on a separate storage than the main
system, if it is possible. This helps when the main storage is corrupted, and the device
can be restored in the field without returning it back to the factory. Plan to update the
rescue system as well: it is software, too, and its bugs should be fixed, too.
DELTA UPDATE WITH SWUPDATE
Overview
The size of update packages is steadily increasing. While once the whole software was just
a bunch of megabytes, it is not unusual now that OS and application on devices running
Linux as OS reach huge size of Gigabytes.
Several mechanisms can be used to reduce the size of downloaded data. The resulting images
can be compressed. However, this is not enough when bandwidth is important and not cheap.
It is very common that a device will be upgraded to a version that is similar to the
running one but add new features and solves some bugs. Specially in case of just fixes,
the new version is pretty much equal as the original one. This asks to find methods to
download just the differences with the current software without downloading a full image.
In case an update is performed from a known base, we talk about delta updates. In the
following chapter some well known algorithms are considered and verified if they can be
integrated into SWUpdate. The following criteria are important to find a suitable
algorithm:
• license must be compatible with GPLv2
• good performance for smaller downloads, but not necessarily the best one.
• SWUpdate remains with the concept to deliver one package (SWU), the same
independently from the source where the SWU is stored (USB, OTA, etc.)
• It must comply to SWUpdate's security requirements (signed images, privilege
separation, etc.)
Specific ad-hoc delta updates mechanisms can be realized when the nature of the updated
files is the same. It is always possible with SWUpdate to install single files, but
coherency and compatibility with the running software must be guaranteed by the integrator
/ manufacturer. This is not covered here: the scope is to get an efficient and content
unaware delta mechanism, that can upgrade in differential mode two arbitrary images,
without any previous knowledge about what they content.
FOSS projects for delta encoding
There are several algorithms for delta encoding, that is to find the difference between
files, generally in binary format. Only algorithms available under a compatible FOSS
license (GPLv2) are considered for SWUpdate. One of the goals in SWUpdate is that it
should work independently which is the format of the artifacts. Very specialized algorithm
and libraries like Google's Courgette used in Chromium will give much better results, but
it works on programs (ELF files) and take advantages of the structure of compiled code. In
case of OTA update, not only software, but any kind of artifact can be delivered, and this
includes configuration data, databases, videos, docs, etc.
librsync
librsync is an independent implementation for rsync and does not use the rsync protocol.
It is well suited to generate offline differential update and it is already integrated
into SWUpdate. However, librsync takes the whole artifact and generates a differential
image that is applied on the whole image. It gives the best results in terms of reduced
size when differences are very small, but the differential output tends to be very large
as soon as the differences are meaningful. Differential images created for SWUpdate show
that, as soon as the difference larger is, the resulting delta image can even become
larger as the original one.
SWUpdate supports librsync as delta encoder via the rdiff handler.
xdelta
xdelta uses the VCDIFF algorithm to compute differences between binaries. It is often used
to deliver smaller images for CD and DVD. The resulting images are created from an
installed image that should be loaded entirely in main memory. For this reason, it does
not scale well when the images are becoming larger and it is unsuitable for embedded
systems and SWUpdate.
casync
casync is, according to his author. a tool for distributing images. It has several
interesting aspects that can be helpful with OTA update. Files itself are grouped
together in chunks and casync creates a "Chunk storage" where each chunk is stored on a
separate file. The chunk storage is part of the delivery, and it must be stored on a
server. casync checks if the chunk is already present on the target, and if not download
it. If this seems to be what is required, there are some drawbacks if casync should be
integrated in SWUpdate:
• because of the nature of casync, each chunk is a separate file. This cause a huge
number of new connections, because each file is a separate GET on the server. The
overhead caused to re-instantiate connection is high on small devices, where SSL
connections are also increasing CPU load. There are downloads of hundreds or
thousands of small files just to recreate the original metadata file.
• casync has no authentication and verification and the index (.caidx or .caibx) are
not signed. This is known, but casync goals and scopes are outside the ones on
embedded devices.
• it is difficult to deliver a whole chunk storage. The common usage for OTA is to
deliver artifacts, and they should be just a few. Thousands of files to be delivered
to let casync to compute the new image is not practical for companies: they have a
new "firmware" or "software" and they need an easy way to deliver this file (the
output from their build system) to the devices. In some cases, they are even not
responsible for that, and the firmware is given to another authority that groups all
packages from vendors and realizes a sort of OTA service.
• casync is quite a huge project - even if it was stated that it will be converted into
a library, this never happened. This makes difficult to interface to SWUpdate, and
using it as external process is a no way in SWUpdate for security reason. It breaks
privilege separation, and adds a lot of code that is difficult to maintain.
For all these reasons, even if the idea of a chunk storage is good for an OTA updater,
casync is not a candidate for SWUpdate. A out-of-the-box solution cannot be found, and it
is required to implement an own solution that better suits for SWUpdate.
Zchunk - compression format
zchunk seems to combine the usage of a chunk storage without having to deliver it on a
server. zchunk is a FOSS project released under BSD by its author. The goal of this
project is something else: zchunk creates a new compression format that adds the ability
to download the differences between new and old file. This matches very well with
SWUpdate. A zchunk file contains a header that has metadata for all chunks, and according
to the header, it is known which chunks must be downloaded and which ones can be reused.
zchunk has utilities to download itself the missing chunks, but it could be just used to
find which part of an artifact must be downloading, and SWUpdate can go on with its own
way to do this.
One big advantage on this approach is that metadata and compressed chunks are still bound
into a single file, that can be built by the buildsystem and delivered as it is used to.
The updater needs first the metadata, that is the header in zchunk file, and processes it
to detect which chunks need to be downloaded. Each chunk has its own hash, and the chunks
already available on the device are verified against the hash to be sure they are not
corrupted.
Zchunk supports multiple sha algorithms - to be compatible with SWUpdate, zchunk should be
informed to generate sha256 hashes.
Design Delta Update in SWUpdate
For all reasons stated before, zchunk is chosen as format to deliver delta update in
SWUpdate. An artifact can be generated in ZCK format and then the ZCK's header (as
described in format) can be extracted and added to the SWU. In this way, a ZCK file is
signed (and if requested compressed and/or encrypted) as part of the SWU, and loading
chunks from an external URL can be verified as well because the corresponding hashes are
already verified as part of the header.
Changes in ZCHUNK project
Zchunk has an API that hides most of its internal, and provides a set of tools for
creating and downloading itself a file in ZCK format. Nevertheless, Zchunk relies on
hashes for the compressed (ZST) chunks, and it was missing for support for uncompressed
data. To combine SWUpdate and zchunk, it is required that a comparison can be done between
uncompressed data, because it is unwanted that a device is obliged to compress big amount
of data just to perform a comparisons. A short list of changes in the Zchunk project is:
• create hashes for uncompressed data and extend format to support it. The header must
be extended to include both size and hash of uncompressed data.
• make the library embedded friendly, that means reports errors in case of failure
instead of exiting and find a suitable way to integrate the log output for the
caller.
• allow to use sha256 (already foreseen in zchunk) as this is the only hash type used
in SWUpdate.
• add API to allow an external caller to take itself the decision if a chunk must be
downloaded or reused.
These changes were merged into Zchunk project - be sure to get a recent version of Zchunk,
at least with commit 1b36f8b5e0ecb, that means newer as 1.1.16.
Most of missing features in Zchunk listed in TODO for the project have no relevance here:
SWUpdate already verifies the downloaded data, and there is no need to add signatures to
Zchunk itself.
Integration in sw-description
The most important part in a Zchunk file is the header: this contains all metadata and
hashes to perform comparisons. The zck tool splits a file in chunks and creates the
header. Size of the header are know, and the header itself can be extracted from the ZCK
file. The header will be part of sw-description: this is the header for the file that
must be installed. Because the header is very small compared to the size of the whole file
(quite 1 %), this header can be delivered into the SWU.
Integration in SWUpdate: the delta handler
The delta handler is responsible to compute the differences and to download the missing
parts. It is not responsible to install the artifact, because this breaks the module
design in SWUpdate and will constrain to have just one artifact type, for example
installing as raw or rawfile. But what about if the artifact should be installed by a
different handler, for example UBI, or a custom handler ? The best way is that the delta
handler does not install, but it creates the stream itself so that this stream can be
passed to another (chained) handler, that is responsible for installing. All current
SWUpdate's handlers can be reused: each handler does not know that the artifact is coming
with separate chunks and it sees just a stream as before. The delta handler has in short
the following duties:
• parse and understand the ZCK header
• create a ZCK header from the file / partition used as source for the comparison
• detect which chunks are missing and which one must be copied.
• build a mixer that copies and downloads all chunks and generates a stream for the
following handler.
• detect any error coming form the chained handler.
Because the delta handler requires to download more data, it must start a connection to
the storage where the original ZCK is stored. This can lead to security issues, because
handlers run with high privileges because they write into the hardware. In fact, this
breaks privilege separation that is part of SWUpdate design. To avoid this, the delta
handler does not download itself. A separate process, that can runs with different userid
and groupid, is responsible for this. The handler sends a request to this process with a
list of ranges that should be downloaded (see HTTP Range request). The delta handler does
not know how the chunks are downloaded, and even if using HTTP Range Request is the most
frequent choice, it is open to further implementations. The downloader process prepares
the connection and asks the server for ranges. If the server is not able to provide
ranges, the update aborts. It is in fact a requirement for delta update that the server
storing the ZCK file is able to answer to HTTP Range Request, and there is no fallback to
download the full file. An easy IPC is implemented between the delta handler and the
downloader process. This allows to exchange messages, and the downloader can inform the
handler if any error occurs so that the update can be stopped. The downloader will send a
termination message when all chunks will be downloaded. Because the number of missing
chunks can be very high, the delta handler must sends and organize several requests to the
downloader, and tracking each of them. The downloader is thought as dummy servant: it
starts the connection, retrieves HTTP headers and data, and sends them back to the caller.
The delta handler is then responsible to parse the answer, and to retrieve the missing
chunks from the multipart HTTP body.
Creation of ZCK Header and ZCK file for SWUpdate
Zchunk supports more SHA algorithms and it sets as default SHA512/128. This is not
compatible with SWUpdate that just support SHA256. Be sure to generate header and chunks
with SHA256 support. You have to enable the generation of hashes for uncompressed chunk,
too. A possible usage of the tool is:
zck --output <output file> -u --chunk-hash-type sha256 <artifact, like rootfs>
The output is the ZCK file with all chunks. This file should be put on a Webserver
accessible to the target, and that supports Range Request (RFC 7233). All modern Webserver
support it.
The SWU must just contain the header. This can be easy extracted from the ZCK file with:
HSIZE=`zck_read_header -v <ZCK file> | grep "Header size" | cut -d':' -f2`
dd if=<ZCK FILE> of=<ZCK HEADER file> bs=1 count=$((HSIZE))
Using ZCK tools to foresee download size
There are tools that can be used at build time to know how many chunks should be
downloaded when a device is upgrading from a known version. You can use zck_cmp_uncomp
from the test directory:
../build/test/zck_cmp_uncomp --verbose <uncompressed old version file> <ZCK file>
This prints a list with all chunks, marking them with SRC if they are the same in the old
version and they should not retrieved and with DST if they are new and must be downloaded.
The tool show at the end a summary with the total number of bytes of the new release
(uncompressed) and how many bytes must be downloaded for the upgrade. Please remember
that these value are just payload. SWUpdate reports a summary, too, but it takes into
account also the HTTP overhead (headers, etc.), so that values are not the same and the
ones from SWUpdate are slightly bigger.
SWUPDATE-CLIENT
swupdate-client is a small tool that sends a SWU image to a running instance of SWUpdate.
It can be used if the update package (SWU) is downloaded by another application external
to SWUpdate. It is an example how to use the IPC to forward an image to SWUpdate.
SYNOPSIS
swupdate-client [OPTIONS] <image.swu to be installed>...
DESCRIPTION
-h print help and exit
-d ask the server to only perform a dry-run
-e <software>,<mode> select software image set and source (for example: stable,main)
-q go quiet, resets verbosity
-v go verbose, essentially print upgrade status messages from server
-p ask the server to run post-update commands if upgrade succeeds
SWUPDATE-PROGRESS
swupdate-progress tries to connect to a running instance of SWUpdate to get the status of
a running update.
SYNOPSIS
swupdate-progress [option]
DESCRIPTION
swupdate-progress is an example how to connect to SWUpdate via the progress interface. It
shows on stdout a simple bar with the percent indication of the current update and reports
the result of the update. It can optionally drive "psplash" or execute a script after an
update.
-c Use colors to show results on stdout
-e Command to be execute after an update
-p send percentage to psplash
-r optionally reboot the target after a successful update
-s path to progress IPC socket in case the default is not taken
-w waits for a SWUpdate connection instead of exit with error
-q don't print progress bar
-h print a help
SWUPDATE-IPC
swupdate-ipc sends a specific command to SWUpdate. The following commands are enabled:
aes
send a new aes key to SWUpdate
setversion
sends a range of versions that can be accepted.
gethawkbit
return status of the connection to Hawkbit.
sendtohawkbit
send data to the Hawkbit Server. A typical use case is acknowledgement for activation by
an application or operator after a new software has been installed. The tool can forward
the result for the activation to the hawkBit server.
sysrestart
send a restart command after a network update
SYNOPSIS
swupdate-ipc <cmd> [option]
Where cmd is one of the listed above.
DESCRIPTION
aes <key> <ivt>
AES key to be used for decryption
setversion <min> <max> <current>
configure the accepted range of versions
hawkbitcfg
configuration for Hawkbit Module
-h help
-p allows one to set the polling time (in seconds)
-e enable suricatta mode
-d disable suricatta mode
swupdate-sendtohawkbit <action id> <status> <finished> <execution> <detail 1> <detail 2>
..
Send Acknowledge to Hawkbit server after an update if SWUpdate is set to wait for.
sysrestart
Used with SWU handler, allow one to perform a network restart
-r optionally reboot the target after a successful update
-w waits for a SWUpdate connection instead of exit with error
-s <path>
path to progress IPC socket
HELP AND SUPPORT
Mailing List
There is a mailing list for this project:
swupdate@googlegroups.com
Issue related to the project or to the documentation are discussed here.
SWUpdate Flyer
A short description about the project and the features (in English and German) can be
found in the flyer
Workshop and SWUpdate integration in project
For quick integration of SWUpdate in your project, you could be interested in the Training
Commercial support and board integration
Please check for services <https://swupdate.org/services> if you need professional support
or you need help to get SWUpdate on your device.
Talks about SWUpdate
• Software Update in Embedded Systems by Stefano Babic
• Updating Embedded Linux devices in field by Chris Simmonds
• OpenEmbedded in the Real World by Scott Murray
• [RFC] Device-side support for software update in AGL by Matt Porter
• Open Source secure software updates for Linux-based IVI systems by Arthur Taylor
• How do you update your embedded Linux devices? by Daniel Sangorrin / Keijiro Yano
• Comparison of Linux Software Update Technologies by Matt Porter
• Software update for IoT: the current state of play by Chris Simmonds, ELCE 2016, ‐
Slides, Video
• OSS Remote Firmware Updates for IoT-like Projects by Silvano Cirujano Cuesta, ELCE
2016, Slides ELCE 2016, Video ELCE 2016
• System Upgrade with SWUpdate by Gabriel Huau, ELC 2017, Slides ELC 2017, Video ELC
2017
• BoF: Secure OTA Collaboration, by Ricardo Salveti and Alan Bennett, ELCE 2017
• Orchestrated Android-Style System Upgrades for Embedded Linux by Diego Rondini, ELCE
2017, Slides Android-Style, Video Android-Style
• Updating an Embedded System with SWUpdate Framework by Stefano Babic, ELCE 2017, ‐
Slides ELCE 2017, Video ELCE 2017
• Upgrading buildroot based devices with SWUpdate by Angelo Compagnucci, LinuxLab 2018,
Slides LinuxLab 2018, Video LinuxLab 2018,
• Evolution of (OTA) Update in the IoT world by Stefano Babic, ELC 2019, Slides ELC
2019, Video ELC 2019,
• Introduction of CIP Software Updates Working Group by Akihiro Suzuki, CIP Mini Summit
2019, Slides CIP 2019,
• There is No Store For Self-Driving Car Parts by Stephen Segal and Matt Fornero
(Cruise LLC), ELC 2020, Slides Cruise ELC 2020, Video Cruise
• Secure Boot and Over-the-Air Updates - That's simple, no ? by Jan Kiszka (Siemens
AG), ELC 2020, Slides Secure OTA ELC 2020, Video Secure OTA
• Diving into SWUpdate: adding new platform support with Yocto/OE! by Pierre-Jean
Texier, LiveEmbedded 2020, Slides Diving into SWUpdate, Video Diving into SWUpdate
• Implementing UEFI-based Secure Boot + OTA Update for Embedded ARM Devices by Jan
Kiszka & Christian Storm, ELCE 2022 Slides Implementing UEFI Secure Boot + OTA
Update, Video Implementing UEFI-based Secure Boot + OTA Update for Embedded ARM
Devices
• Delta OTA Update with SWUpdate by Stefano Babic, ELCE 2022 Slides Delta OTA Update
with SWUpdate, Video Delta OTA Update with SWUpdate
• Ligthning Talk: SWUpdate Over CAN Bus - Can it ? by Stefano Babic, EOSS 2023 Video
SWUpdate Over CAN Bus
• Best Practices with SWUpdate by Stefano Babic, Embedded Recipes 2023 Slides Best
Practices with SWUpdate, Video Best Practices with SWUpdate
• Delta Updates: Making Updates Leaner by Felix Moessbauer & Jan Kiszka, Siemens AG,
Embedded Open Source Summit 2024 Slides Delta Updates: Making Updates Leaner, Video
Delta Updates: Making Updates Leaner
Useful references
• Boundary Devices, Using SWUpdate to upgrade your system
• Présentation de Software Update (French)
• Easy OS upgrades with SWUpdate
• SWUpdate for feature-rich IoT applications
• Implement swupdate - replacing opkg based updating, VictronEnergy
• Variscite, SWUpdate
• Updating Embedded Linux Devices: SWUpdate
• Approach to Software Update Management, Pelux
• SOTA System, Pelux
• Building a Linux system for the STM32MP1: remote firmware updates, Bootlin
CONTRIBUTING TO SWUPDATE
Contributions are welcome ! Please follow the following guideline for contributions.
Contribution Checklist
These are mostly general recommendations and are common practice in a lot of FOSS
projects.
• use git to manage your changes [recommended]
• follow as much as possible kernel codestyle [recommended] Nevertheless, some rules are
not so strict as in kernel. The maximum line length can be extended over 80 chars if
this increase code readability.
• add the required copyright header to each new file introduced [required]
•
add signed-off to all patches [required]
• to certify the "Developer's Certificate of Origin", see below
• check with your employer when not working on your own!
•
add version number for your patches if follow-up versions are requested [recommended]
• Add a "Change from Vx" description under the commit message to take track of
the history of the patch.
• It is suggested to use excellent "patman" tool to manage patches series. This
is part of U-Boot's project (tools/patman), but it can be used in other
projects, too.
• check that your patches do not break build [required]
• There is a set of configuration files in the configs/ directory. Please run a build
for all files in the directory to ensure that SWUpdate is still buildable from
configurations different as yours.
•
post patches to mailing list [required]
• use git format-patch to generate your patches.
• use git send-email if possible. This avoid corruptions due to the mailers
• add a prefix [meta-swupdate] if patches are intended to the Yocto's meta layer.
• send patches inline, do not append them
• no HTML emails!
• do not use github Pull Request. github facilities are not used for this project. The
review is done in a single place : the Mailing List. PR from github are ignored.
Patches are tracked by patchwork (see http://jk.ozlabs.org/projects/patchwork/). You can
see the status of your patches at http://patchwork.ozlabs.org/project/swupdate/list.
Developer's Certificate of Origin 1.1
When signing-off a patch for this project like this
Signed-off-by: Random J Developer <random@developer.example.org>
using your real name (no pseudonyms or anonymous contributions), you declare the
following:
By making a contribution to this project, I certify that:
a. The contribution was created in whole or in part by me and I have the right to
submit it under the open source license indicated in the file; or
b. The contribution is based upon previous work that, to the best of my knowledge,
is covered under an appropriate open source license and I have the right under
that license to submit that work with modifications, whether created in whole or
in part by me, under the same open source license (unless I am permitted to
submit under a different license), as indicated in the file; or
c. The contribution was provided directly to me by some other person who certified
(a), (b) or (c) and I have not modified it.
d. I understand and agree that this project and the contribution are public and
that a record of the contribution (including all personal information I submit
with it, including my sign-off) is maintained indefinitely and may be
redistributed consistent with this project or the open source license(s)
involved.
PROPOSALS TO IMPROVE SWUPDATE
Please take into account that most of the items here are proposals. I get some ideas
talking with customers, some ideas are my own thoughts. There is no plan when these
features will be implemented - this depends if there will be contribution to the project
in terms of patches or financial contributions to develop a feature.
Each item listed here contains a status and if the feature is already planned or I am
looking for sponsor to implement it. This should avoid double work and make more
transparent what is going on with the project.
If you have further ideas about the project, just send your proposal to the Mailing List
or post a patch for this document.
Thanks again to all companies that have supported my work up now and to everybody who has
contributed to the project, let me bring SWUpdate to the current status !
Legende
Feature's status
┌────────┬──────────────────────────────────┐
│Value │ Description │
├────────┼──────────────────────────────────┤
│Wait │ No activity is planned, just │
│ │ proposal │
├────────┼──────────────────────────────────┤
│Design │ Design / Concept is done │
├────────┼──────────────────────────────────┤
│Planned │ Feature will be implemented soon │
├────────┼──────────────────────────────────┤
│WIP │ Feature is current to be │
│ │ implemented and will be posted │
├────────┼──────────────────────────────────┤
│Running │ Implemented │
└────────┴──────────────────────────────────┘
Request for Support
┌────────┬──────────────────────────────────┐
│Value │ Description │
├────────┼──────────────────────────────────┤
│None │ No sponsoring required │
├────────┼──────────────────────────────────┤
│Sponsor │ Looking for sponsors to finance │
│ │ the feature │
├────────┼──────────────────────────────────┤
│Planned │ Feature is already sponsored │
└────────┴──────────────────────────────────┘
Priority
┌───────┬──────────────────────────────────┐
│Value │ Description │
├───────┼──────────────────────────────────┤
│Low │ Probably just an idea, feedback │
│ │ from community asked │
├───────┼──────────────────────────────────┤
│Medium │ Not critical, but nice feature │
│ │ to have │
├───────┼──────────────────────────────────┤
│High │ Critical feature, it should be │
│ │ implemented soon │
└───────┴──────────────────────────────────┘
Main goal
First goal is to reach a quite big audience, making SWUpdate suitable for a large number
of products. This will help to build a community around the project itself.
• Status : Running
• Request for Support : None
Core features
Support for OpenWRT
OpenWRT is used on many routers and has its own way for updating that is not power-cut
safe.
• Status: Wait
• Request for Support : Sponsor
• Priority : Low
Software-Software compatibility
SWUpdate has from the early stage a hardware to software compatibility check. In case
software is split in several components (like OS and application), it is desirable to have
a sort of software compatibility check. For example, SWUpdate verifies if a component
(like an application) is compatible with a runningOS and reject the update in case of
mismatch.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Support files bigger than 4GB
SWUpdate currently uses CPIO to pack updates in the 'newc' and 'crc' formats. These
formats limit single files to 4GB - 1byte in size, which could become a problem as update
size grows.
• Status: Wait
• Request for Support : Sponsor
• Priority : High
Parser
SWUpdate supports two parsers : libconfig and JSON. It would be nice if tools can be used
to convert from one format to the other one. Currently, due to some specialties in
libconfig, a manual conversion is still required.
• Status: Wait
• Request for Support : None
• Priority : Medium
Fetcher and interfaces
Selective downloading
SWUpdate starts to fetch the whole SWU and process it until an error is found. There are
some requests to have a selective downloading, that means SWUpdate will load just the
chunks are needed and not the whole SWU. An example for this use case is in case a single
SWU contains software for multiple devices, and each of them needs a subset of the whole
SWU. Like the delta handler, SWUpdate knows from sw-description which artifacts are to be
installed, and reading the stream could decide to skip unnecessary components.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Tools and utilities
Single tool to create SWU
SWUGenerator is a separate project that helps to create SWUs. It is used on not OE
projects. In OE, the SWU is created using code in meta-swupdate. This leads to a
duplication of code with higher effort to maintain.
Even if it was tried to have the same features, there are some important differences:
• SWUGenerator is able to full understand a sw-description written in libconfig language
and to rewrite it. This means it is not yet possible to write sw-description using JSON
if SWUGenerator is planned. However, it is possible to split sw-description in small
files (for example for the embedded Lua code), and SWUGenerator is able to write the
final sw-description combined all include files.
• meta-swupdate is just able to replace variables known by bitbake, but it has no semantic
knowledge. It is not possible to use @include directive, but it is possible to use JSON
as language.
The logical step will be to use a single tool (SWUGenerator), and let meta-swupdate to use
it. To do this, SWUGenerator should be enhanced to understand and write sw-description in
JSON, too.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Further enhancement to SWUGenerator
SWUGenerator is thought to support multiple subcommands, but it currently supports just
"create". It is thinkable, even if this can be done with other tools, to implement
further commands like:
• extract: take a SWU and extracts all artifacts in a directory
• sign: take a SWU and resign with a new key. This is useful when it is required to
install a new Software, but the certificate or the key on the device is older and
rejects the installation.
• verify: just verify if the SWU is correctly signed.
SWUGenerator does not yet support all features present in meta-swupdate. As replacement
for meta-swupdate and the wish to have just one tool, SWUGenerator should align itself
with meta-swupdate. It will be then possible to drop most of code from meta-swupdate's
classes, and replace with the single call to SWUGenerator.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
swupdate-progress start up
On SystemV (and compatible) systems, swupdate-progress is started from swupdate.sh via
exec. This is not the right solution and was discussed on the Mailing List.
The agreed solution is to create an own startup script for swupdate-progress, and let run
it after SWUpdate is started. This is more generic and let also to identify if
swupdate-progress should be installed or not.
• Status: Wait
• Request for Support : Not required
• Priority : Low
Lua
• API between SWUpdate and Lua is poorly documented.
• Store in SWUpdate's repo Lua libraries and common functions to be reused by projects.
• Status : Running
• Request for Support : None
• Priority : Medium
Handlers:
New Handlers
Users develop own custom handlers - I just enforce and encourage everyone to send them and
discuss how to integrate custom handler in mainline.
Some ideas for new handlers:
• FPGA updater for FPGA with Flash
• Package handler to install packages (ipk, deb) Packages can be inserted into the
SWU and the atomicity is guaranteed by SWUpdate.
• Lua handlers should be added if possible to the project to show how to solve
custom install.
• Status : Running
• Request for Support : None
• Priority : Low
Handlers installable as plugin at runtime
The project supports Lua as script language for pre- and postinstall script. It will be
easy to add a way for installing a handler at run-time written in Lua, allowing to expand
SWUpdate to the cases not covered in the design phase of a product.
Of course, this issue is related to the security features: it must be ensured that only
verified handlers can be added to the system to avoid that malware can get the control of
the target.
Current release supports verified images. That means that a handler written in Lua could
be now be part of the compound image, because a unauthenticated handler cannot run.
• Status : Running
• Request for Support : None
Support for BTRFS snapshot
BTRFS supports subvolume and delta backup for volumes - supporting subvolumes is a way to
move the delta approach to filesystems, while SWUpdate should apply the deltas generated
by BTRFS utilities.
• Status: Design
• Request for Support : Planned
• Priority : Medium
Internal Webserver
SWUpdate make usage of the project "mongoose" as internal Webserver. It fits all
requirements and allows to stream a SWU without temporary copy. However, upgrading the
Webserver code requires to adjust the interface and code. It will be nice to have further
implementation of the Webserver, and/or to open to Webserver that allows streaming.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Security / Crypto engines
• add support for asymmetric decryption
• rework support for crypto engine - let possible to load multiple libraries at the same
time. Currently, there is support for openSSL, WolfSSL and mbedTLS. However, WolfSSL
are missing together. There should be a way to select one or more libraries and
independently the algorithms that SWUpdate should support. Some hacks are currently
built to avoid conflicts (pkcs#7 and CMS are the same thing, but supported by different
libraries), and they should be solved.
• add more algorithms for decryption, as AES-CTR can be very useful to decrypt chunks in
delta updates.
• Support for TPM2 to store secrets (requires rework above).
• Status: Wait
• Request for Support : Sponsor
• Priority : High
Back-end support (suricatta mode)
Back-end: responsiveness for IPC
Suricatta is implemented as process that launches functions for the selected module. This
means that the IPC does not answer if Suricatta is doing something, specially if it is
downloading and upgrading the system. This can be enhanced adding a separate thread for
IPC and of course all required synchronization with the main modules.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Back-end: check before installing
In some cases (for example, where bandwidth is important), it is better to check if an
update must be installed instead of installing and performs checks later. If SWUpdate
provides a way to inform a checker if an update can be accepted before downloading, a
download is only done when it is really necessary.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Back-end: hawkBit Offline support
There are several discussions on hawkBit's ML about how to synchronize an offline update
(done locally or via the internal Web-server) with the hawkBit's server. Currently,
hawkBit thinks to be the only one deploying software. hawkBit DDI API should be extended,
and afterwards changes must be implemented in SWUpdate.
• Status: Wait
• Request for Support : Sponsor
• Priority : Low
Backend: hawkBit support for Delta Update
Delta Update requires two or more files:
• the SWU
• one file ".zck" for each artifact that is upgraded via delta handler.
The .zck must be uploaded somewhere and the URL is defined inside sw-description, that is
then signed. This causes a chicken-egg issue, because the buzild cannot be completed with
hawkBit until the ".zck" files are not uploaded. In fact, hawkBit assigns to each Software
Module an "id" that is unknown at the moment of the build.
It is required to implement a mechanism that let suricatta to inform the core about URLs
passed by the hawkBit server, and they can override the URL set inside sw-description.
This lets the URL for ZCK unknown during the build and it will be detected at runtime.
The authentication to the hawkBit Server does not work in case of delta. In fact,
authentication is performed by the backend connector, but the download of .zck files is
done by a different process ("downloader") that don't use the setup from suricatta.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Back-end: support for generic down-loader
SWUpdate in down-loader mode works as one-shot: it simply try to download a SWU from a
URL. For simple applications, it could be moved into suricatta to detect if a new version
is available before downloading and installing.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Back-end: further connectors
Further connectors could be implemented. The structure in SWUpdate is modular, and allows
to write new connectors, even in Lua. New connectors could be added if there are requests
in this direction.
• Status: Wait
• Request for Support : Sponsor
• Priority : Low
Test and Continuous Integration
The number of configurations and features in SWUpdate is steadily increasing and it
becomes urgent to find a way to test all incoming patch to fix regression issues. One
step in this direction is the support for Travis build - a set of configuration files is
stored with the project and should help to find fast breakages in the build. More in this
direction must be done to perform test on targets. A suitable test framework should be
found. Scope is to have a "SWUpdate factory" where patches are fast integrated and tested
on real hardware.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Binding to languages
libswupdate allows to write an application that can control SWUpdate's behavior and be
informed about a running update. There are bindings for C/C++, Lua and nodejs (just
progress).
Applications can be written in other languages, and binding to Python and Rust can be
implemented, too.
• Status: Wait
• Request for Support : Sponsor
• Priority : Medium
Documentation
Documentation is a central point in SWUpdate - maintaining it up to date is a must in this
project. Help from any user fixing wrong sentence, bad english, adding missing topics is
high appreciated.
• Status : Running
• Request for Support : None
DOCUMENTATION FOR PREVIOUS RELEASES
• 2024.05
• 2023.12
• 2023.05
• 2022.12
• 2022.05
• 2021.11
• 2021.04
• 2020.11
• 2020.04
• 2019.11
• 2019.04
• 2018.11
INDICES AND TABLES
• Index
• Module Index
• Search Page
AUTHOR
Stefano Babic
COPYRIGHT
2013-2024, Stefano Babic