Skip to content

GitLab

Home · Changes

Page history
Update home authored by Wouter Jansen's avatar Wouter Jansen
Hide whitespace changes
Inline Side-by-side
home.md
View page @ 154b82ae
# RTIS Update Usage Documentation
This web application allows you to update any existing RTIS Software packages installed as well as install new ones. Furthermore, this application can update the firmware on a connected RTIS device. This service can run offline and is therefore able to update RTIS devices without an active internet connection.
This web application allows you to update any existing RTIS Software packages installed as well as install new ones.
Furthermore, this application can update the firmware on a connected RTIS device.
This service can run offline and is therefore able to update RTIS devices without an active internet connection.
## Features
* The web interface shows all currently tracked and installed software packages.
* New installation and updates are installed using `.rtispatch` files distributed by a Cosys-Lab member.
* Self-updating can make sure RTIS Update stays up to date.
* New installation and updates are installed using `.rtispatch` files distributed by a Cosys-Lab member.
* Self-upgrade can make sure RTIS Update stays up to date.
* Any software package can be tracked and updated.
* Git bundles can be used to install and update existing software packages with support for Git submodules.
* The STM32 firmware of an RTIS device can be found and flashed.
* Services can be automatically installed and updated. Furthermore, these can be restarted/stopped through the interface.
* The STM32 firmware of an RTIS device can be found and flashed on Jetson systems.
* Services can be automatically installed and updated. Furthermore, these can be restarted/stopped through the interface.
* Next to Git updates the system allows Powershell and Shell scripts on Windows and Linux respectively to be used to further perform installation and update steps.
## Interface usage
The web interface is available at the IP and port specified in the launch arguments. The interface is divided into two sections: the top section is about uploading/installing new software packages and the bottom section shows the currently installed software packages. To update or install a new package use the uploader to upload a `.rtispatch` file. It will then show all available packages in the patch file and allow you to install those that can be. You also have several other functions such as service management and access to the wiki and repository links.
The web interface is available at the IP and port specified in the launch arguments.
The interface is divided into two sections: the top section is about uploading/installing new software packages and the bottom section shows the currently installed software packages.
To update or install a new package use the uploader to upload a `.rtispatch` file. It will then show all available packages in the patch file and allow you to install those that can be.
You also have several other functions such as service management and access to the wiki and repository links.
## Creating a Patch
A patch is the file that a user can upload and apply using RTIS Update. RTIS Update is versatile but will depend on properly created and tested patch files to work without errors.
a patch file uses a simple `.zip` renamed as `.rtispatch` to avoid confusion. The 4 types of files that can be expected in such a compressed archive are:
- `patch_software.json`: This file contains the information about the software packages that are in the patch. It is explained in the [next section](#json-description).
- **GIT bundles**: These are files that contain a Git repository's changes between revisions. They are used to install/update software packages for GIT based applications. It is explained in the [GIT bundles section](#git-bundles).
- **script files**: These are Windows powershell scripts (`.ps1`) or Linux shell scripts (`.sh`) that can be run during installation/update to do additional steps such as dependency installations, compiling, etc. It is explained in the [script files section](#script-files).
- **Additional files**: If your scripts require additional files such as dependencies or compiled binaries, you can include them as well in the patch archive.
A patch is the file that a user can upload and apply using RTIS Update. RTIS Update is versatile but will depend on properly created and tested patch files to work without errors. a patch file uses a simple `.zip` renamed as `.rtispatch` to avoid confusion. The 4 types of files that can be expected in such a compressed archive are:
* `patch_software.json`: This file contains the information about the software packages that are in the patch. It is explained in the [next section](#json-description).
* **GIT bundles**: These are files that contain a Git repository's changes between revisions. They are used to install/update software packages for GIT based applications. It is explained in the [GIT bundles section](#git-bundles).
* **script files**: These are Windows powershell scripts (`.ps1`) or Linux shell scripts (`.sh`) that can be run during installation/update to do additional steps such as dependency installations, compiling, etc. It is explained in the [script files section](#script-files).
* **Additional files**: If your scripts require additional files such as dependencies or compiled binaries, you can include them as well in the patch archive.
### Software requirements
The software you wish to install/update should:
- be either a Python module or a standalone app that can be run with a single command.
- If a standalone app have a `version.txt` which only contains the current version in _Major.Minor.Bugfix_ format.
- If a Python module have a `__version__` variable which contains the current version in _Major.Minor.Bugfix_ format.
- Should be able to be fully installed offline and without user interaction. Anything should be scriptable through Bash on Linux or Powershell on Windows.
### JSON description
A mandatory file within the patch archive is the `patch_software.json` file. This describes the software contained within and is parsed to do the installation of the software, services and running the scripts. It has the following structure: The
A mandatory file within the patch archive is the `patch_software.json` file.
This describes the software contained within and is parsed to do the installation of the software, services and running the scripts. It has the following structure:
The
```json
{
"rtissoftware1": { [string, The name of the software package in the backend. Also used as folder name!]
......@@ -54,10 +63,12 @@ A mandatory file within the patch archive is the `patch_software.json` file. Thi
"workingdirectory": "Python", [OPTIONAL, string, the sub-folder containing the script to launch as service]
"type": "python", [string, the type of service command. if set to 'python' it automatically creates the launching of a Python script set in the 'command'. If not it just runs the 'command' as is]
"command": "RTISSoftware1.py", [string, the command to run. If 'type' is set to 'python' this should be the name of the Python script to launch]
"restart": 1, [0 or 1, if set to 1 it will be installed as an automatic-restart service. If 0, it will act as a one-shot service for example used on boot-up]
"restart": 1, [integer, if set to 1 or higher it will be installed as an automatic-restart service. On linux the value will be used as the restart timer. If 0, it will act as a one-shot service for example used on boot-up]
"enabled": 0, [0 or 1, if set to 1 it will be enabled automatically. If set to 0 it will be disabled]
"after": "multi-user.target myotherservice.service", [additional arguments can be given for Linux services (with supported parameter names)]
"after": "multi-user.target myotherservice.service", [some additional arguments can be given for Linux services (with supported parameter names)]
"environment": "LD_LIBRARY_PATH=/usr/lib/rtiscuda",
"wants": "network-online.target",
"wanted": "network-online.target",
...
},
...
......@@ -69,42 +80,36 @@ A mandatory file within the patch archive is the `patch_software.json` file. Thi
```
### GIT bundles
To install new software or update exising ones, RTIS Update uses GIT bundles. One needs to make a bundle for each repository and indicate which tag/branch/revision to use for it. Each submodule also requires its own bundle.
To install new software or update exising ones, RTIS Update uses GIT bundles. One needs to make a bundle for each repository and indicate which tag/branch/revision to use for it.
Each submodule also requires its own bundle.
To make a bundle for a repository with a specific branch/tag, you can use the following command from the repository/submodule folder:
```shell
```bash
```bash
git bundle create rtissoftware_2_0_1.bundle v2.0.1
```
### Script files
Optionally additional scripts can be included in the patch archive. These scripts can be used to do additional steps such as installing dependencies, compiling, etc. On Windows Powershell scripts are used whereas on Linux shell scripts must be placed within the patch archive. The output of the scripts is displayed to the user. Specific lines can be added to the script to provide feedback to the user.
Optionally additional scripts can be included in the patch archive. These scripts can be used to do additional steps such as installing dependencies, compiling, etc.
On Windows Powershell scripts are used whereas on Linux shell scripts must be placed within the patch archive. The output of the scripts is displayed to the user.
Specific lines can be added to the script to provide feedback to the user. Scripts will always be run from the folder where the patch files are extracted to, which is the [Patches](Patches) folder.
#### Windows Powershell scripts
To indicate completion a progress bar is displayed in the user's browser. To set this progress bar, the following line can be added to the script:
```powershell
echo "UPDATE_PROGRESS_X"
```
with *X* marking the percentage value of the progress bar. For example: `echo "UPDATE_PROGRESS_15"`
with _X_ marking the percentage value of the progress bar. For example: `echo "UPDATE_PROGRESS_15"`
To show a specific message to the user some commands are also available. The following lines can be added to the script:
```powershell
echo "UPDATE_MESSAGE_your message"
echo "UPDATE_WARNING_your warning"
echo "UPDATE_ERROR_your error"
```
This will show the message in either green, yellow or red respectively based on if you use a normal, warning or error message.
An example script using this is for example:
```powershell
echo "UPDATE_MESSAGE_Starting the script!"
echo "UPDATE_PROGRESS_15"
......@@ -118,6 +123,233 @@ echo "UPDATE_MESSAGE_Execution done!"
echo "UPDATE_PROGRESS_100"
```
#### Linux Shell scripts
Another example is how to get the current Python version:
```powershell
$pythonversion = & py -V 2>&1
$supportedpython = 0
if($pythonversion -like '*3.6*'){
echo "UPDATE_MESSAGE_Using Python 3.6"
$supportedpython = 1
}else{
echo "UPDATE_ERROR_Wrong Python version detected."
}
```
To see if a Python module is installed you can use some code like this example:
```powershell
if(py -m pip show rtisdev){
echo "UPDATE_MESSAGE_RTIS Dev module found!"
}else{
echo "UPDATE_ERROR_RTIS Dev module not found, install it!"
}
```
#### Linux Shell scripts
With linux scripts, dont forget to add `#!/usr/bin/env bash` at the top of the script file!
To indicate completion a progress bar is displayed in the user's browser. To set this progress bar, the following line can be added to the script:
```bash
echo "UPDATE_PROGRESS_X"
```
with _X_ marking the percentage value of the progress bar. For example: `echo "UPDATE_PROGRESS_15"`
To show a specific message to the user some commands are also available. The following lines can be added to the script:
```bash
echo "UPDATE_MESSAGE_your message"
echo "UPDATE_WARNING_your warning"
echo "UPDATE_ERROR_your error"
```
This will show the message in either green, yellow or red respectively based on if you use a normal, warning or error message.
An example script using this is for example:
```bash
#!/usr/bin/env bash
echo "UPDATE_MESSAGE_Starting the script!"
echo "UPDATE_PROGRESS_15"
echo "UPDATE_MESSAGE_This is my message"
echo "UPDATE_WARNING_This is my warning"
echo "UPDATE_ERROR_This is my error"
echo "UPDATE_PROGRESS_50"
echo "UPDATE_MESSAGE_Executing the ipconfig command..."
ifconfig
echo "UPDATE_MESSAGE_Execution done!"
echo "UPDATE_PROGRESS_100"
```
Another example is how to get the current Python version:
```bash
PY_VER_MAJOR="0"
PY_VER_MINOR="0"
if which python3 > /dev/null 2>&1;
then
PY_VER_MAJOR="$(python3 -c 'import sys; print(sys.version_info[0])')"
PY_VER_MINOR="$(python3 -c 'import sys; print(sys.version_info[1])')"
fi
if [ "${PY_VER_MAJOR}" = "3" ];
then
if [ "${PY_VER_MINOR}" == "6" ];
then
echo "UPDATE_MESSAGE_Python v3.6 detected."
else
echo "UPDATE_ERROR_Python installation found but is not version 3.6. Please install python 3.6."
exit 1
fi
else
echo "UPDATE_ERROR_Python installation not found with command 'python3'. Please install python 3.6."
exit 1
fi
```
To see if a Python module is installed you can use some code like this example:
```bash
if python3 -c "import rtisdev" > /dev/null 2>&1 -eq 0;
then
echo "UPDATE_MESSAGE_RTIS Dev module found!"
else
echo "UPDATE_ERROR_RTIS Dev module not found, install it!"
fi
```
#### Offline pip installations
To get offline pip installations working, the following steps are required:
- Create a `requirements.txt` file that holds all the packages to be installed. For example:
```text
PyQt5
pyqtgraph
matplotlib
```
- Use the `pip download` command to download all these packages to a folder for a specific python version. For example:
```bash
python3 -m pip download -r requirements.txt --only-binary=:all: --python-version=3.6
```
You can download these for multiple Python versions as well.
- Once you have all the packages downloaded, place them in your patch archive together with the `requirements.txt` file . Then add a `pip install` command within your script:
```bash
python3 -m pip install -r requirements.txt --no-index --find-links .
```
This will automatically search for the packages needed for the python version that is being used.
#### Offline aptitude installations on Linux
To get offline aptitude(`apt-get install...`) installations working, the following steps are required:
- The offline system requires to have `apt-offline` installed.
- Make sure you are running on a **VERY** similar system with online access. Run the `apt-offline set `command as for example:
```bash
apt-offline set --install-packages YOUR_PROGRAM --update apt-offline.sig
```
- Use the `apt-offline get` command to generate an archive holding all the packages and its dependencies. For example:
```bash
apt-offline get --bundle YOUR_PROGRAM.zip apt-offline.sig
```
- Once you have the archive, place it in your patch archive. Then add a `apt-offline install` command within your script:
```bash
apt-offline install YOUR_PROGRAM.zip --skip-changelog
```
This will automatically install/update your program and its dependencies.
#### Offline aptitude installations on Linux
To get offline aptitude(`apt-get install...`) installations working, the following steps are required:
- The offline system requires to have `apt-offline` installed.
- Make sure you are running on a **VERY** similar system with online access. Run the `apt-offline set `command as for example:
```bash
apt-offline set --install-packages YOUR_PROGRAM --update apt-offline.sig
```
- Use the `apt-offline get` command to generate an archive holding all the packages and its dependencies. For example:
```bash
apt-offline get --bundle YOUR_PROGRAM.zip apt-offline.sig
```
- Once you have the archive, place it in your patch archive. Then add a `apt-offline install` command within your script:
```bash
apt-offline install YOUR_PROGRAM.zip --skip-changelog
```
This will automatically install/update your program and its dependencies.
### Firmware installation
Currently only supported on Jetson devices, the RTIS device firmware can also be updated using _stm32loader_ and _Jetson.GPIO_. This requires some particular attributes in the `patch_software.json` file.
The update name and attribute `type` should always be set to `firmware`.
Furthermore, the binary firmware file should also be present in the patch archive as well as defined in the JSON description under the `filename` attribute.
Finally, you can optionally add the `port` attribute to override the automatic detection of the right port as well as customize the stm32 switch-pin with the `stm32pin` attribute (default=7).
An example JSON description:
```json
{
"firmware": {
"name": "RTIS Device Firmware",
"version": "1.5.0",
"type": "firmware",
"filename": "v1.5.0.bin",
"port": "/dev/ttyTHS2",
"stm32pin": 9
}
}
```
### Self-upgrade
RTIS update can also update itself. This works rather similarly to other patches but requires some particular things to work correctly.
The update name should always be `rtisupdate` as this is in the backend hardcoded to be the name of the RTIS Update self-upgrade package.
Furthermore, the version is similarly checked to be higher based on what is listed in the `patch_software.json` file.
A example JSON description:
```json
{
"rtisupdate": {
"name": "RTIS Update",
"version": "5.0.0",
"repo": "https://cosysgit.uantwerpen.be/rtis-software/rtisupdate",
"wiki": "/help",
"type": "standalone",
"patchfile": "rtisupdate_5_0_0.bundle",
"scriptfile": "rtisupdate_5_0_0",
"branch": "v5.0.0"
}
}
```
It should also include a script to install/restart the service that runs RTIS Update in the background.
This script is always launched with arguments that will contain the same launch arguments as the currently running script.
For example on Windows and Linux respectively you can use:
```powershell
echo "RTIS Update offline installation script for Windows Systems - v5.0.0"
if(get-service rtisupdate){
echo " RTIS Update Service..."
Restart-Service -Name rtisupdate -Force
echo "RTIS Update Service restarted!"
}else
{
echo "Installing RTIS Update Service..."
$pythonpath = (python -c "import os, sys; print(os.path.dirname(sys.executable))") + "\python.exe"
$updatepath = (pwd).path
$updatescript = $updatepath + "\RTISUpdateWeb.py " + $args[0]
.\nssm.exe install rtisupdate $pythonpath $updatescript ; .\nssm.exe set rtisupdate AppDirectory $updatepath
echo "RTIS Update Service installed!"
echo "Starting RTIS Update Service..."
Restart-Service -Name rtisupdate -Force
echo "RTIS Update Service started!"
}
echo "Installation completed!"
```
```bash
#!/usr/bin/env bash
TODO
\ No newline at end of file
echo "RTIS Update offline installation script for Linux Systems - v5.0.0"
if systemctl cat rtisupdate;
then
echo " RTIS Update Service..."
service rtisupdate restart
echo "RTIS Update Service restarted!"
else
argumentsgiven="${@:2}"
echo $'[Unit]\nDescription=RTIS Update Service\nAfter=multi-user.target\n\n[Service]' >> /etc/systemd/system/rtisupdate.service
echo "ExecStart=/usr/bin/python3 ${PWD%/*}/RTISUpdateWeb.py ${argumentsgiven}" >> /etc/systemd/system/rtisupdate.service
echo "WorkingDirectory=${PWD%/*}" >> /etc/systemd/system/rtisupdate.service
echo "User=${1}" >> /etc/systemd/system/rtisupdate.service
echo $'Restart=always\nRestartSec=3\n\n[Install]\nWantedBy=multi-user.target' >> /etc/systemd/system/rtisupdate.service
chmod 644 /etc/systemd/system/rtisupdate.service
systemctl daemon-reload
echo "RTIS Update Service installed!"
echo "Starting RTIS Update Service..."
service rtisupdate restart
echo "RTIS Update Service started!"
fi
echo "Installation completed!"
```
\ No newline at end of file