Skip to content

GitLab

RTIS Network

RTIS Network

Project ID: 133
Star 1

This project focuses on both a network-based topology of RTIS nodes and several usage types.

This project enables using RTIS sensors in a star network architecture. Each sensor is a client where a central server allows through an API any application to request sensor data.
It includes distributed signal processing workflows on both client and server with CUDA support.

Usage

A. Configuration

All configuration is done on the RTIS Server side. There are two configuration file types:

The serversettings.json contains all the main settings for the Network. It must always have that name and be located in the Config folder. Here is a more detailed explanation of each parameter:

{
    "serverIp" : "127.0.0.1",                       [string, The IP of the RTIS Server on which to host it]
    "applicationIp": "127.0.0.1",                   [string, The IP of the application on which to connect to it]
    "log" : 2,                                      [integer, Disable or configure log files. 0=off, 1=only warnings and errors, 2=includes info, 3=includes debug]
    "debug": 0,                                     [integer, Disable or configure debug mode where RTIS Sync is simulated.. 0=off, 1=on]
    "ntpEnable" : 1,                                [integer, Disable or enable the NTP Server. Best turned off if internet is available on the network. 0=off, 1=on]
    "ntpPort": 65450,                               [integer, Port on which to host the NTP Server.]
    "syncPort" : "/dev/ttyUSB0",                    [string, Port on which to connect to the RTIS Sync device.]
    "syncFrequency": 12,                            [integer, The default frequency RTIS Sync should use. Can by overwritten by RTIS Control at any time.]
    "limitSyncFrequency": 1,                        [integer, Toggle the optional limiter of measurement frequency based on measurement length of  RTIS Clients. 0=off, 1=on]     
    "syncActivePorts": [1, 1, 1, 1, 1, 1],          [list of integers, The default active connectors RTIS Sync should use left to right. Can by overwritten by RTIS Control at any time.]
    "serverProcessingName" : "server",              [string, Name of the processingSettings option for the RTIS Server DSP pipeline.]
    "clientSettings" : {                            [Seperate list of RTIS Client settings. One is always necessary called default. Then you can create custom ones for specific RTIS devices by using their unique identifier.]
        "default" : {                               
            "processingName" : "clientDefault",     [string, Name of the processingSettings option for this RTIS Client´s DSP pipeline.]
            "x": 0,                                 [float, The forward facing axis position value in meters of this RTIS Client.]    
            "y": 0,                                 [float, The left facing axis position value in meters of this RTIS Client.]    
            "z": 0,                                 [float, The upwards facing axis position value in meters of this RTIS Client.]    
            "pitch": 0,                             [float, The rotation value around the Y-axis in degrees of this RTIS Client.]    
            "yaw": 0,                               [float, The rotation value around the Z-axis in degrees of this RTIS Client.]    
            "roll": 0                               [float, The rotation value around the X-axis in degreesof this RTIS Client.]  
        },
        "eRTIS001" : {                              [string, Use the RTIS Device´s unique identifier to define custom settings.]
            ...                                     [the same values as above for the default configuration are required here too.]
            "required" : 1                          [integer, Set to 1 if this Client is required to be in the 'Ready' state for at least 30 seconds before allowing further operation.]
        },
        ...
    },
    "processingSettings" : {                        [Seperate list of all processingSettings to which the previous settings are linked to.]
        "server": {                                 [string, unique identifier to identify the processingSettings with which is used by serverProcessingName and clientSettings.]
            "workers": 3,                           [integer, The amount of RTIS CUDA GPU workers that should be used for the DSP pipeline.]
            "pdm" : 0,                              [integer, Disable or enable the binary data unpacking and PDM filtering part of the DSP pipeline. 0=off, 1=on]
            "preFilter" : 0,                        [integer, Disable or enable the optional pre processing filter of the DSP pipeline. This expects a 'preFilter.csv' file in the config folder. 0=off, 1=on]
            "matchedFilter" : 0,                    [integer, Disable or enable the optional matched filter part of the DSP pipeline. 0=off, 1=on]
            "beamforming" : 1,                      [integer, Disable or enable beamforming part of the DSP pipeline. 0=off, 1=on]
            "postFilter" : 0,                       [integer, Disable or enable the optional post processing filter of the DSP pipeline.This expects a 'postFilter.csv' file in the config folder. 0=off, 1=on]
            "enveloppe" : 1,                        [integer, Disable or enable enveloppe creation part of the DSP pipeline. 0=off, 1=on]
            "clean" : 1,                            [integer, Disable or enable cleaning of the Energyscape part of the DSP pipeline. 0=off, 1=on]
            "DSPFolderName" : "2D_90_debug"         [string, unique identifier to identify the DSPFiles foldername to use.]

        },
        "clientDefault" : {
            ...
        },
        ...
    }
}

The DSP Config folders contain additional settings to define the RTIS Client device settings and to generate the DSP pipeline files. The most important file in each folder (which each should have a unique name) is the config.json. This defines the RTIS Client device settings as well as other settings that will be used by the automated generation of the .csv files used by the DSP pipeline.

These files can made more easily with the web interface.

Here is a more detailed explanation of the config.json file:

{
  "microphoneLayout" : "eRTIS_v3D1",         [string, The name of the microphone array layout. The available ones can be found in the Config/DSPFiles/MicrophoneLayouts folder.]
  "microphoneSamples" : 163840,                     [integer, The amount of microphone sames the RTIS Client device should use. Must be dividable by 32768.]
  "microphoneSampleFrequency" : 4500000,            [integer, The microphone sample frequency used by the DSP pipeline.]
  "dacSampleFrequency" : 450000,                    [integer, The DAC (emitter) sample frequency used by the RTIS Client device. Must by larger then 160KHz and smaller then 2MHz.]
  "pdmSampleFrequency": 4500000,                    [integer, The ADC sample frequency used by the RTIS Client device. Must be 4.5MHz for ultrasonic measurements or 1.125MHz for audible measurements.]
  "minRange" : 0.5,                                 [float, The minimum distance of the Energyscape used by the DSP pipeline.]
  "maxRange" : 5,                                   [float, The maximum distance of the Energyscape used by the DSP pipeline.]
  "pdmSubsampleFactor" : 10,                        [integer, The subsampling factor used by the PDM filtering used by the RTIS DSP pipeline.]
  "energyscapeSubsampleFactor" : 10,                [integer, The subsampling factor used on the Energyscape used by the RTIS DSP pipeline.]
  "beamformingDrop" : 512,                          [integer, The dropped samples used for making the Enveloppe creation efficient. Used by the RTIS DSP pipeline.]
  "callDuration" : 2.5,                             [float, The call duration used for generating the DAC pulse used by the RTIS Client device.]
  "callMinimumFrequency" : 25000,                   [integer, The minimum frequency of the call used for generating the DAC pulse used by the RTIS Client device.]
  "callMaximumFrequency" : 50000,                   [integer, The maximum frequency of the call used for generating the DAC pulse used by the RTIS Client device.]
  "initialDirections": 91,                          [integer, The amount of directions that are spatially equally distanced for creating the Energyscape. Defines your spatial resolution but scales with performance of the DSP pipeline.]
  "2D": 1                                           [integer, Toggle on or off if the Energyscape should be 2D or 3D. 0==3D, 1=2D]
}

As long as you create a new unique folder name and add this config.json file the server will generate all other necessary files.

B. Control

Currently the RTIS Network can be controlled through a single CLI interface. This is defined by the RTISControl.py script. As described in the Installation steps, this could be binded to a CLI command such as rtiscontrol, which we will use throughout this description of the RTIS Control script. The CLI control uses different types of launch arguments. Note the difference in using -- in front of the arguments and where it isn't used. Here is a list of all available: Standard: These can be used in combination with all other commands.

  • --log: Disable or configure log files. 0=off , 1=only warnings and errors, 2(default)=includes info, 3=includes debug
  • --status: Enable showing RTIS network status with --status on (default=off) Configuration: these can not be mixed together.
  • reload: Reload the serversettings.json and DSPFiles and make Server and Client use (new) settings.
  • idle: Disable GPU workers and measurement listener.
  • prepare: Start GPU workers and measurement listener. This will check if all required Clients are stable(online for 30 seconds) and online.
  • start: Start Sonar measurements with SYNC. This willd o a check to make sure the Server and Client are ready first.
    • --measurements: Choose amount of measurements (infinite by default).
    • --frequency: Choose custom frequency that overrides the default in the serversettings.json file.
    • --active: Choose which connections on RTIS Sync device should be activated that overrides the default in the serversettings.json file. Format: "X,X,X,X,X,X" where X is 1(active) or 0(inactive).
  • stop: Stop Sonar measurements with SYNC.
  • passive: Set the connected RTIS devices to use passive behaviour.
  • active: Set the connected RTIS devices to use active behaviour.
  • counter: By default resets all RTIS device measurement counters to 0 unless --index argument is given.
    • --index: Integer that sets custom device measurement counter.

so for example you can use in your terminal:

rtiscontrol reload
rtiscontrol active
rtiscontrol counter --index 420
rtiscontrol prepare
rtiscontrol start --measurements 250 --frequency 5 --active 1,0,0,1,0,0
rtiscontrol --status on --log 3

If no arguments are given or --status on is used, it will print out the status of the RTIS Network in a continuous print. This will show information on the Client and Server such as their configuration, state and buffer counters. The state can be:

  • Booting: The device has just booted up and is still trying to connect to the RTIS Server and get started.
  • Started The device has successfully booted up and is connected to the RTIS Network.
  • Configuring: The device has received its configuration files and settings and is configuring itself.
  • Ready: The device has been successfully configured and is now in the idle state.
  • Preparing: The device is starting it's GPU workers and preparing to deal with measurements.
  • Online: The device is fully prepared and ready for starting measurements.

When in this status mode of the CLI interface, keyboard shortcuts are also available and listed on the screen to quickly perform certain commands.

B. Web Interface

Currently a simple web interface is available on the RTIS Network LAN. It can be reached from any webbrowser on serverIp:5000. It offers the same functionality as RTIS Control CLI currently but is more user friendly.

Here is a list of all available functions on the Home page:

  • Idle mode: Disable GPU workers and measurement listener.
  • Prepare workers: Start GPU workers and measurement listener.
  • Reload settings: Reload the serversettings.json and DSPFiles and make Server and Client use (new) settings.
  • Set behaviour: Set the connected RTIS devices behaviour using the checkbox toggle.
  • Set counter index: By default resets all RTIS device measurement counters to 0 unless index argument is set differently.
    • Index: Integer that sets custom device measurement counter.
  • Stop: Stop Sonar measurements with SYNC.
  • Start: Start Sonar measurements with SYNC.
    • Measurements: Choose amount of measurements (infinite by default).
    • Frequency: Choose custom frequency that overrides the default in the serversettings.json file.
    • -Customize active connectors: Choose which connections on RTIS Sync device should be activated that overrides the default in the serversettings.json file. Where marking is active.

It will also show the status of the Server and all connected Clients such as their configuration, state and buffer counters. The state can be:

  • Booting: The device has just booted up and is still trying to connect to the RTIS Server and get started.
  • Started The device has successfully booted up and is connected to the RTIS Network.
  • Configuring: The device has received its configuration files and settings and is configuring itself.
  • Ready: The device has been successfully configured and is now in the idle state.
  • Preparing: The device is starting it's GPU workers and preparing to deal with measurements.
  • Online: The device is fully prepared and ready for starting measurements.

Furthermore, on the Configuration page one can safely configure the serversettings.json file from this web interface. Lastly, on the DSP Settings page the DSP Config folders can be managed and generated easily as well.

C. HTTP API

Similarly as to the web API, all control functionality is also available as HTTP commands. They will return status 200 on success and 400 (with error message as data) on failure.

The following functions can therefore be used:

  • Status - http://serverIp:5000/api/status: Get a JSON structure with the full status of the active network.
  • Server Configuration - http://serverIp:5000/api/serverconfig: Get a JSON structure with the full server configuration.
  • DSP Configurations - http://serverIp:5000/api/dspconfigs: Get a JSON structure with all the DSP configurations saved.
  • Idle mode - http://serverIp:5000/api/idle: Disable GPU workers and measurement listener.
  • Prepare workers - http://serverIp:5000/api/prepare: Start GPU workers and measurement listener.
  • Reload settings - http://serverIp:5000/api/reload: Reload the serversettings.json and DSPFiles and make Server and Client use (new) settings.
  • Set behaviour - http://serverIp:5000/api/behaviour: Set the connected RTIS devices behaviour. With no additional argument for mode, it will default to passive.
    • mode - http://serverIp:5000/api/behaviour?mode=integer : Additional Integer argument argument to set the behaviour mode. 1 is active, 0 is passive.
  • Set counter index - http://serverIp:5000/api/counter: By default resets all RTIS device measurement counters to 0 unless index argument is set differently.
    • Index - http://serverIp:5000/api/counter?index=integer : Integer that sets custom device measurement counter.
  • Stop - http://serverIp:5000/api/stop: Stop Sonar measurements with SYNC.
  • Start - http://serverIp:5000/api/start: Start Sonar measurements with SYNC with default parameters.
    • Frequency - http://serverIp:5000/api/start?frequency=integer: Choose custom frequency that overrides the default in the serversettings.json file.
    • Measurements - http://serverIp:5000/api/start?measurements=integer: Additional argument to choose amount of measurements. If not set it will default to continuous.
    • -Customize active connectors - http://serverIp:5000/api/start?active=string: Choose which connections on RTIS Sync device should be activated that overrides the default in the serversettings.json file. Should be a string of the format X,X,X,X,X,X where each X is either 1(enabled) 0(disabled).
    • Like normal HTTP, you can combine multiple arguments. For example: http://serverIp:5000/api/start?frequency=10&measurements=250&active=1,1,1,0,0,0

D. Using and creating a RTIS Network application

The idea of RTIS Network is to provide (processed) RTIS measurements of one or multiple RTIS sensors. In order to use these measurements one has to connect an application. Several 'default' applications are already available:

  • RTIS Storage: Save all measurements made by RTIS Network in .mat files.
  • RTIS ROS: Publish the measurements to the ROS middleware with default and custom message types.
  • RTIS Unity Visualizer: Visualize the processed energyscapes of the measurements in real-time.

Making a new application is rather straight forward. Several functions are available to be used to gain information on the connected RTIS Clients and prepare your application for their data. For this, one should use the Python library RTIS Common Library. The available commands are available in the wiki. There you can also find an example on setting up a data connection to the RTIS Server and what the data format is like.

Troubleshooting

Some issues we encountered while working on this that might come in handy for others in the future:

  • Using PyCharm IDE on Ubuntu not being able to load RTIS CUDA library by not finding library path: See solution here. Basically need to add LD_LIBRARY_PATH with value /usr/lib/rtiscuda (or your custom location) manually to your run configurations.
  • The hotkeys for RTIS Control don't work when not using elevated permissions on ubuntu(use sudo to fix). Also doesn't work when using tools like Synergy. Use natively connected keyboard!
  • If using PyCharm you can use the ideolog plugin to apply automatic syntax highlighting to the RTIS Network logfiles. See here for the settings:
    • message pattern: ^\[([0-9]+)\-([0-9]+)\-([0-9]+)\s([0-9]+):([0-9]+):([0-9]+)\,([0-9]+)\]\s\-\s([a-zA-Z\s]+)\s\-\s([a-zA-Z\']+)\s\-\s(.*)$
    • message start pattern: ^\[
    • time format: yyyy-MM-dd HH:mm:ss,SSS
    • color patterns: ^\s*W(ARNING)?\s*$ , _^\s*I(NFO)?\s*$_, _^\s*E(RROR)?\s*$_, ...