Device plug-ins

Previous tutorial: Video source run time properties

 

Starting from version 2.0.0, Computer Vision Sandbox introduces support for two new plug-in types: Device plug-ins and Communication Device plug-ins. Both plug-in types are aimed for communication to external devices, such as I/O boards, robotics controllers, devices attached to serial port, etc. Device plug-ins allow talking to devices by means of setting/getting plug-in's properties - all communication details are hidden from user. Communication Device plug-ins are the opposite - they provide Read/Write APIs and allow sending raw data using whatever protocol particular device implements. Both types of plug-ins are exposed to scripting only for now and are not accessible from user interface of Computer Vision Sandbox. Let's have a look at some of these plug-ins and how to interact with them.

The very first idea of device plug-ins was mentioned a while back, when it was demonstrated how to change image processing sequence by using CapsLock, NumLock and ScrollLock keys. For this purpose a very simple LedKeys plug-in can be used, which allows querying status of those keys and then perform some custom actions.

ledKeys = Host.CreatePluginInstance( 'LedKeys' )
ledKeys:Connect( )

function Main( )
    if ledKeys:GetProperty( 'capsLock' ) then
        print( 'Caps Lock is ON' )
    end

    if ledKeys:GetProperty( 'numLock' ) then
        print( 'Num Lock is ON' )
    end

    if ledKeys:GetProperty( 'scrollLock' ) then
        print( 'Scroll Lock is ON' )
    end
end

Another basic plug-in allows checking power status of the system – if it is connected to power supply, if battery is charging and what is the current charge level.

powerInfo = Host.CreatePluginInstance( 'PowerInfo' )
powerInfo:Connect( )

function Main( )
    charge    = powerInfo:GetProperty( 'batteryCharge' )
    connected = powerInfo:GetProperty( 'powerIsConnected' )
    charging  = powerInfo:GetProperty( 'batteryIsCharging' )

    print( 'Batter Charge:   ' .. tostring( charge ) )
    print( 'Power Connected: ' .. tostring( connected ) )
    print( 'Charging:        ' .. tostring( charging ) )
end

A more interesting plug-in is the Gamepad device, which can be used in a number of applications. For example, if some camera is mounted on a pan/tilt device, that could be controlled with the help of a gamepad. Or it can be used to control some robot, video processing sequence, etc.

local math = require 'math'

gamepad = Host.CreatePluginInstance( 'Gamepad' )

-- Connected to the first game pad device 
gamepad:SetProperty( 'deviceId', 0 )
if not gamepad:Connect( ) then
    error( 'Failed connecting to game pad' )
end

-- Query name of the device, number of axes and buttons
deviceName   = gamepad:GetProperty( 'deviceName' )
axesCount    = gamepad:GetProperty( 'axesCount' )
buttonsCount = gamepad:GetProperty( 'buttonsCount' )

function Main( )
    -- Query value of all axes and buttons as arrays
    axesValues   = gamepad:GetProperty( 'axesValues' )
    buttonsState = gamepad:GetProperty( 'buttonsState' )
    
    print( 'X: ' .. tostring( math.floor( axesValues[1] * 100 ) / 100 ) )
    print( 'Y: ' .. tostring( math.floor( axesValues[2] * 100 ) / 100 ) )

    -- Query value of the X axis only
    x = gamepad:GetProperty( 'axesValues', 1 )
    
    -- Query status of the first button only
    buttonState1 = gamepad:GetProperty( 'buttonsState', 1 )
    
    if buttonState1 then
        print( "Button 1 is ON" )
    else
        print( "Button 1 is OFF" )
    end
end

To craft your own pan/tilt device, a Phidget Advanced Servo board can be used. A plug-in for this is not included into default package, but can be obtained separately from GitHub. Once added to Computer Vision Sandbox, it can be used either on its own to control servos or with the above mention gamepad device plug-in.

servos = Host.CreatePluginInstance( 'PhidgetAdvancedServo' )

-- Connected to Phidget Servo board connected to the system
if not servos:Connect( ) then
    error( 'Failed connecting to servo board' )
end

-- Check number of supported servos
motorCount = servos:GetProperty( 'motorCount' )

-- Configure velocity limit, acceleration and position range
servos:SetProperty( 'velocityLimit', { 2, 2 } )
servos:SetProperty( 'acceleration', { 20, 20 } )
servos:SetProperty( 'positionRange', { { 105, 115 }, { 135, 145 } } )

-- Engage both servos
servos:SetProperty( 'engaged', { true, true } )

-- Set target position of servo 1 and 2
servos:SetProperty( 'targetPosition', { 110, 140 } )

function Main( )
    -- Check actual position of servos and they are still moving
    actualPosition = servos:GetProperty( 'actualPosition' )
    stopped        = servos:GetProperty( 'stopped' )
    
    -- Set new target positions
    servos:SetProperty( 'targetPosition', 1, 115 )
    servos:SetProperty( 'targetPosition', 2, 135 )
end

Another supported device from the same manufacturer is Phidget Interface Kit, which allows interacting with digital inputs/outputs and with analog inputs. For example, it can be possible to control video processing routine depending on the state of inputs. Or control devices connected to digital outputs depending on what is detected in the video stream.

kit = Host.CreatePluginInstance( 'PhidgetInterfaceKit' )

-- Connected to Phidget Interface Kit board plugged into the system
if not kit:Connect( ) then
    error( 'Failed connecting to interface kit board' )
end

-- Check number of available digital/analog I/O
digitalInputCount  = kit:GetProperty( 'digitalInputCount' )
digitalOutputCount = kit:GetProperty( 'digitalOutputCount' )
analogInputCount   = kit:GetProperty( 'analogInputCount' )

-- Switch OFF all digital inputs (assuming 8 inputs available)
kit:SetProperty( 'digitalOutputs', { false, false, false, false,
                                     false, false, false, false } )

function Main( )
    -- Switch ON 1st and 2nd digital outputs
    kit:SetProperty( 'digitalOutputs', { true, true } )
    -- Also switch ON the 7th output
    kit:SetProperty( 'digitalOutputs', 7, true )
    
    -- Read digital/analog inputs
    analogInputs  = kit:GetProperty( 'analogInputs' )
    digitalInputs = kit:GetProperty( 'digitalInputs' )
    
    for i = 1, #analogInputs do
        print( 'Analog input', i, 'is', analogInputs[i] )
    end

    for i = 1, #digitalInputs do
        print( 'Digital input', i, 'is', digitalInputs[i] )
    end
end

All the above samples demonstrate how to communicate with different devices by means of setting/getting different exposed properties. However, in many cases we may have some arbitrary devices connected over serial port, for example, which implement some specific protocol allowing to control the device. To use devices like this there are two options. First is to develop a new device plug-in for Computer Vision Sandbox, which would allow talking to the device through setting/getting properties and hide details of the communication protocol. Alternatively, we can use the SerialPort communication device plug-in and send commands ourselves using a protocol supported by the device. For example, the script below demonstrates communication with an Arduino device to switch LED on/off and query push button state (it is assumed the Arduino board is running sample sketch from here).

local string = require 'string'

serialPort = Host.CreatePluginInstance( 'SerialPort' )
serialPort:SetProperty( 'portName', 'COM8' )

-- Use blocking input, read operations wait up to the configured
-- timeout value
serialPort:SetProperty( 'blockingInput', true )
-- Total Read Timeout = ioTimeoutConstant + ioTimeoutMultiplier * bytesRequested
serialPort:SetProperty( 'ioTimeoutConstant', 50 )
serialPort:SetProperty( 'ioTimeoutMultiplier', 0 )

function Main()
    
    if serialPort:Connect( ) then
        print( 'Connected' )
        
        -- Test IsConnected() method
        print( 'IsConnected: ' .. tostring( serialPort:IsConnected( ) ) )

        -- Let Arduino board reset and get ready
        sleep( 1500 )
        
        -- Switch LED on - send command as string
        sent, status = serialPort:WriteString( 'led_on\n' )
        
        print( 'status: ' .. tostring( status ) )
        print( 'sent  : ' .. tostring( sent ) )

        strRead, status = serialPort:ReadString( 10 )

        print( 'status  : ' .. tostring( status ) )
        print( 'str read: ' .. strRead )

        -- Switch LED off - sned command as table of bytes 
        sent, status = serialPort:Write( { 0x6C, 0x65, 0x64, 0x5F, 0x6F, 0x66, 0x66, 0x0A } )
        
        print( 'status: ' .. tostring( status ) )
        print( 'sent  : ' .. tostring( sent ) )

        readBuffer, status = serialPort:Read( 10 )
        
        print( 'status    : ' .. tostring( status ) )
        print( 'bytes read: ' )
        for i=1, #readBuffer do
            print( '[', i, ']=', readBuffer[i] )
        end
        
        -- Check button state
        sent, status = serialPort:WriteString( 'btn_state\n' )
        
        print( 'status: ' .. tostring( status ) )
        print( 'sent  : ' .. tostring( sent ) )

        strRead, status = serialPort:ReadString( 10 )

        print( 'status  : ' .. tostring( status ) )
        print( 'str read: ' .. strRead )
        
        if string.sub( strRead, 1, 1 ) == '1' then
            print( 'button is ON' )
        else
            print( 'button is OFF' )
        end

        -- Test that communication is not blocking
        print( 'Testing timeout' )
        strRead, status = serialPort:ReadString( 10 )

        print( 'status  : ' .. tostring( status ) )
        print( 'str read: ' .. strRead )
        
        serialPort:Disconnect( )
    end
end

For now, this is it about the new device plug-ins - the 2.0 version is just an introduction of this plug-in type. Future versions of Computer Vision Sandbox will provide more of those, bringing support for a wider range of device.

 

Next tutorial: Sandbox scripting threads