Detection of circles and rectangles
Previous tutorial: Blobs processing
The 1.2.7 version of Computer Vision Sandbox comes with an updated blobs processing module and provides couple of new interesting plug-ins. Those can be used for detection/filtering of some simple shapes like circles, rectangles and squares. Let's have a look at how that can be done.
Being from blobs processing family, both Filter Circle Blobs and Filter Quadrilateral Blobs plug-ins require a pre-processed image as input. It must be a grayscale image, with black representing background and white (or shades of gray) representing objects. This type of segmentation can be achieved using different image processing filters depending on the source images coming from video source/camera. In the set-up for this tutorial we used a dark background - darker than any of the objects to detect. And so simple thresholding worked well as the initial segmentation step.
We'll start with circles filtering first. Creating a simple video processing graph with the Filter Circle Blobs plug-in like shown above, should produce an image, which contains objects with circle shape only. Everything else is removed.
Using the Filter Circle Blobs plug-in with default settings should work for most of the cases. However, few settings may require tuning to achieve the desired result. The first thing to do is configuring minimum and maximum circles' radius to keep. Circles with radius values outside of the specified range are removed as well. Then using relative distortion and minimum distortion properties it is possible to control how perfect should objects fit a circle shape. Increasing these values will result in accepting objects with a not so circular shape. While keeping them small will accept only nice shaped circles.
Replacing the Filter Circle Blobs plug-in with Filter Quadrilateral Blobs in the above video processing graph will result in keeping quadrilateral shapes only (shapes with 4 corners) and removing everything else. To narrow down the variety of allowed shapes, the plug-in allows choosing from few common quadrilateral shapes like rectangles and squares. In addition it provides properties to filter shapes by size and control acceptable deviation from the ideal shape.
Of course filtering an image by removing unwanted objects is useful in many applications. However, very often it is required to get information about the required objects, like radius and center of circles, or coordinates of rectangles' corners, etc. Both Filter Circle Blobs plug-in with Filter Quadrilateral Blobs plug-ins can provide this information. All is needed is to process an image first and then inspect some of the plug-in's read only properties, which provide information about what was found in the image. This can be done from scripting though. Let's start with circles first.
local math = require "math"
local string = require "string"
-- Create instances of plug-ins to use
grayscalePlugin = Host.CreatePluginInstance( 'Grayscale' )
thresholdPlugin = Host.CreatePluginInstance( 'Threshold' )
circlesFilterPlugin = Host.CreatePluginInstance( 'FilterCircleBlobs' )
drawingPlugin = Host.CreatePluginInstance( 'ImageDrawing' )
-- Set threshold to separate background and objects
thresholdPlugin:SetProperty( 'threshold', 64 )
-- Don't do image filtering, only collect information about circles
circlesFilterPlugin:SetProperty( 'filterImage', false )
-- Set minimum radius of circles to collect
circlesFilterPlugin:SetProperty( 'minRadius', 5 )
-- Color used for drawing
drawingColor = '00FF00'
function Main( )
image = Host.GetImage( )
-- Preprocess image by grayscaling and thresholding it
grayImage = grayscalePlugin:ProcessImage( image )
thresholdPlugin:ProcessImageInPlace( grayImage )
-- Apply circles filter
circlesFilterPlugin:ProcessImageInPlace( grayImage )
circlesFound = circlesFilterPlugin:GetProperty( 'circlesFound' )
circlesCenters = circlesFilterPlugin:GetProperty( 'circlesCenters' )
circlesRadiuses = circlesFilterPlugin:GetProperty( 'circlesRadiuses' )
meanDeviations = circlesFilterPlugin:GetProperty( 'meanDeviations' )
-- Tell how many circles are detected
drawingPlugin:CallFunction( 'DrawText', image, 'Circles: ' .. tostring( circlesFound ),
{ 5, 5 }, drawingColor, '00000000' )
-- TODO
grayImage:Release( )
end
The above script does the first step - it does pre-processing of an image to separate objects from background and then applies circles filtering plug-in. The source image coming from camera is then updated to tell how many circles are detected. Now, let’s add some more code and highlight each detected circle using the information provided by the plug-in (replacing the 'TODO' comment).
-- Highlight each detected circle
for i = 1, circlesFound do
center = { math.floor( circlesCenters[i][1] ), math.floor( circlesCenters[i][2] ) }
radius = math.floor( circlesRadiuses[i] )
dist = math.floor( math.sqrt( radius * radius / 2 ) )
lineStart = { center[1] + radius, center[2] - radius }
lineEnd = { center[1] + dist, center[2] - dist }
drawingPlugin:CallFunction( 'FillRing', image, center, radius + 2, radius, drawingColor )
drawingPlugin:CallFunction( 'DrawLine', image, lineStart, lineEnd, drawingColor )
drawingPlugin:CallFunction( 'DrawLine', image, lineStart,
{ lineStart[1] + 20, lineStart[2] }, drawingColor )
-- Tell radius and mean deviation
drawingPlugin:CallFunction( 'DrawText', image, tostring( radius ),
{ lineStart[1] + 2, lineStart[2] - 12 },
drawingColor, '00000000' )
drawingPlugin:CallFunction( 'DrawText', image, string.format( '%.2f', meanDeviations[i] ),
{ lineStart[1] + 2, lineStart[2] + 3 },
drawingColor, '00000000' )
end
The above code completes the circles' detection script - each detected circle is highlighted, its radius is displayed, as well as mean deviation from the ideal shape (smaller value means the shape looks closer to circle). The end result should look something like the image below.
Using Filter Quadrilateral Blobs plug-in similar result can be achieved for rectangles detection. The script below highlights all detected rectangles, and reports length of their two sides (longer one goes first).
local table = require "table"
local math = require "math"
-- Create instances of plug-ins to use
grayscalePlugin = Host.CreatePluginInstance( 'Grayscale' )
thresholdPlugin = Host.CreatePluginInstance( 'Threshold' )
quadsFilterPlugin = Host.CreatePluginInstance( 'FilterQuadrilateralBlobs' )
drawingPlugin = Host.CreatePluginInstance( 'ImageDrawing' )
-- Set threshold to separate background and objects
thresholdPlugin:SetProperty( 'threshold', 48 )
-- Don't do filtering image, only collect information about rectangles
quadsFilterPlugin:SetProperty( 'filterImage', false )
-- Keep rectangles only (which includes squares)
quadsFilterPlugin:SetProperty( 'quadrilateralsToKeep', 1 )
-- Minimum/Maximum allowed length of rectangle's sides
quadsFilterPlugin:SetProperty( 'minSize', 10 )
quadsFilterPlugin:SetProperty( 'maxSize', 200 )
-- Relative distortion in %
quadsFilterPlugin:SetProperty( 'relDistortion', 7 )
-- Minimum allowed mean distortion
quadsFilterPlugin:SetProperty( 'minDistortion', 4 )
-- Colors used to highlight rectangle's corners
cornerColors = { '00FF00', '008800', 'AAFFAA', '00FFFF' }
-- Color used for drawing
drawingColor = '00FF00'
function Main( )
image = Host.GetImage( )
-- Preprocess image by grayscaling and thresholding it
grayImage = grayscalePlugin:ProcessImage( image )
thresholdPlugin:ProcessImageInPlace( grayImage )
-- Apply quadrilateral filter
quadsFilterPlugin:ProcessImageInPlace( grayImage )
quadsFound = quadsFilterPlugin:GetProperty( 'quadrilateralsFound' )
quads = quadsFilterPlugin:GetProperty( 'quadrilaterals' )
-- Tell how many rectangles are detected
drawingPlugin:CallFunction( 'DrawText', image, 'Rectangles: ' .. tostring( quadsFound ),
{ 5, 5 }, drawingColor, '00000000' )
-- Highlight each detected rectangle
for i = 1, quadsFound do
quad = quads[i]
maxY = 0
minX = image:Width( )
drawingPlugin:CallFunction( 'DrawPolygon', image, quad, drawingColor )
-- Four corners of each rectangle
for j = 1, 4 do
drawingPlugin:CallFunction( 'FillRectangle', image,
{ quad[j][1] - 3, quad[j][2] - 3 },
{ quad[j][1] + 3, quad[j][2] + 3 },
cornerColors[j] )
if ( quad[j][2] > maxY ) then
maxY = quad[j][2]
end
if ( quad[j][1] < minX ) then
minX = quad[j][1]
end
end
-- Calculate length of rectangle's both sides
dx1 = quad[2][1] - quad[1][1]
dy1 = quad[2][2] - quad[1][2]
dx2 = quad[3][1] - quad[2][1]
dy2 = quad[3][2] - quad[2][2]
len1 = math.floor( math.sqrt( dx1 * dx1 + dy1 * dy1 ) )
len2 = math.floor( math.sqrt( dx2 * dx2 + dy2 * dy2 ) )
if ( len2 > len1 ) then
temp = len1
len1 = len2
len2 = temp
end
strLen = tostring( len1 ) .. ' / ' .. tostring( len2 )
drawingPlugin:CallFunction( 'DrawText', image, strLen, { minX, maxY + 4 },
drawingColor, '00000000' )
end
grayImage:Release( )
end
And here is the result of its work. All of the rectangular objects seem to be highlighted as needed.
Both Filter Circle Blobs and Filter Quadrilateral Blobs plug-ins can be very useful in many different applications. Each of them become a very good extension of the already existing blob processing capabilities provided with the Computer Vision Sandbox application. And there is more to be developed.
Note: here are complete scripts for detection of circles and detection of rectangles.
Next tutorial: Creating time lapse video