Processing sub images

Previous tutorial: Switching effects at run time

 

There are cases when it is required to apply certain image processing routine (plug-in) not to the entire image, but only to a some region of interest (ROI). This tutorial will show how to do that for rectangular ROI. Note: the APIs discussed here were made available in scripting API revision 2.

To get a rectangular ROI of an image it is required to use the GetSubImage() method of an image object. This will return a new image object, which is a sub image of the source image. For example, the code below gets a sub image at the center of the source one:

-- Get image to process
image = Host.GetImage( )

halfWidth  = image:Width( )  / 2;
halfHeight = image:Height( ) / 2;

-- Get sub image aligned at the center of the source image
subImage = image:GetSubImage( halfWidth / 2, halfHeight / 2, halfWidth, halfHeight )

Using this technique it is possible to apply different image processing routines to different parts of an image. One of the things to keep in mind however is that sub image's coordinates must stay within the source image. If sub-image's coordinates go beyond source image's boundaries, the method will trigger an error. The code below applies same image processing plug-in to different image's parts, but with with different settings.

setHuePlugin = Host.CreatePluginInstance( 'SetHue' )
...

-- Get image to process
image = Host.GetImage( )

width  = image:Width( )
height = image:Height( )

halfWidth  = width  / 2;
halfHeight = height / 2;

-- top-left part goes red
setHuePlugin:SetProperty( 'hue', 0 )
setHuePlugin:ProcessImageInPlace(
    image:GetSubImage( 0, 0, halfWidth, halfHeight ) )

-- top-right part goes green
setHuePlugin:SetProperty( 'hue', 90 )
setHuePlugin:ProcessImageInPlace(
    image:GetSubImage( width - halfWidth, 0, halfWidth, halfHeight ) )

-- bottom-right part goes cyan
setHuePlugin:SetProperty( 'hue', 180 )
setHuePlugin:ProcessImageInPlace(
    image:GetSubImage( width - halfWidth, height - halfHeight, halfWidth, halfHeight ) )

-- bottom-left part goes purple
setHuePlugin:SetProperty( 'hue', 270 )
setHuePlugin:ProcessImageInPlace(
    image:GetSubImage( 0, height - halfHeight, halfWidth, halfHeight ) )

The main thing to mention about GetSubImage() is that it does not provide a deep copy of the source image's ROI. The returned image object is just a thin wrapper around the memory area occupied by the main image (the code behind it is just setting image pointer at the required location and setting stride appropriately). This makes this method very lightweight and calling it does not invoke any overhead - no memory allocation is done for a new image and no copying is done (well, just a small image structure is allocated to keep image's attributes). So from performance point of view it is all good and fast. However there are two things to remember when working with sub images: 1) when processing a sub image which overlaps with another sub image, the 2nd sub image will also change (depending on the size of overlapping region); 2) sub images can not be used after deallocating the main sub image, since their data/memory has gone.

The first issue is demonstrated by the code below. The two sub images created there are overlapping and so processing one will affect another as well.

-- Get image to process
image = Host.GetImage( )

width  = image:Width( )
height = image:Height( )

halfWidth  = width  / 2;
halfHeight = height / 2;

-- create two overlapping sub images
subImage1 = image:GetSubImage( 0, 0, halfWidth, halfHeight )
subImage2 = image:GetSubImage( halfWidth / 2, 0, halfWidth, halfHeight )

-- process the 1st sub image
setHuePlugin:ProcessImageInPlace( subImage1 )

-- don't expect subImage2 to contain original pixel values from the source image,
-- since it contains nothing at all - just points to the ROI in the source image

And here is the code to demonstrate the second issue. In the best case it will write some garbage to the PNG file. In the worst - the process will get terminated.

-- Allocate new image
mainImage = Image.Create( 320, 240, 'RGB24' )

-- Get sub image
subImage  = mainImage:GetSubImage( 0, 0, 160, 120 )
-- Do something with the image and finally release the main image

mainImage:Release( )

-- Some time later we realized we need to save sub image - uups ..
pngExporter = Host.CreatePluginInstance( 'PngExporter' )
pngExporter:ExportImage( 'subimage.png', subImage )

So the idea of the sample above is simple - don't ever try using sub images after the main image was released explicitly or implicitly (by Lua's garbage collector or by the host application running the script). Also don't expect sub image to stay unmodified if main image or another overlapping sub image is processed. In the case if you really need to get an independent sub image which lives its own life - clone it. This will make a deep copy of the ROI.

-- Get sub image and clone it to make sure it is alive after
-- the main image has gone or was modified 
subImage = image:GetSubImage( halfWidth / 2, halfHeight / 2, halfWidth, halfHeight ):Clone( )

 

So we know how to process a rectangular ROI of an image. However it is not the end of it. Some image processing plug-ins do not support in-place image processing, i.e. they don't support the ProcessImageInPlace() method. Instead they provide ProcessImage() method, which keeps source image unchanged and returns new image as a result of image processing. The question then is how to put that result image back into the source image. The answer for this is PutImage() method, which puts one image into another at the specified location. Here is an example for this API:

-- Create instance of Emboss plug-in, which supports only ProcessImage() method
embossPlugin = Host.CreatePluginInstance( 'Emboss' )

-- Get sub image to process
subImage = image:GetSubImage( halfWidth / 2, halfHeight / 2, halfWidth, halfHeight )

-- Emboss plug-in returns result as a new image
processedImage = embossPlugin:ProcessImage( subImage )

-- Put the embossed image back into the main image
image:PutImage( processedImage, halfWidth / 2, halfHeight / 2 )

-- Release the embossed image if we no longer need it
processedImage:Release( )

The PutImage() method can be used not only with the aim of completing image processing on a ROI. In fact any image (for example an image loaded from file) can be put into another image.

The last note is about putting 8 bpp grayscale image on a 24 bpp color image (for example, if Grayscale plug-in was used instead of Emboss in the above sample). In this case the PutImage() method will fail since pixel formats of the two images don't match. To resolve this the GrayscaleToRgb plug-in can be used to convert 8 bpp grayscale image to 24 bpp color image and the result of it can be put into a color image then.

-- Create image processing plug-ins
grayscalePlugin      = Host.CreatePluginInstance( 'Grayscale' )
grayscaleToRgbPlugin = Host.CreatePluginInstance( 'GrayscaleToRgb' )

-- Get sub image to process
subImage = image:GetSubImage( halfWidth / 2, halfHeight / 2, halfWidth, halfHeight )

-- Grayscale only the center part of the main image
processedImage = grayscalePlugin:ProcessImage( subImage )

-- Put the grayscale image back into the main color image
processedRgbImage = grayscaleToRgbPlugin:ProcessImage( processedImage )
image:PutImage( processedRgbImage, halfWidth / 2, halfHeight / 2 )

-- Release temporary images
processedImage:Release( )
processedRgbImage:Release( )

 

Next tutorial: Image slide show combined with video