• Imagine Framework

  • Completed 2013

Imagine is a 256-bit image processing engine that serves as the foundation for several of my image and computer vision research projects. I began this project in 2009 after scouring the web and being unable to find an image processing library that fit my particular needs. In particular, I was interested in a library that was:


  1. Fast: multi-threaded vectorized CPU implementations plus CUDA GPU implementations

  2. Portable: able to run on PCs and mobile devices without significant updates

  3. Readable: well written and documented code

  4. Accurate: reliable implementation that consistently produces correct results

  5. Usable: the library must support common image file formats

  6. Lightweight: does not contain significant bloat or extraneous features



Design Goals

My primary goal for Imagine was that it be powerful yet also easy to understand. By avoiding unnecessary complexity, I believe that this software is approachable for engineers without extensive image processing experience, and serves as a great learning resource.

Over the years a number of professors have used this library as a teaching aid in their courses and the feedback has been unanimously positive. The code is highly commented with descriptive explanations of both the theories and the practical considerations that went into the implementation.



Features

Imagine contains a 256 bit (64 bits per channel) image processing engine that relies upon a fully customizable set of filters, formats, and types. Types are described by a flexible data format that allows developers to easily create data layouts that are automatically accepted and interpreted by the rest of the engine. A preselected set is provided that includes some of the most common formats, including:


  • VN_IMAGE_FORMAT_R8

  • VN_IMAGE_FORMAT_R16

  • VN_IMAGE_FORMAT_D16

  • VN_IMAGE_FORMAT_R24

  • VN_IMAGE_FORMAT_D32

  • VN_IMAGE_FORMAT_R32

  • VN_IMAGE_FORMAT_R32G32B32

  • VN_IMAGE_FORMAT_R8G8B8A8

  • VN_IMAGE_FORMAT_R10G10B10A2

  • VN_IMAGE_FORMAT_R16G16B16A16

  • VN_IMAGE_FORMAT_R8G8B8S

  • VN_IMAGE_FORMAT_R16G16B16S

  • VN_IMAGE_FORMAT_R32G32B32S

  • VN_IMAGE_FORMAT_R16F

  • VN_IMAGE_FORMAT_R32F

  • VN_IMAGE_FORMAT_R3G3B2

  • VN_IMAGE_FORMAT_R5G6B5

  • VN_IMAGE_FORMAT_R5G5B5A1

  • VN_IMAGE_FORMAT_R4G4B4A4

  • VN_IMAGE_FORMAT_R8G8B8

  • VN_IMAGE_FORMAT_R16G16B16

  • VN_IMAGE_FORMAT_R32G32B32A32

  • VN_IMAGE_FORMAT_R8S

  • VN_IMAGE_FORMAT_R16S

  • VN_IMAGE_FORMAT_R32S

  • VN_IMAGE_FORMAT_R32G32F

  • VN_IMAGE_FORMAT_R16G16B16F

  • VN_IMAGE_FORMAT_R32G32B32F

  • VN_IMAGE_FORMAT_R16G16B16A16F

  • VN_IMAGE_FORMAT_R32G32B32A32F


Filters are the primary workhorse of Imagine and they are comprised of kernels and operators. Kernels provide a common interface to perform image sampling while operators are responsible for image manipulation tasks. Both of these systems are supported by an array of helpers that manage data conversion, packing, and other tasks required by the engine.

Imagine currently supports the following configurable kernels:


  • Average

  • Bicubic

  • Bilinear

  • Bokeh

  • Mitchell-Netravali

  • Coverage

  • Gaussian

  • Catmull-Rom Bicubic

  • Lanczos

  • Nearest Neighbor

  • Cubic Spline

  • Cubic B-Spline


Imagine currently supports the following operators which in turn rely upon the kernels to sample images. In this manner, a simple blur operator automatically supports gaussian blur, coverage blur, etc.


  • Arithmetic Adjust

  • Blit

  • Blur

  • Tilt/Shift

  • Brightness

  • Clone

  • Inverse

  • Precise Mean

  • Resample/Resize

  • Sepia Toning

  • Splat

  • Barrel Correction

  • Fourier Transform

  • Haar Wavelet Transform

  • Gaussian Pyramid

  • Sharpen Image

  • Motion Blur

  • Radial Blur

  • Depth of field (requires D16 companion)

  • Lens Glare and Flare

  • Contrast

  • Convert Format

  • Crop

  • Desaturate

  • Flip

  • Focal Blur

  • Lomo

  • Mosaic

  • Rotate

  • Convert Space (RGB,YUV,HSV, etc.)

  • Vignette

  • Bayer Demosaic

  • Discrete Cosine Transform

  • Harris Detector

  • Detect Edges

  • Clarify (noise removal)

  • Ripple and Wave

  • Anamorphic Flare

  • Motion Vectors

  • HDR Bloom



File Format Support

Imagine supports the following image file formats. Also note, adding a new image format to Imagine is easy and (usually) does not require any updates to the rest of the engine.


  • 24 and 32 bit Bitmaps (.bmp)

  • Portable network graphics (.png)

  • Joint Photographic Experts Group (.jpeg, .jpg)

  • Portable Graymaps (.pgm, .ppm)

  • 24 and 32 bit uncompressed Targa (.tga)

  • Primitive Texture Compression (.ptc)

  • OpenEXR HDR Images (.exr)

  • Tagged Image File Format (.tiff, .tif)



Source Code

Imagine is currently closed source because it is being used under a commercial license by a partner. However, I have made an early and unoptimized version of the resampling code available at the link below.

  Imagine Resampler Source Code



More Information

For more information about the inner workings of Imagine, check out this blog post.