How Image Sensors Impact Video Forensics

Video Forensics
The type, quality, and technology behind digital camera image sensors plays a major role in video forensics and how digital images and video enhancements are completed.

The technology of video forensics is impacted greatly by the technology used in recording a video, or capturing still images. The image sensor is a main component in all digital cameras. Knowledge of the technology of Image sensors and image scanning techniques is essential in the field of video forensics. An Image sensor consists of many photo sites which correspond to the pixels in an image. Each photo site registers the light it receives and converts this light into a corresponding number of electrons, which are interpreted as shades of gray. Bright light will produce more electrons and dull light will produce fewer electrons. There are different methods (RGB, CMY and CMYG color systems) which are used to register colors in digital cameras. A CMY system produces better light sensitivity than RGB. CCD (charge-coupled device) image sensors use the CMYG color system, whereas progressive scan image sensors use the RGB color system. There are two main technologies used to develop image sensors:

  1. CCD (Charge Coupled device)
  2. CMOS (Complementary Metal Oxide Semiconductor)

CCD Technology: In CCD technology, charges from pixels are converted to voltage levels, buffered, and are then sent off the CCD chip as an analog signal. With CCD, every pixel has limited output nodes, therefore the image quality is thus very high. These cameras have been used for more than 35 years and are more sensitive to light than CMOS sensors. However CCD sensors are expensive and consume much more power than CMOS sensors. Think of the video forensics involved in the Rodney King video.

CMOS Technology: In CMOS technology, amplifiers and analog to digital converters are already integrated in the image sensor. This allows better integration and more advanced functions. Recent advancements in CMOS technology have improved the quality of images produced using these sensors. These improvements mean that the field of video forensics has advanced tremendously, especially in the last few years. 

There are, however, many other characteristics in image sensors that impact the quality of images. For example, the size of the image sensor and size of the pixels also impact the image quality. A larger image sensor with more pixels will produce higher resolution images with greater detail. A larger pixel will store more electrons from light exposure and thus will be more sensitive to light. Another factor that has an impact on image quality is the dynamic range of the sensors. An image sensor with high dynamic range will capture both dark and bright objects without “noise” on the image.

Image Scanning Techniques

Information produced by Image sensors is read and displayed using Image Scanning techniques. There are two prevalent image scanning techniques:

  1. Interlaced scanning and
  2. Progressing Scanning

Interlaced Scanning: Interlaced scanning was invented in the 1930’s and is today used in CCD based image sensors and older analog cameras. An interlaced image from a CCD camera produces one field displaying odd lines and another field displaying even lines. At any given time, only half the image lines are transmitted, first the odd and next the even lines are transmitted, then the even and odd lines are combined to form an image on every line. To allow the human visual and cognitive functions to interpret these field lines as a complete image and not separate odd or even field lines, these lines are refreshed at a certain frequency or number of frames per second. These interlacing techniques have been extensively used in analog cameras, television and VHS videos for a very long time.

Deinterlacing Techniques: There are many deinterlacing techniques that are used to show interlaced videos on either television or computer screens. An advanced technique, called motion adaptive deinterlacing, is used to produce sharp and full resolution images. This technique uses blending and calculation of motion to deinterlace the interlaced images. Another method of deinterlacing uses line doubling or interpolation to first remove either of the field lines and then double the lines of the remaining field. This method finishes the comb effect but typically has a negative impact on image quality.

Progressive Scanning: Progressive scanning is a technique that can be used in both CCD and CMOS image sensors. In Progressive scanning, values are used from every pixel of the image sensor and data is scanned sequentially. This produces a full frame image which is then sent over a network or stored. Because there is literally no flickering effect in progressive scanning, it can capture moving objects much better and is preferred in video surveillance applications.