Color is a perceptual phenomenon related to the human response to different wavelengths of light, mainly in the region of 400 to 700 nanometers (nm). The perception of color arises from the sensitivities of three types of neurochemical sensors in the retina, known as the long (L), medium (M), and short (S) cones. The response of these sensors to photons is shown in Figure 1. Note that each sensor responds to a range of wavelengths.

Figure 1: Relative photon sensitivity of long (L), medium (M), and short (S) cones.

Due to this property of the human visual system, all colors can be modeled as combinations of the three primary color components: red (R), green (G), and blue (B). For the purpose of standardization, the CIE (Commission International de l'Eclairage — the International Commission on Illumination) designated the following wavelength values for the three primary colors: blue = 435.8nm435.8nm, green = 546.1nm546.1nm, and red = 700nm700nm.

The relative amounts of the three primary colors of light required to produce a color of a given wavelength are called tristimulus values. Figure 2 shows the plot of tristimulus values using the CIE primary colors. Notice that some of the tristimulus values are negative, which indicates that colors at those wavelengths cannot be reproduced by the CIE primary colors.

Figure 2: Plot of tristimulus values using CIE primary colors.

A color space allows us to represent all the colors perceived by human beings. We previously noted that weighted combinations of stimuli at three wavelengths are sufficient to describe all the colors we perceive. These wavelengths form a natural basis, or coordinate system, from which the color measurement process can be described. In this lab, we will examine two common color spaces: RGBRGB and YCbCrYCbCr. For more information, refer to [1].

In YCbCrYCbCr, the luminance parameter is related to an overall intensity of the image. The chrominance components are a measure of the relative intensities of the blue and red components. The inverse of the transformation in Equation 1 can easily be shown to be the following.

Download the files girl.tif and ycbcr.mat. For help on image command select the link.

You will be displaying both color and monochrome images in the following exercises. Matlab's image command can be used for both image types, but care must be taken for the command to work properly. Please see the help on the image command for details.

Download the RGBRGB color image file girl.tif , and load it into Matlab using the imread command. Check the size of the Matlab array for this image by typing whos. Notice that this is a three dimensional array of type uint8. It contains three gray scale image planes corresponding to the red, green, and blue components for each pixel. Since each color pixel is represented by three bytes, this is commonly known as a 24-bit image. Display the color image using

image(A);

axis('image');

where AA is the 3-D RGBRGB array.

You can extract each of the color components using the following commands.

RGB = imread('girl.tif'); % color image is loaded into matrix RGB

R = RGB(:,:,1); % extract red component from RGB

G = RGB(:,:,2); % extract green component from RGB

B = RGB(:,:,3); % extract blue component from RGB

Use the subplot and image commands to plot the original image, along with each of the three color components. Note that while the original is a color image, each color component separately is a monochrome image. Use the syntax subplot(2,2,n) , where n=1,2,3,4n=1,2,3,4, to place the four images in the same figure. Place a title on each of the images, and print the figure (use a color printer).

We will now examine the YCbCrYCbCr color space representation. Download the file ycbcr.mat , and load it into Matlab using load ycbcr. This file contains a Matlab array for a color image in YCbCrYCbCr format. The array contains three gray scale image planes that correspond to the luminance (YY) and two chrominance (CbCrCbCr) components. Use subplot(3,1,n) and image to display each of the components in the same figure. Place a title on each of the three monochrome images, and print the figure.

In order to properly display this color image, we need to convert it to RGBRGB format. Write a Matlab function that will perform the transformation of Equation 2. It should accept a 3-D YCbCrYCbCr image array as input, and return a 3-D RGBRGB image array.

Now, convert the ycbcr array to an RGBRGB representation and display the color image. Remember to convert the result to type uint8 before using the image command.

An interesting property of the human visual system, with respect to the YCbCrYCbCr color space, is that we are much more sensitive to distortion in the luminance component than in the chrominance components. To illustrate this, we will smooth each of these components with a Gaussian filter and view the results.

You may have noticed when you loaded ycbcr.mat into Matlab that you also loaded a 5×55×5 matrix, hh. This is a 5×55×5 Gaussian filter with σ2=2.0σ2=2.0. (See the first week of the experiment for more details on this type of filter.) Alter the ycbcr array by filtering only the luminance component, ycbcr(:,:,1) , using the Gaussian filter (use the filter2 function). Convert the result to RGBRGB, and display it using image. Now alter ycbcr by filtering both chrominance components, ycbcr(:,:,2) and ycbcr(:,:,3), using the Gaussian filter. Convert this result to RGBRGB, and display it using image.

Use subplot(3,1,n) to place the original and two filtered versions of the ycbcr image in the same figure. Place a title on each of the images, and print the figure (in color). Do you see a significant difference between the filtered versions and the original image? This is the reason that YCbCrYCbCr is often used for digital video. Since we are not very sensitive to corruption of the chrominance components, we can afford to lose some information in the encoding process.

INLAB REPORT

As was discussed in the first week of this lab, an 8-bit monochrome image allows 256 distinct gray levels. Such images can be displayed on a computer monitor if the hardware supports the required number intensity levels. However, some output devices print or display images with much fewer gray levels. In the extreme case, the gray scale images must be converted to binary images, where pixels can only be black or white.

The simplest way of converting to a binary image is based on thresholding, i.e. two-level (one-bit) quantization. Let f(i,j)f(i,j) be a gray scale image, and b(i,j)b(i,j) be the corresponding binary image based on thresholding. For a given threshold TT, the binary image is computed as the following:

Figure 3: (1) Original gray scale image. (2) Binary image produced by simple fixed thresholding.

(a) (b)

Figure 3 shows an example of conversion to a binary image via thresholding, using T=80T=80.

It can be seen in Figure 3 that the binary image is not “shaded” properly–an artifact known as false contouring. False contouring occurs when quantizing at low bit rates (one bit in this case) because the quantization error is dependent upon the input signal. If one reduces this dependence, the visual quality of the binary image is usually enhanced.

One method of reducing the signal dependence on the quantization error is to add uniformly distributed white noise to the input image prior to quantization. To each pixel of the gray scale image f(i,j)f(i,j), a white random number nn in the range [-A,A][-A,A] is added, and then the resulting image is quantized by a one-bit quantizer, as in Equation 3. The result of this method is illustrated in Figure 4, where the additive noise is uniform over [-40,40][-40,40]. Notice that even though the resulting binary image is somewhat noisy, the false contouring has been significantly reduced.

Figure 4: Random noise binarization.

Halftone images are binary images that appear to have a gray scale rendition. Although the random thresholding technique described in "Binary Images" can be used to produce a halftone image, it is not often used in real applications since it yields very noisy results. In this section, we will describe a better halftoning technique known as ordered dithering.

The human visual system tends to average a region around a pixel instead of treating each pixel individually, thus it is possible to create the illusion of many gray levels in a binary image, even though there are actually only two gray levels. With 2×22×2 binary pixel grids, we can represent 5 different “effective” intensity levels, as shown in Figure 5. Similarly for 3×33×3 grids, we can represent 10 distinct gray levels. In dithering, we replace blocks of the original image with these types of binary grid patterns.

Figure 5: Five different patterns of 2×22×2 binary pixel grids.

Remember from "Binary Images" that false contouring artifacts can be reduced if we can reduce the signal dependence or the quantization error. We showed that adding uniform noise to the monochrome image can be used to achieve this decorrelation. An alternative method would be to use a variable threshold value for the quantization process.

Ordered dithering consists of comparing blocks of the original image to a 2-D grid, known as a dither pattern. Each element of the block is then quantized using the corresponding value in the dither pattern as a threshold. The values in the dither matrix are fixed, but are typically different from each other. Because the threshold value varies between adjacent pixels, some decorrelation from the quantization error is achieved, which has the effect of reducing false contouring.

The following is an example of a 2×22×2 dither matrix,

This is a part of a general class of optimum dither patterns known as Bayer matrices. The values of the threshold matrix T(i,j)T(i,j) are determined by the order that pixels turn "ON". The order can be put in the form of an index matrix. For a Bayer matrix of size 2, the index matrix I(i,j)I(i,j) is given by

and the relation between T(i,j)T(i,j) and I(i,j)I(i,j) is given by

where n2n2 is the total number of elements in the matrix.

Figure 6 shows the halftone image produced by Bayer dithering of size 4. It is clear from the figure that the halftone image provides good detail rendition. However the inherent square grid patterns are visible in the halftone image.

Figure 6: The halftone image produced by Bayer dithering of size 4.

Another method for halftoning is random dithering by error diffusion. In this case, the pixels are quantized in a specific order (raster ordering1 is commonly used), and the residual quantization error for the current pixel is propagated (diffused) forward to unquantized pixels. This keeps the overall intensity of the output binary image closer to the input gray scale intensity.

Figure 7: Block diagram of the error diffusion method.

Figure 7 is a block diagram that illustrates the method of error diffusion. The current input pixel f(i,j)f(i,j) is modified by means of past quantization errors to give a modified input f˜(i,j)f˜(i,j). This pixel is then quantized to a binary value by QQ, using some threshold TT. The error e(i,j)e(i,j) is defined as

where b(i,j)b(i,j) is the quantized binary image.

The error e(i,j)e(i,j) of quantizing the current pixel is is diffused to "future" pixels by means of a two-dimensional weighting filter h(i,j)h(i,j), known as the diffusion filter. The process of modifying an input pixel by past errors can be represented by the following recursive relationship.

The most popular error diffusion method, proposed by Floyd and Steinberg, uses the diffusion filter shown in Figure 8. Since the filter coefficients sum to one, the local average value of the quantized image is equal to the local average gray scale value. Figure 9 shows the halftone image produced by Floyd and Steinberg error diffusion. Compared to the ordered dither halftoning, the error diffusion method can be seen to have better contrast performance. However, it can be seen in Figure 9 that error diffusion tends to create "streaking" artifacts, known as worm patterns.

Figure 8: The error diffusion filter proposed by Floyd and Steinberg.

Figure 9: A halftone image produced by the Floyd and Steinberg error diffusion method.

Download the file house.tif for the following section.

We will now implement the halftoning techniques described above. Save the result of each method in MAT files so that you may later analyze and compare their performance. Download the image file house.tif and read it into Matlab. Print out a copy of this image.

First try the simple thresholding technique based on Equation 3, using T=108T=108, and display the result. In Matlab, an easy way to threshold an image XX is to use the command Y = 255*(X>T); . Label the quantized image, and print it out.

Now create an "absolute error" image by subtracting the binary from the original image, and then taking the absolute value. The degree to which the original image is present in the error image is a measure of signal dependence of the quantization error. Label and print out the error image.

Compute the mean square error (MSE), which is defined by

where NMNM is the total number of pixels in each image. Note the MSE on the printout of the quantized image.

Now try implementing Bayer dithering of size 4. You will first have to compute the dither pattern. The index matrix for a dither pattern of size 4 is given by

Based on this index matrix and Equation 6, create the corresponding threshold matrix.

For ordered dithering, it is easiest to perform the thresholding of the image all at once. This can be done by creating a large threshold matrix by repeating the 4×44×4 dither pattern. For example, the command T = [T T; T T]; will increase the dimensions of TT by 2. If this is repeated until TT is at least as large as the original image, TT can then be trimmed so that is is the same size as the image. The thresholding can then be performed using the command Y = 255*(X>T); .

As above, compute an error image and calculate the MSE. Print out the quantized image, the error image, and note the MSE.

Now try halftoning via the error diffusion technique, using a threshold T=108T=108 and the diffusion filter in Figure 8. It is most straightforward to implement this by performing the following steps on each pixel in raster order:

You do not have to quantize the outer border of the image.

As above, compute an error image and calculate the MSE. Print out the quantized image, the error image, and note the MSE.

The human visual system naturally lowpass filters halftone images. To analyze this phenomenon, filter each of the halftone images with the Gaussian lowpass filter hh that you loaded in the previous section (from ycbcr.mat), and measure the MSE of the filtered versions. Make a table that contains the MSE's for both filtered and nonfiltered halftone images for each of the three methods. Does lowpass filtering reduce the MSE for each method?

INLAB REPORT