Recoverable Privacy Protection for Video Content Distribution
 Guangzhen Li^{1},
 Yoshimichi Ito^{1}Email author,
 Xiaoyi Yu^{2},
 Naoko Nitta^{1} and
 Noboru Babaguchi^{1}
DOI: 10.1155/2009/293031
© Guangzhen Li et al. 2009
Received: 16 April 2009
Accepted: 26 November 2009
Published: 10 January 2010
Abstract
This paper presents a method which attains recoverable privacy protection for video content distribution. The method is based on discrete wavelet transform (DWT), which generates scaling coefficients and wavelet coefficients. In our method, scaling coefficients, which can be regarded as a lowresolution image of an original image, are used for producing privacyprotected image. On the other hand, wavelet coefficients, which can be regarded as privacy information, are embedded into the privacyprotected image via information hiding technique. Therefore, privacy protected image can be recovered by authorized viewers if necessary. The proposed method is fully analyzed through experiments from the viewpoints of the amount of the embedded privacy information, the deterioration due to the embedding, and the computational time.
1. Introduction
Recently, video surveillance has received a lot of attention as a useful technology for crime deterrence and investigations and has been widely deployed in many circumstances such as airports, convenience stores, and banks. Video surveillance allows us to remotely monitor a live or recorded video feed which often includes objects such as people. Although video surveillance contributes to realizing a secure and safe community, it also exposes the privacy of the object in the video.
Over the past few years, a lot of techniques on privacy protection in video surveillance system have been proposed [1–7]. Newton et al. [1] proposed an algorithm to protect the privacy of the individuals in video surveillance data by deidentifying faces. Kitahara et al. [2] proposed a video capturing system called Stealth Vision, which protects the privacy of the objects by blurring or pixelizing their images. In [3], Wickramasuriya et al. protect object's privacy based on the authority of either object or viewers. In [4], Boyle et al. considered face obscuring for privacy protection and discussed the effects of blurring and pixelizing. Crowley et al. [5] proposed a method for privacy protection by replacing an socially inappropriate original image with a socially acceptable image using eigenspace coding technique. Chinomi et al. [6] proposed privacyprotected video surveillance system called PriSurv, which adaptively protects objects' privacy based on their privacy policies which are determined according to closeness between objects and viewers.
Although these techniques fulfill some requirements of privacy protection, it also has a potential security flaw when privacyprotected videos produced by the above techniques are distributed on the Internet, because these techniques do not provide methods for recovering the original videos from privacyprotected videos. For example, suppose that a surveillance video camera is installed around school route, and the camera distributes a privacyprotected video on the Internet in usual case. When a crime has occurred around school route, police wants to observe the original image of a suspect in privacyprotected video. In addition, when parents want to observe the situation of their children, they require the video as they are. Thus, in order to improve the security of privacyprotected surveillance system, the privacy protection which can recover the original image from privacyprotected image is strongly required. We refer to such privacy protection as recoverable privacy protection.
Concerning recoverable privacy protection, several techniques have been proposed [8–11]. Dufaux and Ebrahimi [8] and Dufaux et al. [9] proposed a method based on transform domain scrambling of regions of interest in a video sequence. A pioneering work was done by Zhang et al. [10]. They proposed a method for storing original privacy information in video using information hiding technique, and it can recover the original privacy information if necessary. However, the method has the drawback that the large amount of the privacy information must be embedded to recover the original image since the privacy information is obtained from the whole information of the object regions. Even if all the privacy information could be embedded using data compression technique, it requires huge computational loads. In [11], Yu and Babaguchi proposed another method to realize recoverable privacy protection. Their method masks a real face (privacy information) with a virtual face (newly generated face for anonymity). To deal with the huge payload problem of privacy information hiding, the method uses statistical active appearance model (AAM) [12] for privacy information extraction and recovering. It is shown that the method can embed the privacy information into video without affecting its visual quality and keep its practical usefulness. However, the method requires a set of face images for training statistical AAM.
In this paper, we propose a method for recoverable privacy protection based on discrete wavelet transform (DWT). It is well known that DWT is one of the useful tools for multiresolution analysis. DWT generates scaling coefficients and wavelet coefficients. Since an image consisting of scaling coefficients can be regarded as a reducedsize image of its original, we refer to it as a lowresolution image. A lowresolution image is used for producing privacyprotected image by expanding it to the size of the original image. Using wavelet coefficients, together with a lowresolution image, one can recover its original image. Therefore, wavelet coefficients are regarded as privacy information. In order to prevent unauthorized viewers from recovering privacyprotected image, our method embeds wavelet coefficients into the privacyprotected image via information hiding technique. By this, the privacyprotected image can only be recovered by authorized viewers if necessary. Furthermore, it is shown that the amount of the privacy information of the object can significantly be reduced compared to Zhang's method [10]. In addition, in contrast with Yu's method [11], our method requires no training beforehand.
Some results of this paper have already been reported in [13], where a method for bitmap image is developed. In this paper, we extend the method so as to deal with compression technique such as JPEG [14] and JPEG2000 [15] for content distribution on the Internet and provide detailed algorithms for privacy information extracting, hiding, and recovering. Furthermore, we analyze the effectiveness of the proposed method through numerical experiments from the viewpoints of the amount of the embedded privacy information, the deterioration due to the embedding, and the computational time.
This paper is organized as follows. In Section 2, we show the architecture of the proposed system. In Section 3, the discrete wavelet transform is introduced and the new image processing method for privacy information extraction is proposed. The privacy information hiding and recovering are described in Section 4. Experimental results on the proposed method are presented in Section 5. Conclusions are made in Section 6.
2. System Architecture
When there are multiple object regions in a single image, the obscuration for privacy protection would differ according to each object region. However, in this paper, we do not deal with this issue for simplicity. We have considered such an issue in [7].
For the decoding procedure, the privacy information and region information are extracted with the procedure of the information hiding method using the secret key. Then the original image of the objects could be recovered by using the encoding process in the reversed order.
3. Privacy Information Extraction
In our system, the original image of the object region is transformed into two sets of data: a lowresolution image and a set of wavelet coefficients. This process is carried out by using discrete wavelet transform. In what follows, first, the discrete wavelet transform is introduced and then, the proposed method is described.
3.1. Discrete Wavelet Transform
The discrete wavelet transform (DWT) is computed by successive lowpass and highpass filtering of discrete signal. We use a Haar discrete wavelet transform to extract privacy information. The Haar DWT of image is given by the following equations:
where , , . The sequences and , which correspond to the impulse responses of lowpass and highpass filters, respectively, are defined as follows:
3.2. DWTBased Privacy Information Extraction
We can extract the bounding box of the object in the surveillance video, using the background subtraction method of adaptive Gaussian mixture model [16, 17]. The bounding box is referred to as an object region. We transform the object region from color space to color space where is the luminance component, and and are the blue and red chrominance components, respectively. According to the fact that human eyes are only sensitive to the luminance but not sensitive to the chrominance, the sensitive privacy information is only included in image. Therefore, we apply the DWTbased method presented in Section 3.1 for images and produce the lowresolution image and the set of wavelet coefficients . When image is given by and level DWT is employed, and are defined as follows:
4. Privacy Information Hiding and Recovering
Let the size of image be and let the level of DWT be . Then, the encoded data sequence which is embedded in privacyprotected image is generated by the set of wavelet coefficients . The process is shown in Algorithm 1.
Algorithm 1:

Input:
Set of wavelet coefficients:
Quantization step size of the privacy information:

Processing:
Step : Generate the quantized data sequence by
quantizing , Where is the largest integer
that does not exceed .
Step : Find the intervals consist of successive zeros (but the number
of zeros is more than ) from the data sequence , where , , are the number of
such intervals,the th smallest element of the set of starting points of successive zeros
, and the th smallest element of the set
of end points of successive zeros .
For this calculation, we suppose .
Step :
For ( ):
For ( ):
If ( ):
If ( ): Encode the data sequence of the interval with the run length
coding, and add it to the data sequence . Then Goto Next
Else ( ): Goto Next
If ( ): Encode to binary bits, and add the binary bits to the data sequence with
delimiter digit .
Else ( ): Encode to binary bits, and add the binary bits sandwiched by
the sign bit and delimiter digit to the data sequence .

Output:
Data sequence to be embedded:
Next, embed the encoded data sequence to the frequency domain of the privacyprotected image after quantization via amplitude modulo modulation (AMM) [18] according to the following equation:
where is the set of integers given by . And and are the frequency components at frequency before and after embedding, respectively, and the frequencies for embedding are described by a secret key. Therefore, only the viewer who has the secret key can extract the embedded information from the image. The embedded color component is in the order of , , . Finally, we can obtain a compressed privacyprotected image after the entropy coding of JPEG/JPEG2000.
For the recovering procedure, the privacy information and region information are extracted by taking the congruence modulo of the corresponding pixel values. Then the original image of the object is obtained by the recovering process of the Figure 1.
5. Experiments
Average number of pixels of object regions per frame (pixel/frame).
Level 1  Level 2  Level 3  Level 4  Level 5  

ice (352 288) 





ice (704 576) 





deadline (352 288) 





Average number of bits for embedded data sequences per frame (bit/frame).
Level 1  Level 2  Level 3  Level 4  Level 5  

ice (352 288) 





ice (704 576) 





deadline (352 288) 





From Tables 1 and 2, we can observe that the number of pixels of object regions and the amount of embedded data sequence of ice are about four times larger than those of ice . This is quite natural because the resolution of ice is four times larger than that of ice . We can also observe that the amount of embedded data sequences of deadline is larger than that of ice , whereas the numbers of pixels of object regions of deadline and ice are similar.
In the following, we consider the influence of the above values on the performance of the proposed method. We employ the following three measures for performance evaluation: API, PSNR, and processing time.
API is an abbreviation of Average of Privacy Information and is defined as follows:
Namely, API is the average of the required bits of data sequences for recovering one pixel in the object regions and is regarded as a measure of the amount of privacy information which should be embedded. API is also calculated by using the following equation, which is equivalent to (5):
Therefore, API can be calculated by Tables 1 and 2.
PSNR (Peak SignaltoNoise Ratio) is used as a measure of deterioration of recovered image and is also used for evaluating the influence of embedded data sequence on privacyprotected image. PSNR between and is defined as follows:
where MSE (Mean Square Error) is defined by
5.1. Evaluation of the Amount of Privacy Information and the Deterioration Due to Embedding
Next, we consider the deterioration of the privacyprotected image due to the privacy information embedding. The deterioration due to embedding can be estimated by (7) and (8). Since amplitude modulo modulation in Section 4 uses congruence modulo 3, in (8) becomes less than or equal to 2. Therefore, an upper bound of MSE can be calculated as follows:
By this inequality, we obtain
This result implies that the deterioration of the privacyprotected image due to the embedding of privacy information is small enough so that we can ignore the influence of the embedding.
5.2. Evaluation of the Deterioration of the Recovered Image
5.3. Evaluation of Computational Time
Average CPU time for generating recoverable privacyprotected image per frame (sec/frame).
Level 1  Level 2  Level 3  Level 4  Level 5  

ice (352 288)  0.139  0.142  0.144  0.144  0.140 
ice (704 576)  0.646  0.662  0.668  0.664  0.661 
deadline (352 288)  0.137  0.142  0.142  0.143  0.140 
Average CPU time for recovering image per frame (sec/frame).
level 1  level 2  level 3  level 4  level 5  

ice (352 288)  0.110  0.112  0.112  0.110  0.110 
ice (704 576)  0.452  0.472  0.478  0.478  0.467 
deadline (352 288)  0.109  0.113  0.114  0.113  0.112 
6. Conclusion
In this paper, we have presented a method which attains recoverable privacy protection for video content distribution. By the proposed method, all the privacy information can be embedded into the privacyprotected image even if the level of DWT is large and the object region size is equal to the size of the whole image. We also show that proposed method recovers the privacyprotected image with low deterioration, and the computational time for privacy protection and image recovering is small.
The proposed method is based on the idea that the privacy information needed for recovering video sequence is embedded in the video sequence itself (which is referred to as selfrecoverable), and only authorized viewers can extract the privacy information. An alternate approach is that the privacy information for recovering video sequence is stored outside (e.g., in a server), and only authorized viewers can access the privacy information. Such a system would be more secure than selfrecoverable system, although the system is inferior with respect to the convenience. It is desirable to develop the system that can deal with both methods for protecting privacy information.
Currently, we use the same DWT level for each object region in a single image. However, it is better to change the DWT level adaptively according to the size of object region since the permissible visible detail of each object is not identical. The realization of this function is one of our future work. In our current method, when JPEG2000 is applied for image compression, we have to calculate DWT twice; one is for image compression using 5–3 filter, and another one is for privacy protection using Haar bases. If these two DWTs can be unified, the process of recoverable privacy protection becomes much simpler, and the computational time will be further reduced. The development of such methods is also our future work.
Declarations
Acknowledgments
This work was supported in part by a GrantinAid for scientific research from the Japan Society for the Promotion of Science and by SCOPE from the Ministry of Internal Affairs and Communications, Japan.
Authors’ Affiliations
References
 Newton EM, Sweeney L, Malin B: Preserving privacy by deidentifying face images. IEEE Transactions on Knowledge and Data Engineering 2005, 17(2):232243. 10.1109/TKDE.2005.32View ArticleGoogle Scholar
 Kitahara I, Kogure K, Hagita N: Stealth vision for protecting privacy. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 4: 404407.View ArticleGoogle Scholar
 Wickramasuriya J, Datt M, Mehrotra S, Venkatasubramanian N: Privacy protecting data collection in media spaces. Proceedings of the 12th ACM International Conference on Multimedia (ACM Multimedia '04), October 2004, New York, NY, USA 4855.View ArticleGoogle Scholar
 Boyle M, Edwards C, Greenberg S: The effects of filtered video on awareness and privacy. Proceedings of the ACM Conference on Computer Supported Cooperative Work, December 2000, Philadelphia, Pa, USA 110.Google Scholar
 Crowley JL, Coutaz J, Babaguchi F: Things that see. Communications of the ACM 2000, 43(3):5464. 10.1145/330534.330540View ArticleGoogle Scholar
 Chinomi K, Nitta N, Ito Y, Babaguchi N: PriSurv: privacy protected video surveillance system using adaptive visual abstraction. Proceedings of the 14th International Multimedia Modeling Conference (MMM '08), January 2008, Kyoto, Japan, Lecture Notes in Computer Science 4903: 144154.Google Scholar
 Yu X, Chinomi K, Koshimizu T, Nitta N, Ito Y, Babaguchi N: Privacy protecting visual processing for secure video surveillance. Proceedings of the International Conference on Image Processing (ICIP '08), October 2008, San Diego, Calif, USA 16721675.Google Scholar
 Dufaux F, Ebrahimi T: Scrambling for video surveillance with privacy. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2006, New York, NY, USAGoogle Scholar
 Dufaux F, Ouaret M, Abdeljaoued Y, Navarro A, Vergnenègre F, Ebrahimi T: Privacy enabling technology for video surveillance. Mobile Multimedia/Image Processing for Military and Security Applications, April 2006, Kissimmee, Fla, USA, Proceedings of SPIE 6250:View ArticleGoogle Scholar
 Zhang W, Cheung SCS, Chen M: Hiding privacy information in video surveillance system. Proceedings of the International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 3: 868871.Google Scholar
 Yu X, Babaguchi N: Privacy preserving: hiding a face in a face. Proceedings of the 8th Asian Conference on Computer Vision (ACCV '07), November 2007, Tokyo, Japan, Lecture Notes in Computer Science 4844: 651661.Google Scholar
 Cootes TF, Edwards GJ, Taylor CJ: Active appearance models. Proceedings of the European Conference on Computer Vision, 1998 2: 484498.Google Scholar
 Li G, Ito Y, Yu X, Nitta N, Babaguchi N: A discrete wavelet transform based recoverable image processing for privacy protection. Proceedings of the International Conference on Image Processing (ICIP '08), October 2008, San Diego, Calif, USA 13721375.Google Scholar
 Wallace GK: The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics 1992, 38(1):1834. 10.1109/30.125072View ArticleGoogle Scholar
 Skodras A, Christopoulos C, Ebrahimi T: The JPEG 2000 still image compression standard. IEEE Signal Processing Magazine 2001, 18(5):3658. 10.1109/79.952804View ArticleMATHGoogle Scholar
 Grimson WEL, Stauffer C, Romano R, Lee L: Using adaptive tracking to classify and monitor activities in a site. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 1998, Santa Barbara, Calif, USA 2229.Google Scholar
 Stauffer C, Grimson WEL: Learning patterns of activity using realtime tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000, 22(8):747757. 10.1109/34.868677View ArticleGoogle Scholar
 Princeton University,Wu M: Multimedia data hiding, Ph.D. dissertation. Princeton University; 2001.Google Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.