Mouth Segmentation in Images: A Review
DOI:
https://doi.org/10.17533/udea.redin.16688Keywords:
Mouth, segmentation, facial images, lipsAbstract
This article presents a review on lip segmentation techniques, focusing in the advances of the last decade. The methods are introduced in a taxonomic manner, making it easier for interpretation and comparison. Each stage in lip segmentation process is highlighted, from the prior color representation study until the later mouth parameterization. A comparison between different methods is presented, when available. Finally, a discussion on each stage in lip segmentation is presented.
Downloads
References
B. Beaumesnil, F. Luthon. “Real time tracking for 3D realistic lip animation”. Proceedings of the 18th International Conference on Pattern Recognition. ICPR 2006. Vol. 1. 2006. pp. 219-222.
I. Arsic, R. Vilagut, J. P. Thiran. “Automatic extraction of geometric lip features with application to multimodal speaker identification”. 2006 IEEE International Conference on Multimedia and Expo. 2006. pp. 161- 164.
J. B. Gómez, J. E. Hernández, F. Prieto, T. Redarce. “Real-time robot manipulation using mouth gestures in facial video sequences”. Lecture Notes in Computer Science. Vol. 4729. 2007. pp. 224-233.
A. E. Salazar, J. E Hernández, F. Prieto. “Automatic quantitative mouth shape analysis”. Lecture Notes in Computer Science. Vol. 4673. 2007. pp. 416-423.
V. Vezhnevets, V. Sazonov, A. Andreeva. “A survey on pixel-based skin color detection techniques”. Proceedings of GraphiCon 2003. pp. 8. Disponible On Line: http://citeseer.ist.psu.edu/676368.html. Consultada el 1 de marzo de 2008.
M. Liévin, F. Luthon. “Unsupervised lip segmentation under natural conditions”. IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP’99: Vol. 6. 1999. pp. 3065-3068.
Z. Jian, M. N. Kaynak, A. D. Cheok, K. C. Chung. “Real-time lip tracking for virtual lip implementation in virtualenvironments and computer games”. The 10th IEEE International Conference on Fuzzy Systems. Vol. 3. 2001. pp. 1359-1362.
S. L. Wang, W. H. Lau, A.W.C. Liew, S. H. Leung. “Robust lip region segmentation for lip images with complex background”. Pattern Recognition. Vol. 40. 2007. pp. 3481-3491.
G. I. Chiou, J. N. Hwang. “Lipreading from color video”. IEEE Transactions on Image Processing. Vol. 6. 1997. pp. 1192-1195.
X. Zhang, R. M. Mersereau. “Lip feature extraction towards an automatic speechreading system”. Proceedings of IEEE International Conference on Image Processing. Vol. 3. 2000. pp. 226-229.
Y. P. Guan. “Automatic extraction of lip based on wavelet edge detection”. Eighth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC ’06. 2006. pp. 125- 132.
N. Eveno, A. Caplier, P. Y. Coulon. “New color transformation for lips segmentation”. IEEE Fourth Workshop on Multimedia Signal Processing. 2001. pp. 3 - 8.
J. Loaiza, J. B. Gómez, A. Ceballos. “Análisis de discriminancia y selección de características de color en im´agenes de labios utilizando redes neuronales”. Memorias del XII Simposio de Tratamiento de Señales, Imágenes y Visión Artificial STSIVA07. 2007. pp. 4.
A. C. Hurlbert, T. A. Poggio. “Synthesizing a color algorithm from examples”. Science. -1988. Vol. 239. Pp. 447-514.
N. Eveno, A. Caplier, P. Y. Coulon. “A parametric model for realistic lip segmentation”. Seventh International Conference on Control, Automation, Robotics And Vision (ICARCV’O2). 2002. pp. 1426-1431.
L. E. Morán, R. Pinto. “Automatic extraction of the lips shape via statistical lips modelling and chromatic feature”. Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007). 2007. pp. 241- 246.
S. L. Wang, S. H. Leung, W. H. Lau. “Lip segmentation by fuzzy clustering incorporating with shape function”. IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP’02. 2002. Vol. 1. pp. 1077-1080.
A. Salazar, F. Prieto. “Extracción y clasificación de posturas labiales en niños entre 5 y 10 años de la ciudad de Manizales”. DYNA. 2006. Vol. 73. pp. 175-188.
R. Collins, Y. Liu, M. Leordeanu. “On-line selection of discriminative tracking features”. IEEE Transaction on Pattern Analysis and Machine Intelligence. Vol. 27. 2005. pp. 1631-1643.
R. L. Hsu, M. Abdel-Mottaleb, A. K. Jain. “Face detection in color images”. IEEE Trans. on Pattern Analysis and Machine Intelligence. Vol. 24. 2002. pp. 696-706.
J. A. Dargham, A. Chekima. “Lips detection in the normalised RGB colour scheme”. Proceedings of 2nd Information and Communication Technologies. ICTTA. Vol. 1. 2006. pp. 1546-1551.
S. Lucey, S. Sridharan, V. Chandran. “Chromatic lip tracking using a connectivity based fuzzy thresholding technique”. Proceedings of the Fifth International Symposium on Signal Processing and its Applications. ISSPA ’99. 1999. pp. 669-672.
J. M. Zhang, D. J. Wang, L. M. Niu, Y. Z. Zhan. “Research and implementation of real time approach to lip detection in video sequences”. Proceedings of the Second International Conference on Machine Learning and Cybernetics. 2003. pp. 2795-2799.
W. Rongben, G. Lie, T. Bingliang, J. Lisheng. “Monitoring mouth movement for driver fatigue or distraction with one camera”. Proceedings of he 7th International IEEE Conference on Intelligent Transportation Systems. 2004. pp. 314-319.
J. Y. Kim, S.Y. Na, R. Cole. “Lip detection using confidence-based adaptive thresholding”. Lecture Notes in Computer Science. Vol. 4291. 2006. pp. 731- 740.
A. Khan, W. Christmas, J. Kittler. “Lip contour segmentation using kernel methods and level sets”. Lecture Notes in Computer Science. Vol. 4842. 2007. pp. 86-95.
H. Bunke, T. Caelli. “Hidden Markov Models: Applications in Computer Vision”. World Scientific Series In Machine Perception And Artificial Intelligence Series. Vol. 45. World Scientific Publishing Co. 2001. pp. 244.
R. Chellappa, A. K. Jain. Markov Random Fields: Theory and Application. Ed.Academic Press. 1993. pp. 581.
M. Sadeghi, J. Kittler, K. Messer. “Real time segmentation of lip pixels for lip tracker initialization”. Lecture Notes in Computer Science. Vol. 2124. 2001. pp. 317-324.
P. Gacon, P. Y. Coulon, G. Bailly. “Statistical active model for mouth components segmentation”. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’ 05. Vol. 2. 2005. pp. 1021-1024.
B. Goswami, W. J. Christmas, J. Kittler. “Statistical estimators for use in automatic lip segmentation”. Proceedings of the 3rd European Conference on Visual Media Production (CVMP). 2006. pp. 79-86.
I. Mpiperis, S. Malassiotis, M. G. Strintzis. “Expression compensation for face recognition using a polar geodesic representation”. Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’06). 2006. pp. 224-231.
C. Bouvier, P. Y. Coulon, X. Maldague. “Unsupervised lips segmentation based on ROI optimisation and parametric model”. IEEE International Conference on Image Processing, ICIP 2007. Vol. 4. 2007. pp. 301- 304.
A. K. Jain, M. N. Murty, P. J. Flynn. “Data clustering: A review. ACM Computer Surveys”. Vol. 31. 1999. pp. 264-323.
J. C. Bezdek. Pattern Recognition With Fuzzy Objective Function Algorithms. Plenum Press, 1981. pp. 256.
A. K. Jain, M. N. Murty, P. J. Flynn. “Data clustering: a review”. ACM Computing Surveys. Vol. 31. 1999. pp. 264-323.
S. H. Leung, S. L. Wang, W. H. Lau. “Lip Image Segmentation Using Fuzzy Clustering Incorporating an Elliptic Shape Function”. IEEE Transactions on Image Processing. Vol. 13. 2004. pp. 51-62.
A. W. C. Liew, S. H. Leung, W. H. Lau. “Segmentation of color lip images by spatial fuzzy clustering”. IEEE Transactions on Fuzzy Systems. Vol. II. 2003. pp. 542- 549.
S. L. Wang, W. H. Lau, S. H. Leung, A. W. C. Liew. “Lip segmentation with the presence of beards”. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’04. Vol. 3. 2004. pp. 529- 532.
Y. Mitsukura, M. Fukumi, N. Akamatsu. “A design of face detection system by using lip detection neuralnetwork and skin distinction neural network”. Proceedings of IEEE International Conference on Systems, Man, and Cybernetics. Vol. 4. 2000. pp. 2789 - 2793.
H. Takimoto, Y. Mitsukura, M. Fukumi, N. Akamatsu. “Face detection and emotional extraction system using double structure neural network”. Proceedings of the International Joint Conference on Neural Networks. Vol. 2. 2003. pp. 1253 - 1257.
Y. Mitsukura, M. Fukumi, N. Akamatsu. “A design of face detection system using evolutionary computation”. Proceedings of TENCON 2000. Vol. 2. 2000. pp. 398 - 402.
W. N. Lie, H. C. Hsieh. “Lips detection by morphological image processing”. Proceedings of the 1998 Fourth International Conference on Signal Processing, ICSP’98. Vol. 2. 1998. pp. 1084 - 1087.
R. A. Rao, R. M. Mersereau. “Lip modeling for visual speech recognition”. 1994 Conference Record of the Twenty-Eighth Asilomar Conference on Signals, Systems and Computers. Vol. 1. 1994. pp. 587-590.
T. F. Cootes, D. Cooper, C. J. Taylor, J. Graham. “Active shape models - their training and application”. Computer Vision and Image Understanding. Vol. 61. 1995. pp. 38 - 59.
T. F. Cootes, G. J. Edwards, C. J. Taylor. “Active appearance models”. Proceedings of the European Conference on Computer Vision. Vol. 2. 1998. pp. 484- 498.
A. Caplier. “Lip detection and tracking”. Proceedings of 11th International Conference on Image Analysis and Processing. 2001. pp. 8 - 13.
A. Caplier, P. Delmas, D. Lam. “Robust initialisation for lips edges detection”. Proceedings of 11th Scandinavian Conference on Image Analysis. 1999. pp. 523-528.
A. Turkmani, A. Hilton. “Appearance-based inner-lip detection”. Proceedings of the 3rd European Conference on Visual Media Production (CVMP 2006). 2006. pp. 176-176.
M. Jiang, Z. H. Gan, G. M. He, W. Y. Gao. “Combining particle filter and active shape models for lip tracking”. Proceedings of the 6th World Congress on Intelligent Control and Automation (WCICA 2006). Vol. 2. 2006. pp. 9897- 9901.
Y. D. Jian, W. Y. Chang, C. S. Chen. “Attractor-guided particle filtering for lip contour tracking”. Lecture Notes in Computer Science. Vol. 3851. 2006. pp. 653 - 663.
J. E. Hernández, F. Prieto, T. Redarce. “Fast active contours for sampling”. Proceedings of Electronics, Robotics and Automotive Mechanics Conference. Vol. 2. 2006. pp. 9 - 13.
C. Xu, J. L. Prince. “Gradient Vector Flow: A new external force for snakes”. Proceedings of Computer Vision and Pattern Recognition (CVPR‘97). San Juan, Puerto Rico. 1997. pp. 66-71.
C. Xu, J. L. Prince. “Snakes, shapes, and gradient vector flow”. IEEE Transactions on Image Processing. Vol. 7. 1998. pp. 359-369.
A. S. M. Sohail, P. Bhattacharya. “Automated lip contour detection using the level set segmentation method”. 14th International Conference on Image Analysis and Processing (ICIAP 2007). 2007. pp. 425- 430.
P. Viola, M. J. Jones. “Robust real-time face detection”. International Journal of Computer Vision. Vol. 57. 2004. pp. 137-154.
N. Eveno, A. Caplier, P. Y. Coulon. “Accurate and quasi-automatic lip tracking”. IEEE Trans. on Circuits and Systems for Video Technology. Vol. 14. 2004. pp. 706-715.
Z. Hammal, N. Eveno, A. Caplier, P. Y. Coulon. “Parametric models for facial features segmentation”. Signal Processing. Vol. 86. 2005. pp. 399-413.
H. Seyedarabi, W. S. Lee, A. Aghagolzadeh. “Automatic lip tracking and action units classification using two-step active contours and probabilistic neural networks”. Proc. of the Canadian Conf. on Electrical and Computer Engineering, CCECE’06. 2006. pp. 2021-2024.
S. Werda, W. Mahdi, A. B. Hamadou. “Colour and geometric based model for lip localisation: Application for lip-reading system”. 14th International Conference on Image Analysis and Processing (ICIAP 2007). 2007. pp. 9-14.
J. S. Chang, E.Y. Kim, S. H. Park. “Lip contour extraction using level set curve evolution with shape constraint”. Lecture Notes in Computer Science. Vol. 4552. 2007. pp. 583-588.
M. K. Moghaddam, R. Safabakhsh. “TASOM-based lip tracking using the color and geometry of the face”. Proceedings of the Fourth International Conference on Machine Learning and Applications, ICMLA’05. 2005. pp. 6.
L. Xie, X. L. Cai, Z. H. Fu, R. C. Zhao, D. M. Jiang. “A robust hierarchical lip tracking approach for lipreading and audio visual speech recognition”. Proceedings of 2004 International Conference on Machine Learning and Cybernetics. Vol. 6. 2004. pp. 3620-3624.
K. Messer, J. Matas, J. Kittler, J. Luettin, G. Maitre. “XM2VTSDB: the extended M2VTS database”. Proceedings of the Second International Conference on Audio- and Video-based Biometric Person Authentication, AVBPA’99. 1999. pp. 72-77.
A. M. Martínez, R. Benavente. “The AR face database”. Technical Report 24. Computer Vision Center (CVC). Universidad Autónoma de Bacelona, Barcelona, España. 1998.
G. Chetty, M. Wagner. “Automated lip feature extraction for liveness verification in audio-video authentication”. Proceedings of Image and Vision Computing. 2004. pp. 17-22.
Downloads
Published
How to Cite
Issue
Section
License
Revista Facultad de Ingeniería, Universidad de Antioquia is licensed under the Creative Commons Attribution BY-NC-SA 4.0 license. https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en
You are free to:
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material
Under the following terms:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
The material published in the journal can be distributed, copied and exhibited by third parties if the respective credits are given to the journal. No commercial benefit can be obtained and derivative works must be under the same license terms as the original work.