基于AR-WLD和分块相似度加权的遮挡表情识别
|
摘要
针对非约束环境下,局部遮挡可能会对表情识别造成干扰并影响最终判别结果的问题,提出一种基于非对称邻域韦伯局部描述子(AR-WLD)和分块相似度加权的表情识别算法。在特征描述上,相比传统的韦伯局部描述子(WLD),AR-WLD将原有的方形邻域扩展成非对称邻域,并进行了多尺度分析,增强了算子的表征能力。在分类判别时,为了区分不同面部区域对表情识别的贡献度,对表情区域进行了不重叠分块;引入了信息熵来衡量不同子块包含的不确定信息,依据信息量定义相似性距离的权重;通过分块相似度加权求和,实现表情判别。在JAFFE库和CK库上的实验结果表明:当表情图像存在遮挡时,AR-WLD可以有效地提高WLD的分类能力和稳健性,分块相似度加权的分类算法则进一步降低遮挡区域对表情识别的干扰。
We propose an expression recognition algorithm based on asymmetric region Weber local descriptor(ARWLD)and block similarity weighting,which can reduce the interference of occlusion area to facial expression recognition and the impacts of the final discriminants in unconstrained environment.In the feature description,compared with the traditional WLD,the AR-WLD extends the original square neighborhood into an asymmetric neighborhood,and enhances the feature analysis in a multiscale.In order to distinguish the contribution of different facial regions to expression recognition,the non-overlapping expression regions are classified in classification discrimination.Information entropy is introduced to measure the uncertain information contained in different subblocks,and the weight of similarity distance is defined according to the information amount.The facial expression discrimination is achieved by the block similarity weighted summation.The experimental results on the databases of JAFFE and CK show that the AR-WLD can effectively improve the classification performance and robustness of the WLD when the expression image is partially occluded,and the classification algorithm based on block similarity weighting can further reduce the interference of the occlusion area to facial expression recognition.
We propose an expression recognition algorithm based on asymmetric region Weber local descriptor(ARWLD)and block similarity weighting,which can reduce the interference of occlusion area to facial expression recognition and the impacts of the final discriminants in unconstrained environment.In the feature description,compared with the traditional WLD,the AR-WLD extends the original square neighborhood into an asymmetric neighborhood,and enhances the feature analysis in a multiscale.In order to distinguish the contribution of different facial regions to expression recognition,the non-overlapping expression regions are classified in classification discrimination.Information entropy is introduced to measure the uncertain information contained in different subblocks,and the weight of similarity distance is defined according to the information amount.The facial expression discrimination is achieved by the block similarity weighted summation.The experimental results on the databases of JAFFE and CK show that the AR-WLD can effectively improve the classification performance and robustness of the WLD when the expression image is partially occluded,and the classification algorithm based on block similarity weighting can further reduce the interference of the occlusion area to facial expression recognition.
引文
[1]Li Y Q,Li Y J,Li H B,et al.Fusion of global and local various feature for facial expression recognition[J].Acta Optica Sinica,2014,34(5):0515001.李雅倩,李颖杰,李海滨,等.融合全局与局部多样性特征的人脸表情识别[J].光学学报,2014,34(5):0515001.
[2]Happy S L,Routray A.Automatic facial expression recognition using features of salient facial patches[J].IEEE Transactions on Affective Computing,2015,6(1):1-12.
[3]Gu W F,Xiang C,Venkatesh Y V,et al.Facial expression recognition using radial encoding of local Gabor features and classifier synthesis[J].Pattern Recognition,2012,45(1):80-91.
[4]Hu M,Jiang H,Wang X H,et al.A hierarchical classification method of expressions based on geometric and texture features[J].Acta Electronica Sinica,2017,45(1):164-172.胡敏,江河,王晓华,等.基于几何和纹理特征的表情层级分类方法[J].电子学报,2017,45(1):164-172.
[5]Wang X H,Li R J,Hu M,et al.Occluded facial expression recognition based on the fusion of local features[J].Journal of Image and Graphics,2016,21(11):1473-1482.王晓华,李瑞静,胡敏,等.融合局部特征的面部遮挡表情识别[J].中国图象图形学报,2016,21(11):1473-1482.
[6]Saito Y,Kenmochi Y,Kotani K.Estimation of eyeglassless facial images using principal component analysis[C]//Conference on Image Processing,1999,4:197-201.
[7]Park J S,Oh Y H,Ahn S C,et al.Glasses removal from facial image using recursive error compensation[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2005,27(5):805-811.
[8]Chen Z,Hou Y Y,Xu J C.Sign determination and error analysis of phase extraction based on principle component analysis[J].Chinese Journal of Lasers,2016,43(1):0108002.陈曌,侯园园,徐建程.主元分析相位提取算法的符号确定及误差分析[J].中国激光,2016,43(1):0108002.
[9]Ghiasi G,Fowlkes C C.Occlusion coherence:localizing occluded faces with a hierarchical deformable part model[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2014:1899-1906.
[10]Zhang L G,Tjondronegoro D,Chandran V.Random Gabor based templates for facial expression recognition in images with facial occlusion[J].Neurocomputing,2014,145:451-464.
[11]Dapogny A,Bailly K,Dubuisson S.Confidenceweighted local expression predictions for occlusion handling in expression recognition and action unit detection[J].International Journal of Computer Vision,2017:1-17.
[12]Zhi R,Flierl M,Ruan Q,et al.Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition[J].IEEETransactions on Systems,Man&Cybernetics,Part B:Cybernetics,2011,41(1):38-52.
[13]Liu S S,Zhang Y,Liu K P.Facial expression recognition under partial occlusion based on Weber local descriptor histogram and decision fusion[C]//Proceedings of the 33rd Chinese Control Conference,2014:4664-4668.
[14]Zhao G P,Shen Y P,Wang J Y.Adaptive feature fusion object tracking based on circulant structure with Kernel[J].Acta Optica Sinica,2017,54(8):0815001.赵高鹏,沈玉鹏,王建宇.基于核循环结构的自适应特征融合目标跟踪[J].光学学报,2017,37(8):0815001.
[15]Tong Y,Chen R,Cheng Y.Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle[J].Optik-International Journal for Light and Electron Optics,2014,125(16):4186-4189.
[16]Chen J,Shan S,He C,et al.WLD:a robust local image descriptor[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2010,32(9):1705-1720.
[17]Gong D,Li S,Xiang Y.Face recognition using the Weber local descriptor[C]//Proceedings of the first Asian Conference on Pattern Recognition,2011:589-592.
[18]Naika C L S,Das P K,Nair S B.Asymmetric region local binary pattern operator for person-dependent facial expression recognition[C]//Proceedings of2012 International Conference on Computing,Communication and Applications,2012:1-5.
[19]Hu M,Cheng Y H,Wang X H,et al.Facial expression recognition based on asymmetric region local gradient coding[J].Journal of Image and Graphics,2015,20(10):1313-1321.胡敏,程轶红,王晓华,等.基于非对称局部梯度编码的人脸表情识别[J].中国图象图形学报,2015,20(10):1313-1321.
[20]Xia J,Pei D,Wang Q H,et al.Face recognition based on local adaptive ternary derivative pattern coupled with Gabor feature[J].Laser&Optoelectronics Progress,2016,53(11):111004.夏军,裴东,王全州,等.融合Gabor特征的局部自适应三值微分模式的人脸识别[J].激光与光电子学进展,2016,53(11):111004.
[21]Tan X Y,Chen S C,Li J,et al.Learning non-metric partial similarity based on maximal margin criterion[C]//Proceedings of 2006IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2006:138-145.
[22]Lv Y D,Feng Z Y,Xu C.Facial expression recognition via deep learning[C]//Proceedings of International Conference on Smart Computing,2014:303-308.
[2]Happy S L,Routray A.Automatic facial expression recognition using features of salient facial patches[J].IEEE Transactions on Affective Computing,2015,6(1):1-12.
[3]Gu W F,Xiang C,Venkatesh Y V,et al.Facial expression recognition using radial encoding of local Gabor features and classifier synthesis[J].Pattern Recognition,2012,45(1):80-91.
[4]Hu M,Jiang H,Wang X H,et al.A hierarchical classification method of expressions based on geometric and texture features[J].Acta Electronica Sinica,2017,45(1):164-172.胡敏,江河,王晓华,等.基于几何和纹理特征的表情层级分类方法[J].电子学报,2017,45(1):164-172.
[5]Wang X H,Li R J,Hu M,et al.Occluded facial expression recognition based on the fusion of local features[J].Journal of Image and Graphics,2016,21(11):1473-1482.王晓华,李瑞静,胡敏,等.融合局部特征的面部遮挡表情识别[J].中国图象图形学报,2016,21(11):1473-1482.
[6]Saito Y,Kenmochi Y,Kotani K.Estimation of eyeglassless facial images using principal component analysis[C]//Conference on Image Processing,1999,4:197-201.
[7]Park J S,Oh Y H,Ahn S C,et al.Glasses removal from facial image using recursive error compensation[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2005,27(5):805-811.
[8]Chen Z,Hou Y Y,Xu J C.Sign determination and error analysis of phase extraction based on principle component analysis[J].Chinese Journal of Lasers,2016,43(1):0108002.陈曌,侯园园,徐建程.主元分析相位提取算法的符号确定及误差分析[J].中国激光,2016,43(1):0108002.
[9]Ghiasi G,Fowlkes C C.Occlusion coherence:localizing occluded faces with a hierarchical deformable part model[C]//Proceedings of IEEEConference on Computer Vision and Pattern Recognition,2014:1899-1906.
[10]Zhang L G,Tjondronegoro D,Chandran V.Random Gabor based templates for facial expression recognition in images with facial occlusion[J].Neurocomputing,2014,145:451-464.
[11]Dapogny A,Bailly K,Dubuisson S.Confidenceweighted local expression predictions for occlusion handling in expression recognition and action unit detection[J].International Journal of Computer Vision,2017:1-17.
[12]Zhi R,Flierl M,Ruan Q,et al.Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition[J].IEEETransactions on Systems,Man&Cybernetics,Part B:Cybernetics,2011,41(1):38-52.
[13]Liu S S,Zhang Y,Liu K P.Facial expression recognition under partial occlusion based on Weber local descriptor histogram and decision fusion[C]//Proceedings of the 33rd Chinese Control Conference,2014:4664-4668.
[14]Zhao G P,Shen Y P,Wang J Y.Adaptive feature fusion object tracking based on circulant structure with Kernel[J].Acta Optica Sinica,2017,54(8):0815001.赵高鹏,沈玉鹏,王建宇.基于核循环结构的自适应特征融合目标跟踪[J].光学学报,2017,37(8):0815001.
[15]Tong Y,Chen R,Cheng Y.Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle[J].Optik-International Journal for Light and Electron Optics,2014,125(16):4186-4189.
[16]Chen J,Shan S,He C,et al.WLD:a robust local image descriptor[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2010,32(9):1705-1720.
[17]Gong D,Li S,Xiang Y.Face recognition using the Weber local descriptor[C]//Proceedings of the first Asian Conference on Pattern Recognition,2011:589-592.
[18]Naika C L S,Das P K,Nair S B.Asymmetric region local binary pattern operator for person-dependent facial expression recognition[C]//Proceedings of2012 International Conference on Computing,Communication and Applications,2012:1-5.
[19]Hu M,Cheng Y H,Wang X H,et al.Facial expression recognition based on asymmetric region local gradient coding[J].Journal of Image and Graphics,2015,20(10):1313-1321.胡敏,程轶红,王晓华,等.基于非对称局部梯度编码的人脸表情识别[J].中国图象图形学报,2015,20(10):1313-1321.
[20]Xia J,Pei D,Wang Q H,et al.Face recognition based on local adaptive ternary derivative pattern coupled with Gabor feature[J].Laser&Optoelectronics Progress,2016,53(11):111004.夏军,裴东,王全州,等.融合Gabor特征的局部自适应三值微分模式的人脸识别[J].激光与光电子学进展,2016,53(11):111004.
[21]Tan X Y,Chen S C,Li J,et al.Learning non-metric partial similarity based on maximal margin criterion[C]//Proceedings of 2006IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2006:138-145.
[22]Lv Y D,Feng Z Y,Xu C.Facial expression recognition via deep learning[C]//Proceedings of International Conference on Smart Computing,2014:303-308.