Research Areas

Posted on by

Our current research activities focus on two areas:

1. Brain-like and Deep Computing 

To investigate the new-generation intelligent computing system by biologically simulating human vision system and developing brain-like computation models. In particular, we focus on the following topics:

  • Neural inversion computing
  • Deep learning for video analysis
  • Biologically simulating for human vision system

1.1 Neural inversion computing


  • Building deep neural networks to reveal sensing and processing mechanisms of thehuman visual system (e.g., encoding)
  • Proposing brain-inspired novel visual sensing models and efficient spiking neural network models

Related Papers:

  1. Zhaofei Yu, Jian K. Liu*, Shanshan Jia, Yichen Zhang, Yajing Zheng, Yonghong Tian, Tiejun Huang, Towards the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes, Engineering, Volume 6, Issue 4, April 2020, 449-461.
  2. Yichen Zhang, Shanshan Jia, Yajing Zheng, Zhaofei Yu*, Yonghong Tian, Siwei Ma, Tiejun Huang, Jian K. Liu*, Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks, Neural Networks, Volume 125, May 2020, Pages 19-30.
  3. Qi Yan, Yajing Zheng, Shanshan Jia, Yichen Zhang, Zhaofei Yu*, Feng Chen, Yonghong Tian, Tiejun Huang, Jian K. Liu*, Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Network, IEEE Transactions on Cybernetics. 10.1109/TCYB.2020.2972983
  4. Yajing Zheng, Shanshan Jia, Zhaofei Yu*, Tiejun Huang, Jian K. Liu, Yonghong Tian*, Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation, Neural Networks, Volume 126, June 2020, Pages 42-51.

1.2 Deep learning for video analysis

Related Papers:

  1. Zhengying Chen, Yonghong Tian*, Wei Zeng and Tiejun Huang, Detecting Abnormal Behaviors in Surveillance Videos Based on Fuzzy Clustering and Multiple Auto-Encoders, Proc. Int’l Conf. Multimedia and Expo (ICME 2015), Torino, Italy.
  2. Yemin Shi, Wei Zeng, Tiejun Huang, Yaowei Wang∗, Learning Deep Trajectory Descriptor for Action Recognition in Videos using Deep Neural Networks, Proc. Int’l Conf. Multimedia and Expo (ICME 2015), Torino, Italy.
  3. Jilong Zheng, Yaowei Wang, Wei Zeng, and Yonghong Tian, CNN Based Vehicle Counting with Virtual Coil in Traffic Surveillance Video, Proc. IEEE Int’l Conf. Multimedia Big Data (BigMM 2015), Apr 2015, Beijing, China, 280-281.

1.3 Biological simulating for human vision system

  • Modeling the neurons and circuits in the retina and primary visual cortex of primate (Macaque monkey), via detecting the response/output of the retinal ganglion cells and neurons in the shallow layers of V1 to the visual stimulus pattern;
  • Developing software emulating primate retina, LGN and V1, to implement its coding functionality as accurate as possible.

spikingcoding

2. Multimedia Big Data

To address the technological challenges introduced by multimedia big data, including compression, storage, transmission, analysis, recognition, and security. In particular, we focus on the following topics:

  • Background-based Surveillance Video Coding/Transcoding
  • Machine learning for multimedia content analysis
  • Multi-camera cooperated surveillance video analysis
  • Large-scale content-based copy detection
  • Social multimedia computing

2.1. Ultra-Efficient Surveillance Video Coding/Transcoding

With the exponentially increasing deployments of the high-definition surveillance cameras, one major challenge for a real-time video surveillance system is how to effectively reduce the bandwidth and storage costs. To address this problem this study is devoted to develop a high-efficiency and low-complexity video codec suitable for surveillance videos.

background-based-coding

Related Papers:

  1. Xianguo Zhang, Yonghong Tian*, Tiejun Huang, Siwei Dong, Wen Gao, Optimizing the Hierarchical Prediction and Coding in HEVC for Surveillance and Conference Videos with Background Modeling, IEEE Transactions on Image Processing, 23(10), Oct. 2014. 4511-4526. DOI: 1109/TIP.2014.2352036
  2. Xianguo Zhang, Tiejun Huang*, Yonghong Tian*, Wen Gao, Background-Modeling Based Adaptive Prediction for Surveillance Video Coding, IEEE Transactions on Image Processing, 23(2), Feb 2014, 769-784. DOI: 1109/TIP.2013.2294549.
  3. Wen Gao, Yonghong Tian*, Tiejun Huang, Siwei Ma, Xianguo Zhang, IEEE 1857 Standard Empowering Smart Video Surveillance Systems, IEEE Intelligent Systems, 29(5), Sep.-Oct. 2014, 30-39. DOI: 1109/MIS.2013.101.
  4. Tiejun Huang, Siwei Dong, Yonghong Tian*, Representing Visual Objects in HEVC Coding Loop, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Volume 4, Issue 1, March 2014, 5-16. DOI: 10.1109/JETCAS.2014.2298274.
  5. Xianguo Zhang, Tiejun Huang*, Yonghong Tian*, Mingchao Geng, Siwei Ma, Wen Gao, Fast and Efficient Transcoding Based on Low-complexity Background Modeling and Adaptive Block Classification, IEEE Transactions on Multimedia, 15(8), Dec 2013, 1769-1785.
  6. Tiejun Huang, Yonghong Tian*, Wen Gao, IEEE 1857: Boosting Video Applications in CPSS, IEEE Intelligent Systems, 28(5), 24-27, Sept.-Oct. 2013.
  7. Long Zhao, Yonghong Tian*, Tiejun Huang, Background-Foreground Division based Search for Motion Estimation in Surveillance Video Coding, Proc. 2014 IEEE Int’l Conf. Multimedia and Expo, Chengdu, China, 2014.
  8. Peiyin Xing, Yonghong Tian*, Tiejun Huang, Wen Gao, Surveillance Video Coding with Quadtree Partition Based ROI Extraction, Proc. 30th Picture Coding Symposium, Dec 8-11, 2013, San Jose, California, 1-4.
  9. Peiyin Xing, Yonghong Tian*, Xianguo Zhang, Yaowei Wang, Tiejun Huang, A Coding Unit Classification Based AVC-to-HEVC Transcoding with Background Modeling for Surveillance Videos, Proc. 2013 IEEE Int’l Conf. Visual Communication and Image Processing, Kuching, Malaysia, Nov 2013.
  10. Xianguo Zhang, Tiejun Huang, Yonghong Tian, Wen Gao, Overview of the IEEE 1857 Surveillance Groups, Proc. 2013 IEEE Int’l Conf. Image Processing, Melbourne, Australia, 2013, 1505-1509.
  11. Xianguo Zhang, Tiejun Huang, Yonghong Tian, Wen Gao, Hierarchical-and-Adaptive Bit-allocation with Selective Background Prediction for High Efficiency Video Coding (HEVC), Proc. 2013 Data Compression Conference, 535.
  12. Shumin Han, Xianguo Zhang, Yonghong Tian, Tiejun Huang, An Efficient Background Reconstruction Based Coding Method for Surveillance Videos Captured By Moving Camera, Proc. 2012 IEEE Ninth Int’l Conf. Advanced Video and Signal-Based Surveillance, Beijing, China, Sep 18 2012, 160-165.（EI20124515644282）
  13. Mingchao Geng, Xianguo Zhang, Yonghong Tian*, Luhong Liang, Tiejun Huang, A Fast and Performance-Maintained Transcoding Method Based on Background Modeling for Surveillance Video, Proc. 2012 IEEE Int’l Conf. Multimedia and Expo, pp. 61-67, Melbourne, Australia, Jul 2012.（EI20124515636441）

2.2 Machine learning for multimedia content analysis

Machine learning models and algorithms are widely recognized as “the engine” in most pattern recognition and multimedia content analysis technologies. This research mainly focuses on the typical learning problems in multimedia content analysis, investigates the common statistical machine learning models and methods, consequently providing a theoretical foundation for multimedia intelligent analysis and retrieval.

machine_learning

Related Papers:

  1. Jingjing Yang, Yonghong Tian*, Lingyu Duan, Tiejun Huang, Wen Gao. Group-Sensitive Multiple Kernel Learning for Object Recognition, IEEE Transactions on Image Processing, 21(5), May 2012, 2838-2852.
  2. Yuanning Li, Yonghong Tian*, Lingyu Duan, Jingjing Yang, Tiejun Huang, Wen Gao. Sequence Multi-Labeling: A Unified Video Annotation Scheme with Spatial and Temporal Context. IEEE Transactions on Multimedia, 12(8), Dec. 2010, 814-828.
  3. Jingjing Yang, Yuanning Li, Yonghong Tian*, Lingyu Duan, Wen Gao. Per-Sample Multiple Kernel Approach for Visual Concept Learning. EURASIP Journal on Image and Video Processing, Vol 2010, Article ID 461450, 13 pages.
  4. Yonghong Tian, Qiang Yang, Tiejun Huang, Charles X. Ling and Wen Gao, Learning contextual dependency network models for link-based classification. IEEE Transactions on Knowledge and Data Engineering, 18(11), Nov 2006, 1482-1496.
  5. Yonghong Tian, Tiejun Huang, Wen Gao. Latent Linkage Semantic Kernels for Collective Classification of Link Data. Journal of Intelligent Information Systems, 26(3), May 2006, 269-301.
  6. Yonghong Tian. Context-Based Statistical Relational Learning. AI Communications, 19(3), Sep. 2006, 291-293.
  7. Jingjing Yang, Yuanning Li, Yonghong Tian, Ningyu Duan, Wen Gao. Group-Sensitive Multiple Kernel Learning for Object Categorization. Proc. 12th IEEE Int’l Conf. Computer Vision, Kyoto, Japan, 2009, 436 – 443. （EI20102312998138）
  8. Jingjing Yang, Yuanning Li, Yonghong Tian, Ningyu Duan, Wen Gao. Multiple Kernel Active Learning for Image Classification. Proc. IEEE Int’l Conf. Multimedia and Expo, Hilton Cancun, Cancun, Mexico, 2009, 550-553.（EI20094712492019）
  9. Jingjing Yang, Yuanning Li, Yonghong Tian, Lingyu Duan, Wen Gao. A New Multiple Kernel Approach for Visual Concept Learning, Proc. 15th Int’l Multimedia Modeling Conf., MMM 2009, LNCS 5371, Sophia-Antipolis, France, 2009, 250-262.（EI20090611898947）

2.3 Multi-camera cooperated surveillance video analysis

Video surveillance systems have become one of most important infrastructures for social security and emergency management applications. This study puts its main focus on the challenging research issues and key technologies about multi-camera cooperated object detection, tracking, and event analysis on large-scale surveillance video data. Its long-term objective is to provide key technologies and solutions for the next-generation intelligent video surveillance systems and applications.

multi_camera

Related Papers:

  1. Peixi Peng, Yonghong Tian*, Yaowei Wang, Jia Li, Tiejun Huang, Robust Multiple Cameras Pedestrian Detection with Multi-view Bayesian Network, Pattern Recognition, Accepted, 9 Dec 2014.
  2. Yonghong Tian, Yaowei Wang, Zhipeng Hu, Tiejun Huang, Selective Eigenbackground for Background Modeling and Subtraction in Crowded Scenes, IEEE Transactions on Circuits and Systems for Video Technology. 23(11), 2013, 1849-1864.
  3. Teng Xu, Tiejun Huang,Yonghong Tian, Survey on Pedestrian Detection Technology for On-board Vision Systems, Journal of Image and Graphics, 18(4), 2013, 359-367. [In Chinese]
  4. Lan Wei, Yonghong Tian*, Yaowei Wang, Tiejun Huang, Swiss-System based Cascade Ranking for Gait-based Person Re-identification, Proc. AAAI 2015, Jan 26, 2015.
  5. Lan Wei, Yonghong Tian*, Yaowei Wang, Tiejun Huang, Multi-view Gait Recognition with Incomplete Training Data, Proc. 2014 IEEE Int’l Conf. Multimedia and Expo, Chengdu, China, 2014.
  6. Jiaqiu Chen, Yaowei Wang, Yonghong Tian, Tiejun Huang, Wavelet Based Smoke Detection Method with RGB Constrast-Image and Shape Constraint, Proc. 2013 IEEE Int’l Conf. Visual Communication and Image Processing, Kuching, Malaysia, Nov 2013.
  7. Chaoran Gu, Luantian Mou, Yonghong Tian*, Tiejun Huang, MPLBoost-based Mixture Model for Effective Human Detection with Deformable Part Model, Proc. 2013 IEEE Int’l Conf. Multimedia and Expo, San Jose, CA, USA, 2013, 1-6.
  8. Xiaoyu Fang, Yonghong Tian*, Yaowei Wang, Chi Su, Teng Xu, Ziwei Xia, Peixi Peng, Wen Gao, Pair-wise Event Detection using Cubic Features and Sequence Discriminant Learning, Proc. 2013 IEEE Int’l Conf. Multimedia and Expo, San Jose, CA, USA, 2013, 1-6.
  9. Xiaoyu Fang, Ziwei Xia, Chi Su, Teng Xu, Yonghong Tian*, Yaowei Wang, Tiejun Huang, A System based on Sequence Learning for Event Detection in Surveillance Video, Proc. 2013 IEEE Int’l Conf. Image Processing, Melbourne, Australia, 2013, 3587-3591.
  10. Peixi Peng, Yonghong Tian*, Yaowei Wang, Tiejun Huang, Multi-camera Pedestrian Detection with a Multi-view Bayesian Network Model, Proc. 2012 British Machine Vision Conf., paper 69, pp. 1-12, Guildford, UK, 2012.
  11. Teng Xu, Peixi Peng, Xiaoyu Fang, Chi Su, Yaowei Wang*, Yonghong Tian*, Wei Zeng, Tiejun Huang, Single and Multiple View Detection, Tracking and Video Analysis in Crowded Environments, Proc. 2012 IEEE Ninth Int’l Conf. Advanced Video and Signal-Based Surveillance, pp. 494-499, Beijing, China, Sep 2012.（EI20124515644337）

2.4 Large-scale content-based copy detection

The Internet is revolutionizing multimedia content distribution, offering users unprecedented opportunities to share digital images, audio, and video but also presenting major challenges for digital rights management (DRM) challenges. Base on the audio-visual perception theory and mechanism, this study is trying to investigate the theory and methodologies of robust mediaprinting technology which can be used to efficiently identify media objects with same or similar content. It is deemed that this technology will play an important role in the new-generation multimedia security systems.

mediaprinting (From: T.J. Huang, Y.H. Tian, W. Gao, J. Lu, Mediaprinting: identifying multimedia content for digital rights management, Computer, 43(12), 2010, 28-35.)

Related Papers:

  1. Yonghong Tian, Mengren Qian, Tiejun Huang, TASC: A Transformation-Aware Soft Cascading Approach for Multimodal Video Copy Detection, ACM Transactions on Information Systems, Accepted, 21 Oct 2014.
  2. Luntian Mou, Tiejun Huang, Yonghong Tian, Menglin Jiang, Wen Gao, Content-Based Copy Detection through Multimodal Feature Representation and Temporal Pyramid Matching, ACM Trans. Multimedia Comput. Commun. Appl., 10(1), Article 5 (December 2013), 20
  3. Yonghong Tian, Tiejun Huang, Menglin Jiang, and Wen Gao, Video Copy Detection and Localization with a Scalable Cascading Framework, IEEE Multimedia, 20(3), Sep. 2013, 72-86.
  4. Yonghong Tian, Tiejun Huang, Wen Gao, Multimodal Video Copy Detection using Multi-Detectors Fusion, IEEE COMSOC MMTC E-Letter, 7(5), September 2012, 3-6.
  5. Tiejun Huang, Yonghong Tian*, Wen Gao, Jian Lu. Mediaprinting: Identifying Multimedia Content for Digital Rights Management. Computer, 43(12), Dec. 2010, 28-35.
  6. Mengren Qian, Luntian Mou, Jia Li, and Yonghong Tian*. Video Picture-in-Picture Detection using Spatio-Temporal Slicing. Proc. ICME’2014 Workshop on Emerg. Multimedia Sys. and Appl., Chengdu, China, 2014.
  7. Menglin Jiang, Yonghong Tian*, Tiejun Huang, Video Copy Detection Using a Soft Cascade of Multimodal Features, Proc. 2012 IEEE Int’l Conf. Multimedia and Expo, Melbourne, Australia, 374-379, 2012.（EI20124515636492）
  8. Luntian Mou, Xilin Chen, Yonghong Tian, Tiejun Huang. Robust and Disrimnative Image Authentication Based on Standard Model Feature, Proc. 2012 IEEE Int’l Symposium on Circuit & System, Seoul, Korea, 1131-1134, 2012.
  9. Yonghong Tian, Menglin Jiang, Luntian Mou, Xiaoyu Fang, Tiejun Huang. A Multimodal Video Copy Detection Approach with Sequential Pyramid Matching, Proc. IEEE Int’l Conf. Image Processing (ICIP 2011), Brussels, Belgium, Sep. 2011, 3690-3693. （EI20120514730536）

2.5 Social multimedia computing

Social multimedia and interactive video are becoming two of the most attractive technologies in new media applications. This research focuses on the fundamental theory, models, and methodologies in various social multimedia applications.

Social_multimedia_computing

Related Papers:

  1. Yonghong Tian, Jaideep Srivastava, Tiejun Huang, and Noshir Contractor. Social Multimedia Computing. Computer, 43(8), Aug. 2010, 27-36. (WOS:000280949000008)(Cover Feature)
  2. Wen Gao, Yonghong Tian*, Tiejun Huang, Qiang. Vlogging: A Survey of Video Blogging Technology on the Web. ACM Computing Surveys, 42(4), Jun. 2010, article 15, 57 pages.
  3. Yonghong Tian, Shui Yu, Chin-Yung Lin, Wen Gao, Wanlei Zhou, Special Issue on Social Multimedia Computing: Challenges, Techniques, and Applications: Guest Editorial, Journal of Multimedia, 9(1), 2014, 1-3.
  4. Shui Yu, Yonghong Tian*, Song Guo, Dapeng Oliver Wu, Can We Beat DDoS Attacks in Clouds? IEEE Transactions on Parallel and Distributed Systems, 25(9), Sep. 2013, 2245-2254. DOI: 10.1109/TPDS.2013.181.
  5. Zhongfei Zhang, Zhengyou Zhang, Ramesh Jain, Yueting Zhuang, Noshir CONTRACTOR, Alexander G. HAUPTMANN, Alejandro (Alex) JAIMES, Wanqing LI, Alexander C. LOUI, Tao MEI, Nicu SEBE, Yonghong Tian, Vincent S. TSENG, Qing WANG, Changsheng XU, Huimin YU, Shiwen YU, Societally connected multimedia across cultures, Journal of Zhejiang University SCIENCE C, 13(12), 2012, 875-880.(WOS:000312185500001)
  6. Amogh Mahapatra, Xin Wan, Yonghong Tian, and Jaideep Srivastava. Augmenting Image Processing with Social Tag Mining for Landmark Recognition. Proc. 17th Int’l Multimedia Modeling Conf., MMM 2011, Jan 5-6, 2011, Taiwan, China, 273-283.（EI20110413622029）