Abstract
We propose a way to learn visual features that are compatible with previously computed ones even when they have different dimensions and are learned via different neural network architectures and loss functions. Compatible means that, if such features are used to compare images, then “new” features can be compared directly to “old” features, so they can be used interchangeably. This enables visual search systems to bypass computing new features for all previously seen images when updating the embedding models, a process known as backfilling. Backward compatibility is critical to quickly deploy new embedding mod-els that leverage ever-growing large-scale training datasets and improvements in deep learning architectures and training methods. We propose a framework to train embed-ding models, called backward-compatible training (BCT), as the first step towards backward-compatible representation learning. In experiments on learning embeddings for face recognition, models trained with BCT successfully achieve backward compatibility without sacrificing accuracy, thus enabling backfill-free model updates of visual embeddings.
-
IEEE Keywords
- Training ,
- Visualization ,
- Task analysis ,
- Feature extraction ,
- Computational modeling ,
- Measurement ,
- Computer architecture
Introduction
Visual classification in an “open universe” setting is of-ten accomplished by mapping each image onto a vector space using a function (“model”) implemented by a deep neural network (DNN). The output of such a function in response to an image is often called its “embedding” [8,31]. The dissimilarity between a pair of images can then be measured by some type of distance between their embedding vectors. A good embedding is expected to cluster images belonging to the same class in the embedding space.
As images of a new class become available, their em-bedding vectors are used to spawn a new cluster in the open universe, possibly modifying its metric to avoid crowding,in a form of “life-long learning.” This process is known as indexing. It is common in modern applications to have millions, in some cases billions, of images indexed into hundreds of thousands to millions of clusters. This collection of images is usually referred to as the gallery set. A common use for the indexed gallery set is to identify the closest clusters to one or a set of input images, a process known as visual searchorvisual retrieval. The set of input images for this task is known as the query set. Besides the gallery and the query set, there is usually a separate large repository of the image used for training the embedding model [33,30], called the embedding training set. As time goes by, the datasets grow and the quality of the embeddings improves with newly trained models[37, 34, 6, 35]. However, to harvest the benefits of newmodels, one has to use the new models tore-process all im-agesin the gallery set to generate their embedding and re-create the clusters, a process known as “backfilling” or “re-indexing.”∗In this paper, we aim to design a system that enables new models to be deployed without having to re-index existing image collections. We call such a systembackfill-free, the resulting embeddingbackward-compatible representation, and the enabling process backward-compatible training(BCT). We summarize our contributions as follows: 1) We formalize the problem of backward-compatible representation learning in the context of open-set classification or visual retrieval. The goal is to enable new models to be deployed without having to re-process the previously indexed gallery set. The core of this problem is backward compatibility, which requires a new embedding’s output to be usable for comparisons against the old embedding model without compromising recognition accuracy. 2) We pro-pose a novel backward compatible training (BCT) approach by adding an influence loss, which uses the learned classifier of the old embedding model in training the new em-bedding model. 3) We achieve backward-compatible representation learning with minimal loss of accuracy, enablingbackfill-free updates of the models. We empirically verify that BCT is robust against multiple changing factors in training the embedding models,e.g., neural network architectures, loss functions, and data growth. Finally, 4) we show that compatibility between multiple models can be attained via chain-like pairwise BCT training.
Discussion
We have presented a method for achieving backward-compatible representation learning, illustrated specific in-stances, and compared them with both baselines and paragons. Our approach has several limitations. The first is the accuracy gap of the new models trained with BCTrelative to the new model oblivious of previous constraints. Though the gap is reduced by slightly more sophisticated forms of BCT, there is still work wo to be done in characterizing and achieving the attainable accuracy limits.
About KSRA
The Kavian Scientific Research Association (KSRA) is a non-profit research organization to provide research / educational services in December 2013. The members of the community had formed a virtual group on the Viber social network. The core of the Kavian Scientific Association was formed with these members as founders. These individuals, led by Professor Siavosh Kaviani, decided to launch a scientific / research association with an emphasis on education.
KSRA research association, as a non-profit research firm, is committed to providing research services in the field of knowledge. The main beneficiaries of this association are public or private knowledge-based companies, students, researchers, researchers, professors, universities, and industrial and semi-industrial centers around the world.
Our main services Based on Education for all Spectrum people in the world. We want to make an integration between researches and educations. We believe education is the main right of Human beings. So our services should be concentrated on inclusive education.
The KSRA team partners with local under-served communities around the world to improve the access to and quality of knowledge based on education, amplify and augment learning programs where they exist, and create new opportunities for e-learning where traditional education systems are lacking or non-existent.
FULL Paper PDF file:
Towards Backward-Compatible Representation LearningBibliography
author
Year
2020
Title
Towards Backward-Compatible Representation Learning
Publish in
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 6367-6376,
Doi
10.1109/CVPR42600.2020.00640.
PDF reference and original file: Click here
Somayeh Nosrati was born in 1982 in Tehran. She holds a Master's degree in artificial intelligence from Khatam University of Tehran.
Professor Siavosh Kaviani was born in 1961 in Tehran. He had a professorship. He holds a Ph.D. in Software Engineering from the QL University of Software Development Methodology and an honorary Ph.D. from the University of Chelsea.
Nasim Gazerani was born in 1983 in Arak. She holds a Master's degree in Software Engineering from UM University of Malaysia.
