Modzy ModelOps AI Platform transfer learning image racecar

Transfer Learning & Retraining

In simple terms, transfer learning is a machine learning approach where a model that is already trained on a specific data set and developed for a specific task is reused as the starting point for training on a different data set for a different task. Transfer learning is a popular approach for model training, and pre-trained computer vision and natural language processing models are used commonly as a starting point for specific user applications. At Modzy, the secure ModelOps AI Platform, our models are designed to work out of the box for your specific applications, but we also provide the option for retraining via CPU, utilizing transfer learning and domain adaptation so that our models are even more tailored to your specific applications.

Why it Matters

The underlying assumption made for transfer learning and domain adaptation is that the source and target domains differ in terms of marginal data distributions, but that the labeled data sets for the two domains are the same. There are also cases where the marginal distributions of the source and target data sets are related, but the source and target tasks have different labeled data sets. Depending on the specific application, the transferred knowledge can be in the form of data instances, feature representations, or model parameters. In our specific solution, we focus on features learned when the model was trained on the source data set for the source task, then adapt those features to a new target data set and target task. As an example, if we have a YOLO-based object detection model for detecting buildings in a specific data set, the features learned by that model can be utilized on the target data set to detect buildings in a data set with a different pixel distribution; this is done by performing a limited re-training for a few of layers in the YOLO model.

Modzy Approach to Transfer Learning & Retraining

  • Limited time and computation power. Retraining of our models should take only a short time and require only limited computation resources, but should enhance performance on the user’s data set.
  • Retraining is done according to the science behind feature-based transfer learning and domain adaptation. As an example, a deep learning model used for re-training will have most of its layers frozen, and the re-training of the model will only affect the weights in a few layers so that the features learned previously are utilized more efficiently for the new application and data set.

What this means for you

Author, Entrepreneur and business leader. Machine Learning @ Modzy