What is Transfer Learning?

What is Transfer Learning?

During Transfer learning, the knowledge leveraged and rapid progress from a source task is used to improve the learning and development to a new target task. Read on for a deeper dive on the subject.

Transfer learning is a machine learning method where the application of knowledge obtained from a model used in one task, can be reused as a foundation point for another task.

Machine learning algorithms use historical data as their input to make predictions and produce new output values. They are typically designed to conduct isolated tasks. A source task is a task from which knowledge is transferred to a target task. A target task is when improved learning occurs due to the transfer of knowledge from a source task. 

During transfer learning, the knowledge leveraged and rapid progress from a source task is used to improve the learning and development to a new target task. The application of knowledge is using the source task’s attributes and characteristics, which will be applied and mapped onto the target task. 

However, if the transfer method results in a decrease in the performance of the new target task, it is called a negative transfer. One of the major challenges when working with transfer learning methods is being able to provide and ensure the positive transfer between related tasks, whilst avoiding the negative transfer between less related tasks. 

Before we dive into the methodology behind transfer learning, it is good to know the different forms of transfer learning. We will go through three different types of transfer learning scenarios, based on relationships between the source task and target task. Below is an overview of the different types of transfer learning:

  Inductive Transfer Learning: In this type of transfer learning, the source and target task are the same, however, they are still different from one another. The model will use inductive biases from the source task to help improve the performance of the target task. The source task may or may not contain labeled data, further leading onto the model using multitask learning and self-taught learning. 

Unsupervised Transfer Learning: I assume you know what unsupervised learning is, however, if you don’t, it is when an algorithm is subjected to being able to identify patterns in datasets that have not been labeled or classified. In this case, the source and target are similar, however, the task is different, where both data is unlabelled in both source and target. Techniques such as dimensionality reduction and clustering are well known in unsupervised learning. 

Transductive Transfer Learning: In this last type of transfer learning, the source and target tasks share similarities, however, the domains are different. The source domain contains a lot of labeled data, whereas there is an absence of labeled data in the target domain, further leading onto the model using domain adaptation. 

  Fine-tuning is an optional step in transfer learning and is primarily incorporated to improve the performance of the model. The difference between Transfer learning and Fine-tuning is all in the name.

Transfer learning is built on adopting features learned from one task and “transferring” the leveraged knowledge onto a new task. Transfer learning is usually used on tasks where the dataset is too small, to train a full-scale model from scratch. Fine-tuning is built on making “fine” adjustments to a process in order to obtain the desired output to further improve performance.

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

How tech savvy does the CEO need to be?

4 Mar, 2017

As digital transformation efforts take hold in many enterprises, CEOs must step up and provide a vision for how the …

Read more

The State of Information Visualization, 2017

27 Jan, 2017

Between Surprise Maps, Dear Data, and election graphics, 2016 was an exciting year. There have also been some gaps, however, …

Read more

Data Product Canvas — A practical framework for building high-performance data products

27 Aug, 2022

Do you know why most data products fail? It is because they are not always aligned with the real objectives …

Read more

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.