Merge pull request #26 from shreydan/vit-transfer-learning-od

Transfer learning sub section in the fine tuning notebook

notebooks/Unit 3 - Vision Transformers/Fine-tuning Vision Transformers for Object detection.ipynb

@@ -17,7 +17,9 @@
         "\twidth=\"850\"\n",
         "\theight=\"450\">\n",
         "</iframe>\n",
-        "```"
+        "```\n",
+        "\n",
+        "Also there is a small section if you are interested in Transfer learning instead of fine tuning only."
       ]
     },
     {
@@ -1526,6 +1528,55 @@
       "source": [
         "Well, that's not bad. We can improve the results if we fine-tune further. You can find this fine-tuned checkpoint [here](hf-vision/detr-resnet-50-dc5-harhat-finetuned). "
       ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## How about Transfer learning ?"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "In this notebook , we primarily discussed about the fine-tuning a certain model to our custom dataset. What if , we only want transfer learning? Actually that is easy peasy! In transfer learning , we have to keep the parameter values aka weights, of the pretrained model frozen. We just train the classifier layer (in some cases, one or two more layers). In this case before starting the training process, we can do the following, "
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "vscode": {
+          "languageId": "plaintext"
+        }
+      },
+      "outputs": [],
+      "source": [
+        "from transformers import AutoModelForObjectDetection\n",
+        "\n",
+        "id2label = {0:'head', 1:'helmet', 2:'person'}\n",
+        "label2id = {v: k for k, v in id2label.items()}\n",
+        "\n",
+        "\n",
+        "model = AutoModelForObjectDetection.from_pretrained(\n",
+        "    checkpoint,\n",
+        "    id2label=id2label,\n",
+        "    label2id=label2id,\n",
+        "    ignore_mismatched_sizes=True,\n",
+        ")\n",
+        "\n",
+        "for name,p in model.named_parameters():\n",
+        "    if not 'bbox_predictor' in name or not name.startswith('class_label'):\n",
+        "        p.requires_grad = False"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "That means, after loading the model , we freeze all of the layers except last 6 layers. Which are `bbox_predictor.layers` and `class_labels_classifier`."
+      ]
     }
   ],
   "metadata": {