learn_pytorch_14_pytorch图像数据处理之任务模型models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 pytorch解决不同图像处理任务的包 torchvision.models\n",
    "官网：https://pytorch.org/docs/stable/torchvision/models.html#torchvision-models  \n",
    "\n",
    "任务包括：  \n",
    "\n",
    "图像分类： image classification  \n",
    "语义分割：semantic segmentation  \n",
    "目标检测：object detection   \n",
    "实例分割：instance segmentation  \n",
    "人关键点检测：person keypoint detection  \n",
    "视频分类：video classification  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2 torchvision.models包的源码查看和结果解析\n",
    "该包的\\__init\\__\\.py:  \n",
    "\n",
    "from .alexnet import *       &emsp;&emsp; &emsp;  导入torchvision.models包alexnet模块中的所有   \n",
    "from .resnet import *        &emsp;&emsp; &emsp;导入torchvision.models包resnet模块中的所有  \n",
    "from .vgg import *          &emsp;&emsp; &emsp;导入 torchvision.models包vgg模块中的所有  \n",
    "from .squeezenet import *     &emsp;&emsp; &emsp;导入 torchvision.models包squeezenet模块中的所有  \n",
    "from .inception import *     &emsp;&emsp; &emsp;导入torchvision.models包inception模块中的所有  \n",
    "from .densenet import *      &emsp;&emsp; &emsp;导入torchvision.models包densenet模块中的所有  \n",
    "from .googlenet import *     &emsp;&emsp; &emsp;导入torchvision.models包googlenet模块中的所有  \n",
    "from .mobilenet import *     &emsp;&emsp; &emsp;导入 torchvision.models包mobilenet 模块中的所有  \n",
    "from .mnasnet import *       &emsp;&emsp; &emsp;导入torchvision.models包mnasnet模块中的所有  \n",
    "from .shufflenetv2 import *   &emsp;&emsp; &emsp;导入torchvision.models包shufflenetv2模块中的所有  \n",
    "from . import segmentation    &emsp;&emsp; &emsp;导入torchvision.models包segmentation包  \n",
    "from . import detection      &emsp;&emsp; &emsp;导入torchvision.models包detection包   \n",
    "from . import video         &emsp;&emsp; &emsp;导入torchvision.models包video包   \n",
    "\n",
    "文件结构：\n",
    "![jupyter](./torch_models.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3 alexnet模块\n",
    "\\__all\\__\\= ['AlexNet', 'alexnet']\n",
    "\n",
    "使用alexnet架构：2种方法   \n",
    "alexnet = models.alexnet() 随机初始化权重  \n",
    "alexnet = models.alexnet(pretrained=True) 使用预训练权重imagenet数据集  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4 vgg模块 \n",
    "\\__all\\__= ['VGG', 'vgg11', 'vgg11_bn', 'vgg13', 'vgg13_bn', 'vgg16', 'vgg16_bn',  'vgg19_bn', 'vgg19']   \n",
    "使用8种vgg架构：每种2种方法   \n",
    "vgg11 = models.vgg11() 随机初始化权重  \n",
    "vgg11 = models.vgg11(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg11_bn = models.vgg11_bn() 随机初始化权重  \n",
    "vgg11_bn = models.vgg11_bn(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg13 = models.vgg13() 随机初始化权重  \n",
    "vgg13 = models.vgg13(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg13_bn = models.vgg13_bn() 随机初始化权重  \n",
    "vgg13_bn = models.vgg13_bn(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg16 = models.vgg16() 随机初始化权重  \n",
    "vgg16 = models.vgg16(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg16_bn = models.vgg16_bn() 随机初始化权重  \n",
    "vgg16_bn = models.vgg16_bn(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg19 = models.vgg19() 随机初始化权重  \n",
    "vgg19 = models.vgg19(pretrained=True) 使用预训练权重  \n",
    "\n",
    "vgg19_bn = models.vgg19_bn() 随机初始化权重  \n",
    "vgg19_bn = models.vgg19_bn(pretrained=True) 使用预训练权重  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5 resnet模块 \n",
    "\\__all\\__=['ResNet','resnet18','resnet34','resnet50','resnet101','resnet152','resnext50_32x4d','resnext101_32x8d','wide_resnet50_2', 'wide_resnet101_2']   \n",
    "使用9种resnet架构：每种2种方法   \n",
    "resnet18 = models.resnet18() 随机初始化权重  \n",
    "resnet18 = models.resnet18(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnet34 = models.resnet34() 随机初始化权重  \n",
    "resnet34 = models.resnet34(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnet50 = models.resnet50() 随机初始化权重  \n",
    "resnet50 = models.resnet50(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnet101 = models.resnet101() 随机初始化权重  \n",
    "resnet101 = models.resnet101(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnet152 = models.resnet152() 随机初始化权重  \n",
    "resnet152 = models.resnet152(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnet50_32x4d = models.resnet50_32x4d() 随机初始化权重  \n",
    "resnet50_32x4d = models.resnet50_32x4d(pretrained=True) 使用预训练权重  \n",
    "\n",
    "resnext101_32x8d = models.resnext101_32x8d() 随机初始化权重  \n",
    "resnext101_32x8d = models.resnext101_32x8d(pretrained=True) 使用预训练权重  \n",
    "\n",
    "wide_resnet50_2 = models.wide_resnet50_2() 随机初始化权重  \n",
    "wide_resnet50_2 = models.wide_resnet50_2(pretrained=True) 使用预训练权重  \n",
    "\n",
    "wide_resnet101_2 = models.wide_resnet101_2() 随机初始化权重  \n",
    "wide_resnet101_2 = models.wide_resnet101_2(pretrained=True) 使用预训练权重  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6 squeezenet模块 \n",
    "\\__all\\__=['SqueezeNet', 'squeezenet1_0', 'squeezenet1_1']   \n",
    "使用2种squeezenet架构：每种2种方法   \n",
    "squeezenet1_0 = models.squeezenet1_0() 随机初始化权重  \n",
    "squeezenet1_0 = models.squeezenet1_0(pretrained=True) 使用预训练权重  \n",
    "\n",
    "squeezenet1_1 = models.squeezenet1_1() 随机初始化权重  \n",
    "squeezenet1_1 = models.squeezenet1_1(pretrained=True) 使用预训练权重  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7 inception模块 \n",
    "\\__all\\__=['DenseNet', 'densenet121', 'densenet169', 'densenet201', 'densenet161']   \n",
    "使用4种inception架构：每种2种方法   \n",
    "\n",
    "densenet121 = models.densenet121() 随机初始化权重  \n",
    "densenet121 = models.densenet121(pretrained=True) 使用预训练权重  \n",
    "\n",
    "densenet169 = models.densenet169() 随机初始化权重  \n",
    "densenet169 = models.densenet169(pretrained=True) 使用预训练权重  \n",
    "\n",
    "densenet201 = models.densenet201() 随机初始化权重  \n",
    "densenet201 = models.densenet201(pretrained=True) 使用预训练权重  \n",
    "\n",
    "densenet161 = models.densenet161() 随机初始化权重  \n",
    "densenet161 = models.densenet161(pretrained=True) 使用预训练权重  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8 googlenet模块 \n",
    "\\__all\\__=['GoogLeNet', 'googlenet']   \n",
    "使用1种googlenet架构：每种2种方法   \n",
    "\n",
    "googlenet = models.googlenet() 随机初始化权重  \n",
    "googlenet = models.googlenet(pretrained=True) 使用预训练权重  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9 mobilenet模块 \n",
    "\\__all\\__=['MobileNetV2', 'mobilenet_v2']   \n",
    "使用1种mobilenet架构：每种2种方法   \n",
    "\n",
    "mobilenet_v2 = models.mobilenet_v2() 随机初始化权重  \n",
    "mobilenet_v2 = models.mobilenet_v2(pretrained=True) 使用预训练权重  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10 mnasnet模块 \n",
    "\\__all\\__=['MNASNet', 'mnasnet0_5', 'mnasnet0_75', 'mnasnet1_0', 'mnasnet1_3']   \n",
    "使用4种mnasnet架构：每种2种方法   \n",
    "\n",
    "mnasnet0_5 = models.mnasnet0_5() 随机初始化权重  \n",
    "mnasnet0_5 = models.mnasnet0_5(pretrained=True) 使用预训练权重  \n",
    "\n",
    "mnasnet0_75 = models.mnasnet0_75() 随机初始化权重  \n",
    "mnasnet0_75 = models.mnasnet0_75(pretrained=True) 使用预训练权重 \n",
    "\n",
    "mnasnet1_0 = models.mnasnet1_0() 随机初始化权重  \n",
    "mnasnet1_0 = models.mnasnet1_0(pretrained=True) 使用预训练权重 \n",
    "\n",
    "mnasnet1_3 = models.mnasnet1_3() 随机初始化权重  \n",
    "mnasnet1_3 = models.mnasnet1_3(pretrained=True) 使用预训练权重 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 11 shufflenetv2模块 \n",
    "\\__all\\__=['ShuffleNetV2', 'shufflenet_v2_x0_5', 'shufflenet_v2_x1_0','shufflenet_v2_x1_5', 'shufflenet_v2_x2_0']   \n",
    "使用4种mnasnet架构：每种2种方法   \n",
    "\n",
    "shufflenet_v2_x0_5 = models.shufflenet_v2_x0_5() 随机初始化权重  \n",
    "shufflenet_v2_x0_5 = models.shufflenet_v2_x0_5(pretrained=True) 使用预训练权重  \n",
    "\n",
    "shufflenet_v2_x1_0 = models.shufflenet_v2_x1_0() 随机初始化权重  \n",
    "shufflenet_v2_x1_0 = models.shufflenet_v2_x1_0(pretrained=True) 使用预训练权重 \n",
    "\n",
    "shufflenet_v2_x1_5 = models.shufflenet_v2_x1_5() 随机初始化权重  \n",
    "shufflenet_v2_x1_5 = models.shufflenet_v2_x1_5(pretrained=True) 使用预训练权重 \n",
    "\n",
    "shufflenet_v2_x2_0 = models.shufflenet_v2_x2_0() 随机初始化权重  \n",
    "shufflenet_v2_x2_0 = models.shufflenet_v2_x2_0(pretrained=True) 使用预训练权重 \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12 segmentation包\n",
    "\\__init\\__\\.py : \n",
    "from .segmentation import *   \n",
    "from .fcn import *  \n",
    "from .deeplabv3 import *  \n",
    "\n",
    "segmentation包的segmentation模块  \n",
    "\\__all\\__=['fcn_resnet50', 'fcn_resnet101', 'deeplabv3_resnet50', 'deeplabv3_resnet101'] \n",
    "使用4种mnasnet架构：每种2种方法   \n",
    "\n",
    "fcn_resnet50 = models.segmentation.fcn_resnet50() 随机初始化权重  \n",
    "segmentation.fcn_resnet50 =models.segmentation.fcn_resnet50(pretrained=True) 使用预训练权重COCO数据集 \n",
    "\n",
    "fcn_resnet101 = models.segmentation.fcn_resnet101() 随机初始化权重  \n",
    "fcn_resnet101 = models.segmentation.fcn_resnet101(pretrained=True) 使用预训练权重 \n",
    "\n",
    "deeplabv3_resnet50 = models.segmentation.deeplabv3_resnet50() 随机初始化权重  \n",
    "deeplabv3_resnet50 = models.segmentation.deeplabv3_resnet50(pretrained=True) 使用预训练权重 \n",
    "\n",
    "deeplabv3_resnet101 = models.segmentation.deeplabv3_resnet101() 随机初始化权重  \n",
    "deeplabv3_resnet101 = models.segmentation.deeplabv3_resnet101(pretrained=True) 使用预训练权重 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13 detection包\n",
    "\\__init\\__\\.py:  \n",
    "from .faster_rcnn import *  \n",
    "from .mask_rcnn import *  \n",
    "from .keypoint_rcnn import *  \n",
    "\n",
    "detection包的faster_rcnn模块  \n",
    "\\__all\\__=[\"FasterRCNN\", \"fasterrcnn_resnet50_fpn\"]   \n",
    "fasterrcnn_resnet50_fpn = models.detection.fasterrcnn_resnet50_fpn() 随机初始化权重    \n",
    "fasterrcnn_resnet50_fpn = models.detection.fasterrcnn_resnet50_fpn(pretrained=True) 使用预训练权重COCO数据集 \n",
    "\n",
    "detection包的mask_rcnn模块  \n",
    "\\__all\\__=[\"MaskRCNN\", \"maskrcnn_resnet50_fpn\"]   \n",
    "maskrcnn_resnet50_fpn = models.detection.maskrcnn_resnet50_fpn() 随机初始化权重    \n",
    "maskrcnn_resnet50_fpn = models.detection.maskrcnn_resnet50_fpn(pretrained=True) 使用预训练权重COCO数据集   \n",
    "\n",
    "detection包的keypoint_rcnn模块  \n",
    "\\__all\\__=[\"KeypointRCNN\", \"keypointrcnn_resnet50_fpn\"]  \n",
    "keypointrcnn_resnet50_fpn = models.detection.keypointrcnn_resnet50_fpn() 随机初始化权重    \n",
    "keypointrcnn_resnet50_fpn = models.detection.keypointrcnn_resnet50_fpn(pretrained=True) 使用预训练权重COCO数据集 \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14 video包\n",
    "\\__init\\__\\.py:  \n",
    "from .resnet import *  \n",
    "\n",
    "video包的resnet模块  \n",
    "\\__all\\__=['r3d_18', 'mc3_18', 'r2plus1d_18']   \n",
    "r3d_18 = models.video.r3d_18() 随机初始化权重    \n",
    "r3d_18 = models.video.r3d_18(pretrained=True) 使用预训练权重Kinetics-400数据集 \n",
    "\n",
    "mc3_18 = models.video.mc3_18() 随机初始化权重    \n",
    "mc3_18 = models.video.mc3_18(pretrained=True) 使用预训练权重Kinetics-400数据集 \n",
    "\n",
    "r2plus1d_18 = models.video.r2plus1d_18() 随机初始化权重    \n",
    "r2plus1d_18 = models.video.r2plus1d_18(pretrained=True) 使用预训练权重Kinetics-400数据集 \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 15 查看网络的组成结构 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchvision import models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AlexNet(\n",
       "  (features): Sequential(\n",
       "    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))\n",
       "    (1): ReLU(inplace=True)\n",
       "    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n",
       "    (4): ReLU(inplace=True)\n",
       "    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (7): ReLU(inplace=True)\n",
       "    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (9): ReLU(inplace=True)\n",
       "    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (11): ReLU(inplace=True)\n",
       "    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))\n",
       "  (classifier): Sequential(\n",
       "    (0): Dropout(p=0.5, inplace=False)\n",
       "    (1): Linear(in_features=9216, out_features=4096, bias=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "    (3): Dropout(p=0.5, inplace=False)\n",
       "    (4): Linear(in_features=4096, out_features=4096, bias=True)\n",
       "    (5): ReLU(inplace=True)\n",
       "    (6): Linear(in_features=4096, out_features=1000, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.alexnet()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "VGG(\n",
       "  (features): Sequential(\n",
       "    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "    (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (5): ReLU(inplace=True)\n",
       "    (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (7): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (8): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (9): ReLU(inplace=True)\n",
       "    (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (11): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (12): ReLU(inplace=True)\n",
       "    (13): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (14): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (15): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (16): ReLU(inplace=True)\n",
       "    (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (19): ReLU(inplace=True)\n",
       "    (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (22): ReLU(inplace=True)\n",
       "    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (24): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (25): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (26): ReLU(inplace=True)\n",
       "    (27): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (28): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (29): ReLU(inplace=True)\n",
       "    (30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (31): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (32): ReLU(inplace=True)\n",
       "    (33): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "    (34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (35): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (36): ReLU(inplace=True)\n",
       "    (37): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (39): ReLU(inplace=True)\n",
       "    (40): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (41): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (42): ReLU(inplace=True)\n",
       "    (43): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  )\n",
       "  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))\n",
       "  (classifier): Sequential(\n",
       "    (0): Linear(in_features=25088, out_features=4096, bias=True)\n",
       "    (1): ReLU(inplace=True)\n",
       "    (2): Dropout(p=0.5, inplace=False)\n",
       "    (3): Linear(in_features=4096, out_features=4096, bias=True)\n",
       "    (4): ReLU(inplace=True)\n",
       "    (5): Dropout(p=0.5, inplace=False)\n",
       "    (6): Linear(in_features=4096, out_features=1000, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.vgg16_bn()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GoogLeNet(\n",
       "  (conv1): BasicConv2d(\n",
       "    (conv): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
       "    (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (maxpool1): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)\n",
       "  (conv2): BasicConv2d(\n",
       "    (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (conv3): BasicConv2d(\n",
       "    (conv): Conv2d(64, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (maxpool2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)\n",
       "  (inception3a): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(192, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(96, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(192, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception3b): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(128, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(32, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (maxpool3): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True)\n",
       "  (inception4a): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(480, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(96, 208, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(208, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(480, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(16, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(480, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception4b): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(512, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(112, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(112, 224, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(224, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(24, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(24, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception4c): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(256, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(24, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(24, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception4d): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(112, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(144, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(144, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(288, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(512, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception4e): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(528, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(256, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(528, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(320, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(528, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(32, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (maxpool4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=True)\n",
       "  (inception5a): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(832, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(256, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(832, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(320, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(832, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(32, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (inception5b): Inception(\n",
       "    (branch1): BasicConv2d(\n",
       "      (conv): Conv2d(832, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch2): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(832, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch3): Sequential(\n",
       "      (0): BasicConv2d(\n",
       "        (conv): Conv2d(832, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(48, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (branch4): Sequential(\n",
       "      (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True)\n",
       "      (1): BasicConv2d(\n",
       "        (conv): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (aux1): InceptionAux(\n",
       "    (conv): BasicConv2d(\n",
       "      (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (fc1): Linear(in_features=2048, out_features=1024, bias=True)\n",
       "    (fc2): Linear(in_features=1024, out_features=1000, bias=True)\n",
       "  )\n",
       "  (aux2): InceptionAux(\n",
       "    (conv): BasicConv2d(\n",
       "      (conv): Conv2d(528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (fc1): Linear(in_features=2048, out_features=1024, bias=True)\n",
       "    (fc2): Linear(in_features=1024, out_features=1000, bias=True)\n",
       "  )\n",
       "  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n",
       "  (dropout): Dropout(p=0.2, inplace=False)\n",
       "  (fc): Linear(in_features=1024, out_features=1000, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.googlenet()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Inception3(\n",
       "  (Conv2d_1a_3x3): BasicConv2d(\n",
       "    (conv): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), bias=False)\n",
       "    (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (Conv2d_2a_3x3): BasicConv2d(\n",
       "    (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (Conv2d_2b_3x3): BasicConv2d(\n",
       "    (conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (Conv2d_3b_1x1): BasicConv2d(\n",
       "    (conv): Conv2d(64, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(80, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (Conv2d_4a_3x3): BasicConv2d(\n",
       "    (conv): Conv2d(80, 192, kernel_size=(3, 3), stride=(1, 1), bias=False)\n",
       "    (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (Mixed_5b): InceptionA(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_1): BasicConv2d(\n",
       "      (conv): Conv2d(192, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_2): BasicConv2d(\n",
       "      (conv): Conv2d(48, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_5c): InceptionA(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_1): BasicConv2d(\n",
       "      (conv): Conv2d(256, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_2): BasicConv2d(\n",
       "      (conv): Conv2d(48, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_5d): InceptionA(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(288, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_1): BasicConv2d(\n",
       "      (conv): Conv2d(288, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch5x5_2): BasicConv2d(\n",
       "      (conv): Conv2d(48, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(288, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(288, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_6a): InceptionB(\n",
       "    (branch3x3): BasicConv2d(\n",
       "      (conv): Conv2d(288, 384, kernel_size=(3, 3), stride=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(288, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_6b): InceptionC(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_2): BasicConv2d(\n",
       "      (conv): Conv2d(128, 128, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_3): BasicConv2d(\n",
       "      (conv): Conv2d(128, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(128, 128, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(128, 128, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_4): BasicConv2d(\n",
       "      (conv): Conv2d(128, 128, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_5): BasicConv2d(\n",
       "      (conv): Conv2d(128, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_6c): InceptionC(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_2): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_3): BasicConv2d(\n",
       "      (conv): Conv2d(160, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_4): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_5): BasicConv2d(\n",
       "      (conv): Conv2d(160, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_6d): InceptionC(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_2): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_3): BasicConv2d(\n",
       "      (conv): Conv2d(160, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_4): BasicConv2d(\n",
       "      (conv): Conv2d(160, 160, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_5): BasicConv2d(\n",
       "      (conv): Conv2d(160, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_6e): InceptionC(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_2): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7_3): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_3): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_4): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7dbl_5): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (AuxLogits): InceptionAux(\n",
       "    (conv0): BasicConv2d(\n",
       "      (conv): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (conv1): BasicConv2d(\n",
       "      (conv): Conv2d(128, 768, kernel_size=(5, 5), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(768, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (fc): Linear(in_features=768, out_features=1000, bias=True)\n",
       "  )\n",
       "  (Mixed_7a): InceptionD(\n",
       "    (branch3x3_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_2): BasicConv2d(\n",
       "      (conv): Conv2d(192, 320, kernel_size=(3, 3), stride=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(320, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7x3_1): BasicConv2d(\n",
       "      (conv): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7x3_2): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(1, 7), stride=(1, 1), padding=(0, 3), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7x3_3): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(7, 1), stride=(1, 1), padding=(3, 0), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch7x7x3_4): BasicConv2d(\n",
       "      (conv): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_7b): InceptionE(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(1280, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(320, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_1): BasicConv2d(\n",
       "      (conv): Conv2d(1280, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_2a): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(1, 3), stride=(1, 1), padding=(0, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_2b): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(1280, 448, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(448, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(448, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3a): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(1, 3), stride=(1, 1), padding=(0, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3b): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(1280, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (Mixed_7c): InceptionE(\n",
       "    (branch1x1): BasicConv2d(\n",
       "      (conv): Conv2d(2048, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(320, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_1): BasicConv2d(\n",
       "      (conv): Conv2d(2048, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_2a): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(1, 3), stride=(1, 1), padding=(0, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3_2b): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_1): BasicConv2d(\n",
       "      (conv): Conv2d(2048, 448, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(448, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_2): BasicConv2d(\n",
       "      (conv): Conv2d(448, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3a): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(1, 3), stride=(1, 1), padding=(0, 1), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch3x3dbl_3b): BasicConv2d(\n",
       "      (conv): Conv2d(384, 384, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0), bias=False)\n",
       "      (bn): BatchNorm2d(384, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (branch_pool): BasicConv2d(\n",
       "      (conv): Conv2d(2048, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (fc): Linear(in_features=2048, out_features=1000, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.inception_v3()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ResNet(\n",
       "  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
       "  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (relu): ReLU(inplace=True)\n",
       "  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
       "  (layer1): Sequential(\n",
       "    (0): BasicBlock(\n",
       "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (1): BasicBlock(\n",
       "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (2): BasicBlock(\n",
       "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (layer2): Sequential(\n",
       "    (0): BasicBlock(\n",
       "      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (downsample): Sequential(\n",
       "        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (1): BasicBlock(\n",
       "      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (2): BasicBlock(\n",
       "      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (3): BasicBlock(\n",
       "      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (layer3): Sequential(\n",
       "    (0): BasicBlock(\n",
       "      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (downsample): Sequential(\n",
       "        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (1): BasicBlock(\n",
       "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (2): BasicBlock(\n",
       "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (3): BasicBlock(\n",
       "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (4): BasicBlock(\n",
       "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (5): BasicBlock(\n",
       "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (layer4): Sequential(\n",
       "    (0): BasicBlock(\n",
       "      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (downsample): Sequential(\n",
       "        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (1): BasicBlock(\n",
       "      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "    (2): BasicBlock(\n",
       "      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    )\n",
       "  )\n",
       "  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n",
       "  (fc): Linear(in_features=512, out_features=1000, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.resnet34()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DenseNet(\n",
       "  (features): Sequential(\n",
       "    (conv0): Conv2d(3, 96, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
       "    (norm0): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (relu0): ReLU(inplace=True)\n",
       "    (pool0): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
       "    (denseblock1): _DenseBlock(\n",
       "      (denselayer1): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(96, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer2): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(144, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer3): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(192, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer4): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(240, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer5): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(288, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer6): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(336, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(336, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "    )\n",
       "    (transition1): _Transition(\n",
       "      (norm): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv): Conv2d(384, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)\n",
       "    )\n",
       "    (denseblock2): _DenseBlock(\n",
       "      (denselayer1): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(192, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer2): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(240, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer3): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(288, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer4): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(336, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(336, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer5): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(384, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer6): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(432, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(432, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer7): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer8): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(528, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(528, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer9): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(576, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer10): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(624, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(624, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer11): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(672, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer12): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(720, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(720, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "    )\n",
       "    (transition2): _Transition(\n",
       "      (norm): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv): Conv2d(768, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)\n",
       "    )\n",
       "    (denseblock3): _DenseBlock(\n",
       "      (denselayer1): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(384, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer2): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(432, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(432, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer3): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer4): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(528, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(528, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer5): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(576, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer6): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(624, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(624, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer7): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(672, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer8): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(720, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(720, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer9): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(768, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer10): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(816, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(816, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer11): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(864, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer12): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(912, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(912, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer13): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(960, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer14): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1008, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1008, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer15): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1056, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1056, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer16): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1104, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1104, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer17): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer18): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1200, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer19): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1248, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1248, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer20): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1296, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1296, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer21): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1344, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1344, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer22): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1392, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1392, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer23): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1440, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1440, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer24): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1488, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1488, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer25): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1536, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1536, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer26): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1584, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1584, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer27): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1632, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1632, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer28): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1680, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1680, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer29): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1728, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1728, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer30): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1776, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1776, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer31): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1824, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1824, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer32): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1872, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1872, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer33): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1920, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1920, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer34): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1968, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1968, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer35): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2016, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2016, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer36): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2064, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2064, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "    )\n",
       "    (transition3): _Transition(\n",
       "      (norm): BatchNorm2d(2112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (relu): ReLU(inplace=True)\n",
       "      (conv): Conv2d(2112, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (pool): AvgPool2d(kernel_size=2, stride=2, padding=0)\n",
       "    )\n",
       "    (denseblock4): _DenseBlock(\n",
       "      (denselayer1): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1056, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1056, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer2): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1104, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1104, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer3): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1152, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer4): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1200, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer5): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1248, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1248, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer6): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1296, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1296, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer7): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1344, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1344, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer8): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1392, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1392, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer9): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1440, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1440, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer10): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1488, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1488, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer11): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1536, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1536, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer12): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1584, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1584, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer13): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1632, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1632, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer14): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1680, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1680, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer15): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1728, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1728, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer16): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1776, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1776, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer17): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1824, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1824, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer18): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1872, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1872, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer19): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1920, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1920, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer20): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(1968, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(1968, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer21): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2016, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2016, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer22): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2064, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2064, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer23): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2112, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "      (denselayer24): _DenseLayer(\n",
       "        (norm1): BatchNorm2d(2160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu1): ReLU(inplace=True)\n",
       "        (conv1): Conv2d(2160, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (norm2): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu2): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(192, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "      )\n",
       "    )\n",
       "    (norm5): BatchNorm2d(2208, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (classifier): Linear(in_features=2208, out_features=1000, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.densenet161()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MobileNetV2(\n",
       "  (features): Sequential(\n",
       "    (0): ConvBNReLU(\n",
       "      (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "      (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (2): ReLU6(inplace=True)\n",
       "    )\n",
       "    (1): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)\n",
       "          (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (2): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (2): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(16, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)\n",
       "          (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (3): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)\n",
       "          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(144, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (4): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)\n",
       "          (1): BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(144, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (5): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (6): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192, bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (7): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(32, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192, bias=False)\n",
       "          (1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (8): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (9): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (10): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (11): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384, bias=False)\n",
       "          (1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(384, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (12): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (13): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (14): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576, bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(576, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (15): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (16): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (17): InvertedResidual(\n",
       "      (conv): Sequential(\n",
       "        (0): ConvBNReLU(\n",
       "          (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (1): ConvBNReLU(\n",
       "          (0): Conv2d(960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960, bias=False)\n",
       "          (1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "          (2): ReLU6(inplace=True)\n",
       "        )\n",
       "        (2): Conv2d(960, 320, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (18): ConvBNReLU(\n",
       "      (0): Conv2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "      (1): BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      (2): ReLU6(inplace=True)\n",
       "    )\n",
       "  )\n",
       "  (classifier): Sequential(\n",
       "    (0): Dropout(p=0.2, inplace=False)\n",
       "    (1): Linear(in_features=1280, out_features=1000, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.mobilenet_v2()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ShuffleNetV2(\n",
       "  (conv1): Sequential(\n",
       "    (0): Conv2d(3, 24, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "    (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "  )\n",
       "  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
       "  (stage2): Sequential(\n",
       "    (0): InvertedResidual(\n",
       "      (branch1): Sequential(\n",
       "        (0): Conv2d(24, 24, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=24, bias=False)\n",
       "        (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (4): ReLU(inplace=True)\n",
       "      )\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(24, 24, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=24, bias=False)\n",
       "        (4): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (1): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)\n",
       "        (4): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (2): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)\n",
       "        (4): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (3): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)\n",
       "        (4): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (stage3): Sequential(\n",
       "    (0): InvertedResidual(\n",
       "      (branch1): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=48, bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (4): ReLU(inplace=True)\n",
       "      )\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (1): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (2): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (3): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (4): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (5): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (6): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (7): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "        (4): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (stage4): Sequential(\n",
       "    (0): InvertedResidual(\n",
       "      (branch1): Sequential(\n",
       "        (0): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)\n",
       "        (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (3): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (4): ReLU(inplace=True)\n",
       "      )\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96, bias=False)\n",
       "        (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (1): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
       "        (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (2): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
       "        (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "    (3): InvertedResidual(\n",
       "      (branch2): Sequential(\n",
       "        (0): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (2): ReLU(inplace=True)\n",
       "        (3): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
       "        (4): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (5): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "        (6): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (7): ReLU(inplace=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (conv5): Sequential(\n",
       "    (0): Conv2d(192, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "    (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "  )\n",
       "  (fc): Linear(in_features=1024, out_features=1000, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.shufflenet_v2_x0_5()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MNASNet(\n",
       "  (layers): Sequential(\n",
       "    (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "    (1): BatchNorm2d(32, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "    (3): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)\n",
       "    (4): BatchNorm2d(32, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "    (5): ReLU(inplace=True)\n",
       "    (6): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "    (7): BatchNorm2d(16, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "    (8): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=48, bias=False)\n",
       "          (4): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(16, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "          (4): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(16, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (2): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
       "          (4): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(16, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (9): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(48, 48, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=48, bias=False)\n",
       "          (4): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(24, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(24, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(72, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(72, 72, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=72, bias=False)\n",
       "          (4): BatchNorm2d(72, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(72, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(24, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (2): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(24, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(72, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(72, 72, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=72, bias=False)\n",
       "          (4): BatchNorm2d(72, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(72, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(24, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (10): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(144, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(144, 144, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=144, bias=False)\n",
       "          (4): BatchNorm2d(144, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(144, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(40, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(240, 240, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=240, bias=False)\n",
       "          (4): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(40, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (2): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(240, 240, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=240, bias=False)\n",
       "          (4): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(40, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (11): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)\n",
       "          (4): BatchNorm2d(240, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(240, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(48, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(288, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(288, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=288, bias=False)\n",
       "          (4): BatchNorm2d(288, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(288, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(48, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (12): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(48, 288, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(288, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(288, 288, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=288, bias=False)\n",
       "          (4): BatchNorm2d(288, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(288, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(96, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(576, 576, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=576, bias=False)\n",
       "          (4): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(96, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (2): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(576, 576, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=576, bias=False)\n",
       "          (4): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(96, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (3): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(576, 576, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=576, bias=False)\n",
       "          (4): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(96, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (13): Sequential(\n",
       "      (0): _InvertedResidual(\n",
       "        (layers): Sequential(\n",
       "          (0): Conv2d(96, 576, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (1): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (2): ReLU(inplace=True)\n",
       "          (3): Conv2d(576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576, bias=False)\n",
       "          (4): BatchNorm2d(576, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "          (5): ReLU(inplace=True)\n",
       "          (6): Conv2d(576, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "          (7): BatchNorm2d(160, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (14): Conv2d(160, 1280, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
       "    (15): BatchNorm2d(1280, eps=1e-05, momentum=0.00029999999999996696, affine=True, track_running_stats=True)\n",
       "    (16): ReLU(inplace=True)\n",
       "  )\n",
       "  (classifier): Sequential(\n",
       "    (0): Dropout(p=0.2, inplace=True)\n",
       "    (1): Linear(in_features=1280, out_features=1000, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models.mnasnet0_5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 备注：关于图像处理具体任务后续以实例进行详解，这里介绍模型以备后续使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}