Swin Transformer2: Model Structure

24 Apr 2023 in Engineer on PaperNotes

• Overview
  • Swin Transformer Structure
  • Swin Transformer Category
• Swin-T
  • Swin-T Parameters
  • Swin-T Model Structure in Code

This explains the Swin-T model structure based on python functions

• Githbub: swin_transformer_simplified

Overview

Swin Transformer Structure

Swin Transformer Category

Swin-T

Swin-T Parameters

swin_t(hidden_dim=96, layers=(2, 2, 6, 2), heads=(3, 6, 12, 24), **kwargs)

• hidden_dim: is the head_attention_size * number_of_head = 96
• layers: the number of regular_attention_module & shifted_attention_module in every layers
• heads: the number of head in every layers

Swin-T Model Structure in Code