pytorch常用函数

squeeze和unsqueeze

squeeze()主要用于删除tensor张量中维数为1的指定维度,而unsqueeze用于在指定维度上插入一个新的维数为1的维度。

Example:

1
2
3
4
5
6
7
8
9
10
11
12
>>> x = torch.zeros(2, 1, 2, 1, 2)
>>> x.size()
torch.Size([2, 1, 2, 1, 2])
>>> y1 = torch.squeeze(x)
>>> y1.size()
torch.Size([2, 2, 2])
>>> y1 = torch.squeeze(x, 0)
>>> y1.size()
torch.Size([2, 1, 2, 1, 2])
>>> y1 = torch.squeeze(x, 1)
>>> y1.size()
torch.Size([2, 2, 1, 2])

Example:

1
2
3
4
5
6
7
8
>>> x = torch.tensor([1, 2, 3, 4])
>>> torch.unsqueeze(x, 0)
tensor([[ 1, 2, 3, 4]])
>>> torch.unsqueeze(x, 1)
tensor([[ 1],
[ 2],
[ 3],
[ 4]])

MaxPool的ceil_mode参数

在介绍MaxPool的ceil_mode参数前,首先介绍下MaxPool2d的相关配置:

MaxPool2d

Example:

1
2
3
4
5
6
>>> # pool of square window of size=3, stride=2
>>> m = nn.MaxPool2d(3, stride=2)
>>> # pool of non-square window
>>> m = nn.MaxPool2d((3, 2), stride=(2, 1))
>>> input = torch.randn(20, 16, 50, 32)
>>> output = m(input)

ceil_mode

上面官网给出ceil_mode相关介绍:

  • ceil_mode – when True, will use ceil instead of floor to compute the output shape

将其置为True,则对输出进行向上取整;默认False情况下,对输出进行向下取整。

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# coding:utf-8
import torch
import torch.nn as nn
from torch.autograd import Variable


class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.maxp = nn.MaxPool2d(kernel_size=2, ceil_mode=False)

def forward(self, x):
x = self.maxp(x)
return x

square_size = 6
inputs = torch.randn(1, 1, square_size, square_size)
for i in range(square_size):
inputs[0][0][i] = i * torch.ones(square_size)
inputs = Variable(inputs)
print(inputs)

net = Net()
outputs = net(inputs)
print(outputs.size())
print(outputs)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Variable containing: 

(0 ,0 ,.,.) =

0 0 0 0 0 0

1 1 1 1 1 1

2 2 2 2 2 2

3 3 3 3 3 3

4 4 4 4 4 4

5 5 5 5 5 5

[torch.FloatTensor of size 1x1x6x6]

torch.Size([1, 1, 3, 3])

Variable containing:

(0 ,0 ,.,.) =

1 1 1

3 3 3

5 5 5

[torch.FloatTensor of size 1x1x3x3]

在上面的代码中,无论ceil_mode是True or False,结果都是一样
但是如果设置square_size=5,那么

ceil_mode=True

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Variable containing: 

(0 ,0 ,.,.) =

0 0 0 0 0

1 1 1 1 1

2 2 2 2 2

3 3 3 3 3

4 4 4 4 4

[torch.FloatTensor of size 1x1x5x5]
torch.Size([1, 1, 3, 3])
Variable containing:

(0 ,0 ,.,.) =

1 1 1

3 3 3

4 4 4

[torch.FloatTensor of size 1x1x3x3]

ceil_mode=False

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Variable containing: 

(0 ,0 ,.,.) =

0 0 0 0 0

1 1 1 1 1

2 2 2 2 2

3 3 3 3 3

4 4 4 4 4

[torch.FloatTensor of size 1x1x5x5]

torch.Size([1, 1, 2, 2])

Variable containing:

(0 ,0 ,.,.) =

1 1

3 3

[torch.FloatTensor of size 1x1x2x2]

ceil模式会把不足square_size的边给保留下来,单独另算。而floor模式则是直接把不足square_size的边给舍弃

Conv2d的dilation参数

首先对Conv2d的各项配置参数进行介绍,然后再详细解释dilation参数

Conv2d

1
CLASS torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros')

Example:

1
2
3
4
5
6
7
8
>>> # With square kernels and equal stride
>>> m = nn.Conv2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> # non-square kernels and unequal stride and with padding and dilation
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)

dilation

dilation参数意味着对卷积核进行膨胀,膨胀的部分用0进行填充,用术语讲称之为空洞卷积。

空洞卷积在增加卷积核感受野的同时,参数并未增加,有效减少计算量

计算公式:

在 $kernel=3,dilation=2$ 情况下,卷积核膨胀的效果图如下:

img

Reference

PyTorch

Pytorch中 maxpool的ceil_mode

caffe卷积层延伸:卷积核膨胀详细解析

Some say the world will end in fire,

Some say in ice.

From what I’ve tasted of desire

I hold with those who favor fire.

But if it had to perish twice,

I think I know enough of hate

To say that for destruction ice

Is also great

And would suffice.

-------------本文结束感谢阅读-------------
Thanks for your rewarding ~
0%