Local Response Normalization (LRN) has different implementations in different Frameworks.
Caffe🔗
-
Parameters (LRNParameter lrn_param) Optional
- local_size [default 5]: the number of channels to sum over (for cross channel LRN) or the side length of the square region to sum over (for within channel LRN)
- alpha [default 1]: the scaling parameter (see below)
- beta [default 5]: the exponent (see below)
- norm_region [default ACROSS_CHANNELS]: whether to sum over adjacent channels (ACROSS_CHANNELS) or nearby spatial locations (WITHIN_CHANNEL)
-
计算公式
$$ b_{x,y}^i = a_{x,y}^i \left(1 + \frac{\alpha}{n} \sum_{j=\max(0, i-n/2)}^{\min(N-1,i+n/2)}a_{x,y}^2\right)^{-\beta} $$
TensorFlow🔗
different from Caffe version. refer to this [post](<https://blog.csdn.net/newworld123made/article/details/78880724).
- 计算公式
$$ b_{x,y}^i = a_{x,y}^i \left(b + \alpha \sum_{j=\max(0, i-r)}^{\min(N-1,i+r)}a_{x,y}^2\right)^{-\beta} $$
PyTorch🔗
same as Caffe version, the parameters can be used directly.
- 计算公式
$$ b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c’=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c’}^2\right)^{-\beta} $$