[Submitted on 7 Aug 2017 (v1), last revised 7 Feb 2018 (this version, v2)]

Focal Loss for Dense Object Detection

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár

The highest accuracy object detectors to date are based on a two-stage approach popularized by R-CNN, where a classifier is applied to a sparse set of candidate object locations. In contrast, one-stage detectors that are applied over a regular, dense sampling of possible object locations have the potential to be faster and simpler, but have trailed the accuracy of two-stage detectors thus far. In this paper, we investigate why this is the case. We discover that the extreme foreground-background class imbalance encountered during training of dense detectors is the central cause. We propose to address this class imbalance by reshaping the standard cross entropy loss such that it down-weights the loss assigned to well-classified examples. Our novel Focal Loss focuses training on a sparse set of hard examples and prevents the vast number of easy negatives from overwhelming the detector during training. To evaluate the effectiveness of our loss, we design and train a simple dense detector we call RetinaNet. Our results show that when trained with the focal loss, RetinaNet is able to match the speed of previous one-stage detectors while surpassing the accuracy of all existing state-of-the-art two-stage detectors. Code is at: this https URL.

Subjects: Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:1708.02002 [cs.CV]   (or arXiv:1708.02002v2 [cs.CV] for this version)   https://doi.org/10.48550/arXiv.1708.02002

Focus to learn more

Focal Loss for Dense Object Detection

Tsung-Yi Lin

Priya Goyal

Ross Girshick

Kaiming He

Piotr Doll´ar

Facebook AI Research (FAIR)

0

0.2

0.4

0.6

0.8

1

probability of ground truth class

0

1

2

3

4

5

loss

= 0

= 0.5

= 1

= 2

= 5

well-classifed

examples

well-classifed

examples

CE(pt) = − log(pt)

FL(pt) = −(1 − pt)γ log(pt)

Figure 1. We propose a novel loss we term the Focal Loss that

adds a factor (1 − pt)γ to the standard cross entropy criterion.

Setting γ > 0 reduces the relative loss for well-classified examples

(pt > .5), putting more focus on hard, misclassified examples. As

our experiments will demonstrate, the proposed focal loss enables

training highly accurate dense object detectors in the presence of

vast numbers of easy background examples.

Abstract

The highest accuracy object detectors to date are based

on a two-stage approach popularized by R-CNN, where a

classifier is applied to a sparse set of candidate object lo-

cations. In contrast, one-stage detectors that are applied

over a regular, dense sampling of possible object locations

have the potential to be faster and simpler, but have trailed

the accuracy of two-stage detectors thus far. In this paper,

we investigate why this is the case. We discover that the ex-

treme foreground-background class imbalance encountered

during training of dense detectors is the central cause. We

propose to address this class imbalance by reshaping the

standard cross entropy loss such that it down-weights the

loss assigned to well-classified examples. Our novel Focal

Loss focuses training on a sparse set of hard examples and

prevents the vast number of easy negatives from overwhelm-

ing the detector during training. To evaluate the effective-

ness of our loss, we design and train a simple dense detector

we call RetinaNet. Our results show that when trained with

the focal loss, RetinaNet is able to match the speed of pre-

vious one-stage detectors while surpassing the accuracy of

all existing state-of-the-art two-stage detectors.

50

100

150

200

250

inference time (ms)

28

30

32

34

36

38

COCO AP

B

C

D

E

F

G

RetinaNet-50

RetinaNet-101

AP

time

[A] YOLOv2† [26]

21.6

25

[B] SSD321 [21]

28.0

61

[C] DSSD321 [9]

28.0

85

[D] R-FCN‡ [3]

29.9

85

[E] SSD513 [21]

31.2

125

[F] DSSD513 [9]

33.2

156

[G] FPN FRCN [19] 36.2

172

RetinaNet-50-500

32.5

73

RetinaNet-101-500

34.4

90

RetinaNet-101-800

37.8

198

†Not plotted

‡Extrapolated time

Figure 2. Speed (ms) versus accuracy (AP) on COCO test-dev.

Enabled by the focal loss, our simple one-stage RetinaNet detec-

tor outperforms all previous one-stage and two-stage detectors, in-

cluding the best reported Faster R-CNN [27] system from [19]. We

show variants of RetinaNet with ResNet-50-FPN (blue circles) and

ResNet-101-FPN (orange diamonds) at five scales (400-800 pix-

els). Ignoring the low-accuracy regime (AP<25), RetinaNet forms

an upper envelope of all current detectors, and a variant trained for

longer (not shown) achieves 39.1 AP. Details are given in §5.

1. Introduction

Current state-of-the-art object detectors are based on

a two-stage, proposal-driven mechanism. As popularized

in the R-CNN framework [11], the first stage generates a

sparse set of candidate object locations and the second stage

classifies each candidate location as one of the foreground

classes or as background using a convolutional neural net-

work. Through a sequence of advances [10, 27, 19, 13], this

two-stage framework consistently achieves top accuracy on

the challenging COCO benchmark [20].

Despite the success of two-stage detectors, a natural

question to ask is: could a simple one-stage detector achieve

similar accuracy? One stage detectors are applied over a

regular, dense sampling of object locations, scales, and as-

pect ratios. Recent work on one-stage detectors, such as

YOLO [25, 26] and SSD [21, 9], demonstrates promising

results, yielding faster detectors with accuracy within 10-

40% relative to state-of-the-art two-stage methods.

This paper pushes the envelop further: we present a one-

stage object detector that, for the first time, matches the

12980

state-of-the-art COCO AP of more complex two-stage de-

tectors, such as the Feature Pyramid Network (FPN) [19]

or Mask R-CNN [13] variants of Faster R-CNN [27]. To

achieve this result, we identify class imbalance during train-

ing as the main obstacle impeding one-stage detector from

achieving state-of-the-art accuracy and propose a new loss

function that eliminates this barrier.

Class imbalance is addressed in R-CNN-like detectors

by a two-stage cascade and sampling heuristics. The pro-

posal stage (e.g., Selective Search [34], EdgeBoxes [37],

DeepMask [23, 24], RPN [27]) rapidly narrows down the

number of candidate object locations to a small number

(e.g., 1-2k), filtering out most background samples. In the

second classification stage, sampling heuristics, such as a

fixed foreground-to-background ratio (1:3), or online hard

example mining (OHEM) [30], are performed to maintain a

manageable balance between foreground and background.

In contrast, a one-stage detector must process a much

larger set of candidate object locations regularly sampled

across an image. In practice this often amounts to enumer-

ating ``∼100k locations that densely cover spatial positions,

scales, and aspect ratios. While similar sampling heuris-

tics may also be applied, they are inefficient as the training

procedure is still dominated by easily classified background

examples. This inefficiency is a classic problem in object

detection that is typically addressed via techniques such as

bootstrapping [32, 28] or hard example mining [36, 8, 30].

In this paper, we propose a new loss function that acts

as a more effective alternative to previous approaches for

dealing with class imbalance. The loss function is a dy-

namically scaled cross entropy loss, where the scaling factor

decays to zero as confidence in the correct class increases,

see Figure 1. Intuitively, this scaling factor can automati-

cally down-weight the contribution of easy examples during

training and rapidly focus the model on hard examples. Ex-

periments show that our proposed Focal Loss enables us to

train a high-accuracy, one-stage detector that significantly

outperforms the alternatives of training with the sampling

heuristics or hard example mining, the previous state-of-

the-art techniques for training one-stage detectors. Finally,

we note that the exact form of the focal loss is not crucial,

and we show other instantiations can achieve similar results.

To demonstrate the effectiveness of the proposed focal

loss, we design a simple one-stage object detector called

RetinaNet, named for its dense sampling of object locations

in an input image. Its design features an efficient in-network

feature pyramid and use of anchor boxes. It draws on a va-

riety of recent ideas from [21, 6, 27, 19]. RetinaNet is effi-

cient and accurate; our best model, based on a ResNet-101-

FPN backbone, achieves a COCO test-dev AP of 39.1

while running at 5 fps, surpassing the previously best pub-

lished single-model results from both one and two-stage de-

tectors, see Figure 2.

2. Related Work

Classic Object Detectors: The sliding-window paradigm,

in which a classifier is applied on a dense image grid, has

a long and rich history. One of the earliest successes is the

classic work of LeCun et al. who applied convolutional neu-

ral networks to handwritten digit recognition [18, 35]. Vi-

ola and Jones [36] used boosted object detectors for face

detection, leading to widespread adoption of such models.

The introduction of HOG [4] and integral channel features

[5] gave rise to effective methods for pedestrian detection.

DPMs [8] helped extend dense detectors to more general

object categories and had top results on PASCAL [7] for

many years. While the sliding-window approach was the

leading detection paradigm in classic computer vision, with

the resurgence of deep learning [17], two-stage detectors,

described next, quickly came to dominate object detection.

Two-stage Detectors: The dominant paradigm in modern

object detection is based on a two-stage approach. As pio-

neered in the Selective Search work [34], the first stage gen-

erates a sparse set of candidate proposals that should con-

tain all objects while filtering out the majority of negative

locations, and the second stage classifies the proposals into

foreground classes / background. R-CNN [11] upgraded the

second-stage classifier to a convolutional network yielding

large gains in accuracy and ushering in the modern era of

object detection. R-CNN was improved over the years, both

in terms of speed [14, 10] and by using learned object pro-

posals [6, 23, 27]. Region Proposal Networks (RPN) inte-

grated proposal generation with the second-stage classifier

into a single convolution network, forming the Faster R-

CNN framework [27]. Numerous extensions to this frame-

work have been proposed, e.g. [19, 30, 31, 15, 13].

One-stage Detectors: OverFeat [29] was one of the first

modern one-stage object detector based on deep networks.

More recently SSD [21, 9] and YOLO [25, 26] have re-

newed interest in one-stage methods. These detectors have

been tuned for speed but their accuracy trails that of two-

stage methods. SSD has a 10-20% lower AP, while YOLO

focuses on an even more extreme speed/accuracy trade-off.

See Figure 2. Recent work showed that two-stage detectors

can be made fast simply by reducing input image resolution

and the number of proposals, but one-stage methods trailed

in accuracy even with a larger compute budget [16]. In con-

trast, the aim of this work is to understand if one-stage de-

tectors can match or surpass the accuracy of two-stage de-

tectors while running at similar or faster speeds.

The design of our RetinaNet detector shares many simi-

larities with previous dense detectors, in particular the con-

cept of ‘anchors’ introduced by RPN [27] and use of fea-

tures pyramids as in SSD [21] and FPN [19]. We empha-

size that our simple detector achieves top results not based

on innovations in network design but due to our novel loss.

22981

Class Imbalance: Both classic one-stage object detection

methods, like boosted detectors [36, 5] and DPMs [8], and

more recent methods, like SSD [21], face a large class

imbalance during training. These detectors evaluate 104-

105 candidate locations per image but only a few loca-

tions contain objects. This imbalance causes two problems:

(1) training is inefficient as most locations are easy nega-

tives that contribute no useful learning signal; (2) en masse,

the easy negatives can overwhelm training and lead to de-

generate models. A common solution is to perform some

form of hard negative mining [32, 36, 8, 30, 21] that sam-

ples hard examples during training or more complex sam-

pling/reweighing schemes [2]. In contrast, we show that our

proposed focal loss naturally handles the class imbalance

faced by a one-stage detector and allows us to efficiently

train on all examples without sampling and without easy

negatives overwhelming the loss and computed gradients.

Robust Estimation: There has been much interest in de-

signing robust loss functions (e.g., Huber loss [12]) that re-

duce the contribution of outliers by down-weighting the loss

of examples with large errors (hard examples). In contrast,

rather than addressing outliers, our focal loss is designed

to address class imbalance by down-weighting inliers (easy

examples) such that their contribution to the total loss is

small even if their number is large. In other words, the focal

loss performs the opposite role of a robust loss: it focuses

training on a sparse set of hard examples.

3. Focal Loss

The Focal Loss is designed to address the one-stage ob-

ject detection scenario in which there is an extreme im-

balance between foreground and background classes during

training (e.g., 1:1000). We introduce the focal loss starting

from the cross entropy (CE) loss for binary classification1:

CE(p, y) =

�

− log(p)

if y = 1

− log(1 − p)

otherwise.

(1)

In the above y ∈ {±1} specifies the ground-truth class and

p ∈ [0, 1] is the model’s estimated probability for the class

with label y = 1. For notational convenience, we define pt:

pt =

�

p

if y = 1

1 − p

otherwise,

(2)

and rewrite CE(p, y) = CE(pt) = − log(pt).

The CE loss can be seen as the blue (top) curve in Fig-

ure 1. One notable property of this loss, which can be easily

seen in its plot, is that even examples that are easily clas-

sified (pt ≫ .5) incur a loss with non-trivial magnitude.

When summed over a large number of easy examples, these

small loss values can overwhelm the rare class.

1Extending the focal loss to the multi-class case is straightforward and

works well; for simplicity we focus on the binary loss in this work.

3.1. Balanced Cross Entropy

A common method for addressing class imbalance is to

introduce a weighting factor α ∈ [0, 1] for class 1 and 1−α

for class −1. In practice α may be set by inverse class fre-

quency or treated as a hyperparameter to set by cross valida-

tion. For notational convenience, we define αt analogously

to how we defined pt. We write the α-balanced CE loss as:

CE(pt) = −αt log(pt).

(3)

This loss is a simple extension to CE that we consider as an

experimental baseline for our proposed focal loss.

3.2. Focal Loss Definition

As our experiments will show, the large class imbalance

encountered during training of dense detectors overwhelms

the cross entropy loss. Easily classified negatives comprise

the majority of the loss and dominate the gradient. While

α balances the importance of positive/negative examples, it

does not differentiate between easy/hard examples. Instead,

we propose to reshape the loss function to down-weight

easy examples and thus focus training on hard negatives.

More formally, we propose to add a modulating factor

(1 − pt)γ to the cross entropy loss, with tunable focusing

parameter γ ≥ 0. We define the focal loss as:

FL(pt) = −(1 − pt)γ log(pt).

(4)

The focal loss is visualized for several values of γ ∈

[0, 5] in Figure 1. We note two properties of the focal loss.

(1) When an example is misclassified and pt is small, the

modulating factor is near 1 and the loss is unaffected. As

pt → 1, the factor goes to 0 and the loss for well-classified

examples is down-weighted. (2) The focusing parameter γ

smoothly adjusts the rate at which easy examples are down-

weighted. When γ = 0, FL is equivalent to CE, and as γ is

increased the effect of the modulating factor is likewise in-

creased (we found γ = 2 to work best in our experiments).

Intuitively, the modulating factor reduces the loss contri-

bution from easy examples and extends the range in which

an example receives low loss. For instance, with γ = 2, an

example classified with pt = 0.9 would have 100× lower

loss compared with CE and with pt ≈ 0.968 it would have

1000× lower loss. This in turn increases the importance

of correcting misclassified examples (whose loss is scaled

down by at most 4× for pt ≤ .5 and γ = 2).

In practice we use an α-balanced variant of the focal loss:

FL(pt) = −αt(1 − pt)γ log(pt).

(5)

We adopt this form in our experiments as it yields slightly

improved accuracy over the non-α-balanced form. Finally,

we note that the implementation of the loss layer combines

the sigmoid operation for computing p with the loss com-

putation, resulting in greater numerical stability.

32982

While in our main experimental results we use the focal

loss definition above, its precise form is not crucial. In the

online appendix we consider other instantiations of the focal

loss and demonstrate that these can be equally effective.

3.3. Class Imbalance and Model Initialization

Binary classification models are by default initialized to

have equal probability of outputting either y = −1 or 1.

Under such an initialization, in the presence of class imbal-

ance, the loss due to the frequent class can dominate total

loss and cause instability in early training. To counter this,

we introduce the concept of a ‘prior’ for the value of p es-

timated by the model for the rare class (foreground) at the

start of training. We denote the prior by π and set it so that

the model’s estimated p for examples of the rare class is low,

e.g. 0.01. We note that this is a change in model initializa-

tion (see §4.1) and not of the loss function. We found this

to improve training stability for both the cross entropy and

focal loss in the case of heavy class imbalance.

3.4. Class Imbalance and Two-stage Detectors

Two-stage detectors are often trained with the cross en-

tropy loss without use of α-balancing or our proposed loss.

Instead, they address class imbalance through two mech-

anisms: (1) a two-stage cascade and (2) biased minibatch

sampling.

The first cascade stage is an object proposal

mechanism [34, 23, 27] that reduces the nearly infinite set

of possible object locations down to one or two thousand.

Importantly, the selected proposals are not random, but are

likely to correspond to true object locations, which removes

the vast majority of easy negatives. When training the sec-

ond stage, biased sampling is typically used to construct

minibatches that contain, for instance, a 1:3 ratio of posi-

tive to negative examples. This ratio is like an implicit α-

balancing factor that is implemented via sampling. Our pro-

posed focal loss is designed to address these mechanisms in

a one-stage detection system directly via the loss function.

4. RetinaNet Detector

RetinaNet is a single, unified network composed of a

backbone network and two task-specific subnetworks. The

backbone is responsible for computing a convolutional fea-

ture map over an entire input image and is an off-the-self

convolutional network. The first subnet performs convo-

lutional object classification on the backbone’s output; the

second subnet performs convolutional bounding box regres-

sion. The two subnetworks feature a simple design that we

propose specifically for one-stage, dense detection, see Fig-

ure 3. While there are many possible choices for the details

of these components, most design parameters are not partic-

ularly sensitive to exact values as shown in the experiments.

We describe each component of RetinaNet next.

Feature Pyramid Network Backbone: We adopt the Fea-

ture Pyramid Network (FPN) from [19] as the backbone

network for RetinaNet.

In brief, FPN augments a stan-

dard convolutional network with a top-down pathway and

lateral connections so the network efficiently constructs a

rich, multi-scale feature pyramid from a single resolution

input image, see Figure 3(a)-(b). Each level of the pyramid

can be used for detecting objects at a different scale. FPN

improves multi-scale predictions from fully convolutional

networks (FCN) [22], as shown by its gains for RPN [27]

and DeepMask-style proposals [23], as well at two-stage

detectors such as Fast R-CNN [10] or Mask R-CNN [13].

Following [19], we build FPN on top of the ResNet ar-

chitecture [15].

We construct a pyramid with levels P3

through P7, where l indicates pyramid level (Pl has reso-

lution 2l lower than the input). As in [19] all pyramid levels

have C = 256 channels. Details of the pyramid generally

follow [19] with a few modest differences.2 While many

design choices are not crucial, we emphasize the use of the

FPN backbone is; preliminary experiments using features

from only the final ResNet layer yielded low AP.

Anchors: We use translation- ...

Focal Loss for Dense Object Detection

Tsung-Yi Lin

Priya Goyal

Ross Girshick

Kaiming He

Piotr Doll´ar

Facebook AI Research (FAIR)

0

0.2

0.4

0.6

0.8

1

probability of ground truth class

0

1

2

3

4

5

loss

= 0

= 0.5

= 1

= 2

= 5

well-classifed

examples

well-classifed

examples

CE(pt) = − log(pt)

FL(pt) = −(1 − pt)γ log(pt)

Figure 1. We propose a novel loss we term the Focal Loss that

adds a factor (1 − pt)γ to the standard cross entropy criterion.

Setting γ > 0 reduces the relative loss for well-classified examples

(pt *> .*5), putting more focus on hard, misclassified examples. As

our experiments will demonstrate, the proposed focal loss enables

training highly accurate dense object detectors in the presence of

vast numbers of easy background examples.

Abstract

The highest accuracy object detectors to date are based

on a two-stage approach popularized by R-CNN, where a

classifier is applied to a sparse set of candidate object lo-

cations. In contrast, one-stage detectors that are applied

over a regular, dense sampling of possible object locations

have the potential to be faster and simpler, but have trailed

the accuracy of two-stage detectors thus far. In this paper,

we investigate why this is the case. We discover that the ex-

treme foreground-background class imbalance encountered

during training of dense detectors is the central cause. We

propose to address this class imbalance by reshaping the

standard cross entropy loss such that it down-weights the

loss assigned to well-classified examples. Our novel Focal

Loss focuses training on a sparse set of hard examples and

prevents the vast number of easy negatives from overwhelm-

ing the detector during training. To evaluate the effective-

ness of our loss, we design and train a simple dense detector

we call RetinaNet. Our results show that when trained with

the focal loss, RetinaNet is able to match the speed of pre-

vious one-stage detectors while surpassing the accuracy of

all existing state-of-the-art two-stage detectors. Code is at:

https://github.com/facebookresearch/Detectron.

50

100

150

200

250

inference time (ms)

28

30

32

34

36

38

COCO AP

B

C

D

E

F

G

RetinaNet-50

RetinaNet-101

AP

time

[A] YOLOv2*†* [27]

21.6

25

[B] SSD321 [22]

28.0

61

[C] DSSD321 [9]

28.0

85

[D] R-FCN*‡* [3]

29.9

85

[E] SSD513 [22]

31.2

125

[F] DSSD513 [9]

33.2

156

[G] FPN FRCN [20] 36.2

172

RetinaNet-50-500

32.5

73

RetinaNet-101-500

34.4

90

RetinaNet-101-800

37.8

198

*†*Not plotted

*‡*Extrapolated time

Figure 2. Speed (ms) versus accuracy (AP) on COCO test-dev.

Enabled by the focal loss, our simple one-stage RetinaNet detec-

tor outperforms all previous one-stage and two-stage detectors, in-

cluding the best reported Faster R-CNN [28] system from [20].

We show variants of RetinaNet with ResNet-50-FPN (blue circles)

and ResNet-101-FPN (orange diamonds) at five scales (400-800

pixels). Ignoring the low-accuracy regime (AP*<*25), RetinaNet

forms an upper envelope of all current detectors, and an improved

variant (not shown) achieves 40.8 AP. Details are given in *§*5.

1. Introduction

Current state-of-the-art object detectors are based on

a two-stage, proposal-driven mechanism. As popularized

in the R-CNN framework [11], the first stage generates a

sparse set of candidate object locations and the second stage

classifies each candidate location as one of the foreground

classes or as background using a convolutional neural net-

work. Through a sequence of advances [10, 28, 20, 14], this

two-stage framework consistently achieves top accuracy on

the challenging COCO benchmark [21].

Despite the success of two-stage detectors, a natural

question to ask is: could a simple one-stage detector achieve

similar accuracy? One stage detectors are applied over a

regular, dense sampling of object locations, scales, and as-

pect ratios. Recent work on one-stage detectors, such as

YOLO [26, 27] and SSD [22, 9], demonstrates promising

results, yielding faster detectors with accuracy within 10-

40% relative to state-of-the-art two-stage methods.

This paper pushes the envelop further: we present a one-

stage object detector that, for the first time, matches the

state-of-the-art COCO AP of more complex two-stage de-

1

arXiv:1708.02002v2 [cs.CV] 7 Feb 2018

tectors, such as the Feature Pyramid Network (FPN) [20]

or Mask R-CNN [14] variants of Faster R-CNN [28]. To

achieve this result, we identify class imbalance during train-

ing as the main obstacle impeding one-stage detector from

achieving state-of-the-art accuracy and propose a new loss

function that eliminates this barrier.

Class imbalance is addressed in R-CNN-like detectors

by a two-stage cascade and sampling heuristics. The pro-

posal stage (e.g., Selective Search [35], EdgeBoxes [39],

DeepMask [24, 25], RPN [28]) rapidly narrows down the

number of candidate object locations to a small number

(e.g., 1-2k), filtering out most background samples. In the

second classification stage, sampling heuristics, such as a

fixed foreground-to-background ratio (1:3), or online hard

example mining (OHEM) [31], are performed to maintain a

manageable balance between foreground and background.

In contrast, a one-stage detector must process a much

larger set of candidate object locations regularly sampled

across an image. In practice this often amounts to enumer-

ating *∼*100k locations that densely cover spatial positions,

scales, and aspect ratios. While similar sampling heuris-

tics may also be applied, they are inefficient as the training

procedure is still dominated by easily classified background

examples. This inefficiency is a classic problem in object

detection that is typically addressed via techniques such as

bootstrapping [33, 29] or hard example mining [37, 8, 31].

In this paper, we propose a new loss function that acts

as a more effective alternative to previous approaches for

dealing with class imbalance. The loss function is a dy-

namically scaled cross entropy loss, where the scaling factor

decays to zero as confidence in the correct class increases,

see Figure 1. Intuitively, this scaling factor can automati-

cally down-weight the contribution of easy examples during

training and rapidly focus the model on hard examples. Ex-

periments show that our proposed Focal Loss enables us to

train a high-accuracy, one-stage detector that significantly

outperforms the alternatives of training with the sampling

heuristics or hard example mining, the previous state-of-

the-art techniques for training one-stage detectors. Finally,

we note that the exact form of the focal loss is not crucial,

and we show other instantiations can achieve similar results.

To demonstrate the effectiveness of the proposed focal

loss, we design a simple one-stage object detector called

RetinaNet, named for its dense sampling of object locations

in an input image. Its design features an efficient in-network

feature pyramid and use of anchor boxes. It draws on a va-

riety of recent ideas from [22, 6, 28, 20]. RetinaNet is effi-

cient and accurate; our best model, based on a ResNet-101-

FPN backbone, achieves a COCO test-dev AP of 39.1

while running at 5 fps, surpassing the previously best pub-

lished single-model results from both one and two-stage de-

tectors, see Figure 2.

2. Related Work

Classic Object Detectors: The sliding-window paradigm,

in which a classifier is applied on a dense image grid, has

a long and rich history. One of the earliest successes is the

classic work of LeCun et al. who applied convolutional neu-

ral networks to handwritten digit recognition [19, 36]. Vi-

ola and Jones [37] used boosted object detectors for face

detection, leading to widespread adoption of such models.

The introduction of HOG [4] and integral channel features

[5] gave rise to effective methods for pedestrian detection.

DPMs [8] helped extend dense detectors to more general

object categories and had top results on PASCAL [7] for

many years. While the sliding-window approach was the

leading detection paradigm in classic computer vision, with

the resurgence of deep learning [18], two-stage detectors,

described next, quickly came to dominate object detection.

Two-stage Detectors: The dominant paradigm in modern

object detection is based on a two-stage approach. As pio-

neered in the Selective Search work [35], the first stage gen-

erates a sparse set of candidate proposals that should con-

tain all objects while filtering out the majority of negative

locations, and the second stage classifies the proposals into

foreground classes / background. R-CNN [11] upgraded the

second-stage classifier to a convolutional network yielding

large gains in accuracy and ushering in the modern era of

object detection. R-CNN was improved over the years, both

in terms of speed [15, 10] and by using learned object pro-

posals [6, 24, 28]. Region Proposal Networks (RPN) inte-

grated proposal generation with the second-stage classifier

into a single convolution network, forming the Faster R-

CNN framework [28]. Numerous extensions to this frame-

work have been proposed, e.g. [20, 31, 32, 16, 14].

One-stage Detectors: OverFeat [30] was one of the first

modern one-stage object detector based on deep networks.

More recently SSD [22, 9] and YOLO [26, 27] have re-

newed interest in one-stage methods. These detectors have

been tuned for speed but their accuracy trails that of two-

stage methods. SSD has a 10-20% lower AP, while YOLO

focuses on an even more extreme speed/accuracy trade-off.

See Figure 2. Recent work showed that two-stage detectors

can be made fast simply by reducing input image resolution

and the number of proposals, but one-stage methods trailed

in accuracy even with a larger compute budget [17]. In con-

trast, the aim of this work is to understand if one-stage de-

tectors can match or surpass the accuracy of two-stage de-

tectors while running at similar or faster speeds.

The design of our RetinaNet detector shares many simi-

larities with previous dense detectors, in particular the con-

cept of ‘anchors’ introduced by RPN [28] and use of fea-

tures pyramids as in SSD [22] and FPN [20]. We empha-

size that our simple detector achieves top results not based

on innovations in network design but due to our novel loss.

2

Class Imbalance: Both classic one-stage object detection

methods, like boosted detectors [37, 5] and DPMs [8], and

more recent methods, like SSD [22], face a large class

imbalance during training. These detectors evaluate 104-

105 candidate locations per image but only a few loca-

tions contain objects. This imbalance causes two problems:

(1) training is inefficient as most locations are easy nega-

tives that contribute no useful learning signal; (2) en masse,

the easy negatives can overwhelm training and lead to de-

generate models. A common solution is to perform some

form of hard negative mining [33, 37, 8, 31, 22] that sam-

ples hard examples during training or more complex sam-

pling/reweighing schemes [2]. In contrast, we show that our

proposed focal loss naturally handles the class imbalance

faced by a one-stage detector and allows us to efficiently

train on all examples without sampling and without easy

negatives overwhelming the loss and computed gradients.

Robust Estimation: There has been much interest in de-

signing robust loss functions (e.g., Huber loss [13]) that re-

duce the contribution of outliers by down-weighting the loss

of examples with large errors (hard examples). In contrast,

rather than addressing outliers, our focal loss is designed

to address class imbalance by down-weighting inliers (easy

examples) such that their contribution to the total loss is

small even if their number is large. In other words, the focal

loss performs the opposite role of a robust loss: it focuses

training on a sparse set of hard examples.

3. Focal Loss

The Focal Loss is designed to address the one-stage ob-

ject detection scenario in which there is an extreme im-

balance between foreground and background classes during

training (e.g., 1:1000). We introduce the focal loss starting

from the cross entropy (CE) loss for binary classification1:

CE(p, y) =

�

− log(p)

if y = 1

− log(1 − p)

otherwise.

(1)

In the above y ∈ {±1} specifies the ground-truth class and

p ∈ [0*,* 1] is the model’s estimated probability for the class

with label y = 1. For notational convenience, we define pt:

pt =

�

p

if y = 1

1 − p

otherwise,

(2)

and rewrite CE(p, y) = CE(pt) = − log(pt).

The CE loss can be seen as the blue (top) curve in Fig-

ure 1. One notable property of this loss, which can be easily

seen in its plot, is that even examples that are easily clas-

sified (pt *≫ .*5) incur a loss with non-trivial magnitude.

When summed over a large number of easy examples, these

small loss values can overwhelm the rare class.

1Extending the focal loss to the multi-class case is straightforward and

works well; for simplicity we focus on the binary loss in this work.

3.1. Balanced Cross Entropy

A common method for addressing class imbalance is to

introduce a weighting factor α ∈ [0*,* 1] for class 1 and 1*−α*

for class *−*1. In practice α may be set by inverse class fre-

quency or treated as a hyperparameter to set by cross valida-

tion. For notational convenience, we define αt analogously

to how we defined pt. We write the α-balanced CE loss as:

CE(pt) = −αt log(pt).

(3)

This loss is a simple extension to CE that we consider as an

experimental baseline for our proposed focal loss.

3.2. Focal Loss Definition

As our experiments will show, the large class imbalance

encountered during training of dense detectors overwhelms

the cross entropy loss. Easily classified negatives comprise

the majority of the loss and dominate the gradient. While

α balances the importance of positive/negative examples, it

does not differentiate between easy/hard examples. Instead,

we propose to reshape the loss function to down-weight

easy examples and thus focus training on hard negatives.

More formally, we propose to add a modulating factor

(1 − pt)γ to the cross entropy loss, with tunable focusing

parameter γ ≥ 0. We define the focal loss as:

FL(pt) = −(1 − pt)γ log(pt).

(4)

The focal loss is visualized for several values of γ ∈

[0*,* 5] in Figure 1. We note two properties of the focal loss.

(1) When an example is misclassified and pt is small, the

modulating factor is near 1 and the loss is unaffected. As

pt → 1, the factor goes to 0 and the loss for well-classified

examples is down-weighted. (2) The focusing parameter γ

smoothly adjusts the rate at which easy examples are down-

weighted. When γ = 0, FL is equivalent to CE, and as γ is

increased the effect of the modulating factor is likewise in-

creased (we found γ = 2 to work best in our experiments).

Intuitively, the modulating factor reduces the loss contri-

bution from easy examples and extends the range in which

an example receives low loss. For instance, with γ = 2, an

example classified with pt = 0*.9 would have 100×* lower

loss compared with CE and with pt ≈ 0*.*968 it would have

1000*×* lower loss. This in turn increases the importance

of correcting misclassified examples (whose loss is scaled

down by at most 4*×* for pt *≤ .*5 and γ = 2).

In practice we use an α-balanced variant of the focal loss:

FL(pt) = −αt(1 − pt)γ log(pt).

(5)

We adopt this form in our experiments as it yields slightly

improved accuracy over the non-α-balanced form. Finally,

we note that the implementation of the loss layer combines

the sigmoid operation for computing p with the loss com-

putation, resulting in greater numerical stability.

3

While in our main experimental results we use the focal

loss definition above, its precise form is not crucial. In the

appendix we consider other instantiations of the focal loss

and demonstrate that these can be equally effective.

3.3. Class Imbalance and Model Initialization

Binary classification models are by default initialized to

have equal probability of outputting either y = *−*1 or 1.

Under such an initialization, in the presence of class imbal-

ance, the loss due to the frequent class can dominate total

loss and cause instability in early training. To counter this,

we introduce the concept of a ‘prior’ for the value of p es-

timated by the model for the rare class (foreground) at the

start of training. We denote the prior by π and set it so that

the model’s estimated p for examples of the rare class is low,

e.g. 0*.*01. We note that this is a change in model initializa-

tion (see *§*4.1) and not of the loss function. We found this

to improve training stability for both the cross entropy and

focal loss in the case of heavy class imbalance.

3.4. Class Imbalance and Two-stage Detectors

Two-stage detectors are often trained with the cross en-

tropy loss without use of α-balancing or our proposed loss.

Instead, they address class imbalance through two mech-

anisms: (1) a two-stage cascade and (2) biased minibatch

sampling.

The first cascade stage is an object proposal

mechanism [35, 24, 28] that reduces the nearly infinite set

of possible object locations down to one or two thousand.

Importantly, the selected proposals are not random, but are

likely to correspond to true object locations, which removes

the vast majority of easy negatives. When training the sec-

ond stage, biased sampling is typically used to construct

minibatches that contain, for instance, a 1:3 ratio of posi-

tive to negative examples. This ratio is like an implicit α-

balancing factor that is implemented via sampling. Our pro-

posed focal loss is designed to address these mechanisms in

a one-stage detection system directly via the loss function.

4. RetinaNet Detector

RetinaNet is a single, unified network composed of a

backbone network and two task-specific subnetworks. The

backbone is responsible for computing a convolutional fea-

ture map over an entire input image and is an off-the-self

convolutional network. The first subnet performs convo-

lutional object classification on the backbone’s output; the

second subnet performs convolutional bounding box regres-

sion. The two subnetworks feature a simple design that we

propose specifically for one-stage, dense detection, see Fig-

ure 3. While there are many possible choices for the details

of these components, most design parameters are not partic-

ularly sensitive to exact values as shown in the experiments.

We describe each component of RetinaNet next.

Feature Pyramid Network Backbone: We adopt the Fea-

ture Pyramid Network (FPN) from [20] as the backbone

network for RetinaNet.

In brief, FPN augments a stan-

dard convolutional network with a top-down pathway and

lateral connections so the network efficiently constructs a

rich, multi-scale feature pyramid from a single resolution

input image, see Figure 3(a)-(b). Each level of the pyramid

can be used for detecting objects at a different scale. FPN

improves multi-scale predictions from fully convolutional

networks (FCN) [23], as shown by its gains for RPN [28]

and DeepMask-style proposals [24], as well at two-stage

detectors such as Fast R-CNN [10] or Mask R-CNN [14].

Following [20], we build FPN on top of the ResNet ar-

chitecture [16].

We construc ...

Detectron Model Zoo and Baselines

Introduction

This file documents a large collection of baselines trained with Detectron, primarily in late December 2017. We refer to these results as the 12_2017_baselines. All configurations for these baselines are located in the configs/12_2017_baselines directory. The tables below provide results and useful statistics about training and inference. Links to the trained models as well as their output are provided. Unless noted differently below (see "Notes" under each table), the following common settings are used for all training and inference runs.

Common Settings and Notes

• All baselines were run on Big Basin servers with 8 NVIDIA Tesla P100 GPU accelerators (with 16GB GPU memory, CUDA 8.0, and cuDNN 6.0.21).
• All baselines were trained using 8 GPU data parallel sync SGD with a minibatch size of either 8 or 16 images (see the im/gpu column).
• For training, only horizontal flipping data augmentation was used.
• For inference, no test-time augmentations (e.g., multiple scales, flipping) were used.
• All models were trained on the union of coco_2014_train and coco_2014_valminusminival, which is exactly equivalent to the recently defined coco_2017_train dataset.
• All models were tested on the coco_2014_minival dataset, which is exactly equivalent to the recently defined coco_2017_val dataset.
• Inference times are often expressed as "X + Y", in which X is time taken in reasonably well-optimized GPU code and Y is time taken in unoptimized CPU code. (The CPU code time could be reduced substantially with additional engineering.)
• Inference results for boxes, masks, and keypoints ("kps") are provided in the COCO json format.
• The model id column is provided for ease of reference.
• To check downloaded file integrity: for any download URL on this page, simply append .md5sum to the URL to download the file's md5 hash.
• All models and results below are on the COCO dataset.
• Baseline models and results for the Cityscapes dataset are coming soon!

Training Schedules

We use three training schedules, indicated by the lr schd column in the tables below.

• 1x: For minibatch size 16, this schedule starts at a LR of 0.02 and is decreased by a factor of * 0.1 after 60k and 80k iterations and finally terminates at 90k iterations. This schedules results in 12.17 epochs over the 118,287 images in coco_2014_train union coco_2014_valminusminival (or equivalently, coco_2017_train).
• 2x: Twice as long as the 1x schedule with the LR change points scaled proportionally.
• s1x ("stretched 1x"): This schedule scales the 1x schedule by roughly 1.44x, but also extends the duration of the first learning rate. With a minibatch size of 16, it reduces the LR by * 0.1 at 100k and 120k iterations, finally ending after 130k iterations.

All training schedules also use a 500 iteration linear learning rate warm up. When changing the minibatch size between 8 and 16 images, we adjust the number of SGD iterations and the base learning rate according to the principles outlined in our paper Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour.

License

All models available for download through this document are licensed under the Creative Commons Attribution-ShareAlike 3.0 license.

ImageNet Pretrained Models

The backbone models pretrained on ImageNet are available in the format used by Detectron. Unless otherwise noted, these models are trained on the standard ImageNet-1k dataset.

• R-50.pkl: converted copy of MSRA's original ResNet-50 model
• R-101.pkl: converted copy of MSRA's original ResNet-101 model
• X-101-64x4d.pkl: converted copy of FB's original ResNeXt-101-64x4d model trained with Torch7
• X-101-32x8d.pkl: ResNeXt-101-32x8d model trained with Caffe2 at FB
• X-152-32x8d-IN5k.pkl: ResNeXt-152-32x8d model trained on ImageNet-5k with Caffe2 at FB (see our ResNeXt paper for details on ImageNet-5k)

Log Files

Training and inference logs are available for most models in the model zoo.

Proposal, Box, and Mask Detection Baselines

RPN Proposal Baselines

        backbone         type lr
schd im/
gpu train
mem
(GB) train
time
(s/iter) train
time
total
(hr) inference
time
(s/im) box
AP mask
AP kp
AP prop.
AR model id download
links R-50-C4 RPN 1x 2 4.3 0.187 4.7 0.113 - - - 51.6 35998355 model | props: 1, 2, 3 R-50-FPN RPN 1x 2 6.4 0.416 10.4 0.080 - - - 57.2 35998814 model | props: 1, 2, 3 R-101-FPN RPN 1x 2 8.1 0.503 12.6 0.108 - - - 58.2 35998887 model | props: 1, 2, 3 X-101-64x4d-FPN RPN 1x 2 11.5 1.395 34.9 0.292 - - - 59.4 35998956 model | props: 1, 2, 3 X-101-32x8d-FPN RPN 1x 2 11.6 1.102 27.6 0.222 - - - 59.5 36760102 model | props: 1, 2, 3

Notes:

• Inference time only includes RPN proposal generation.
• "prop. AR" is proposal average recall at 1000 proposals per image.
• Proposal download links ("props"): "1" is coco_2014_train; "2" is coco_2014_valminusminival; and "3" is coco_2014_minival.

Fast & Mask R-CNN Baselines Using Precomputed RPN Proposals

        backbone         type lr
schd im/
gpu train
mem
(GB) train
time
(s/iter) train
time
total
(hr) inference
time
(s/im) box
AP mask
AP kp
AP prop.
AR model id download
links R-50-C4 Fast 1x 1 6.0 0.456 22.8 0.241 + 0.003 34.4 - - - 36224013 model | boxes R-50-C4 Fast 2x 1 6.0 0.453 45.3 0.241 + 0.003 35.6 - - - 36224046 model | boxes R-50-FPN Fast 1x 2 6.0 0.285 7.1 0.076 + 0.004 36.4 - - - 36225147 model | boxes R-50-FPN Fast 2x 2 6.0 0.287 14.4 0.077 + 0.004 36.8 - - - 36225249 model | boxes R-101-FPN Fast 1x 2 7.7 0.448 11.2 0.102 + 0.003 38.5 - - - 36228880 model | boxes R-101-FPN Fast 2x 2 7.7 0.449 22.5 0.103 + 0.004 39.0 - - - 36228933 model | boxes X-101-64x4d-FPN Fast 1x 1 6.3 0.994 49.7 0.292 + 0.003 40.4 - - - 36226250 model | boxes X-101-64x4d-FPN Fast 2x 1 6.3 0.980 98.0 0.291 + 0.003 39.8 - - - 36226326 model | boxes X-101-32x8d-FPN Fast 1x 1 6.4 0.721 36.1 0.217 + 0.003 40.6 - - - 37119777 model | boxes X-101-32x8d-FPN Fast 2x 1 6.4 0.720 72.0 0.217 + 0.003 39.7 - - - 37121469 model | boxes R-50-C4 Mask 1x 1 6.4 0.466 23.3 0.252 + 0.020 35.5 31.3 - - 36224121 model | boxes | masks R-50-C4 Mask 2x 1 6.4 0.464 46.4 0.253 + 0.019 36.9 32.5 - - 36224151 model | boxes | masks R-50-FPN Mask 1x 2 7.9 0.377 9.4 0.082 + 0.019 37.3 33.7 - - 36225401 model | boxes | masks R-50-FPN Mask 2x 2 7.9 0.377 18.9 0.083 + 0.018 37.7 34.0 - - 36225732 model | boxes | masks R-101-FPN Mask 1x 2 9.6 0.539 13.5 0.111 + 0.018 39.4 35.6 - - 36229407 model | boxes | masks R-101-FPN Mask 2x 2 9.6 0.537 26.9 0.109 + 0.016 40.0 35.9 - - 36229740 model | boxes | masks X-101-64x4d-FPN Mask 1x 1 7.3 1.036 51.8 0.292 + 0.016 41.3 37.0 - - 36226382 model | boxes | [masks](https://dl.fbaipublicfiles.com/detectron/36226382/12_2017_baselines/mask_rcnn_X-101-64x4d-FPN_1x.yaml.08_56_59.rUCejrBN/output/test/coco_2014_minival/generalized_rcnn/segment ...

Detectron Model Zoo and Baselines

Introduction

This file documents a large collection of baselines trained with Detectron, primarily in late December 2017. We refer to these results as the 12_2017_baselines. All configurations for these baselines are located in the configs/12_2017_baselines directory. The tables below provide results and useful statistics about training and inference. Links to the trained models as well as their output are provided. Unless noted differently below (see "Notes" under each table), the following common settings are used for all training and inference runs.

Common Settings and Notes

• All baselines were run on Big Basin servers with 8 NVIDIA Tesla P100 GPU accelerators (with 16GB GPU memory, CUDA 8.0, and cuDNN 6.0.21).
• All baselines were trained using 8 GPU data parallel sync SGD with a minibatch size of either 8 or 16 images (see the im/gpu column).
• For training, only horizontal flipping data augmentation was used.
• For inference, no test-time augmentations (e.g., multiple scales, flipping) were used.
• All models were trained on the union of coco_2014_train and coco_2014_valminusminival, which is exactly equivalent to the recently defined coco_2017_train dataset.
• All models were tested on the coco_2014_minival dataset, which is exactly equivalent to the recently defined coco_2017_val dataset.
• Inference times are often expressed as "X + Y", in which X is time taken in reasonably well-optimized GPU code and Y is time taken in unoptimized CPU code. (The CPU code time could be reduced substantially with additional engineering.)
• Inference results for boxes, masks, and keypoints ("kps") are provided in the COCO json format.
• The model id column is provided for ease of reference.
• To check downloaded file integrity: for any download URL on this page, simply append .md5sum to the URL to download the file's md5 hash.
• All models and results below are on the COCO dataset.
• Baseline models and results for the Cityscapes dataset are coming soon!

Training Schedules

We use three training schedules, indicated by the lr schd column in the tables below.

• 1x: For minibatch size 16, this schedule starts at a LR of 0.02 and is decreased by a factor of * 0.1 after 60k and 80k iterations and finally terminates at 90k iterations. This schedules results in 12.17 epochs over the 118,287 images in coco_2014_train union coco_2014_valminusminival (or equivalently, coco_2017_train).
• 2x: Twice as long as the 1x schedule with the LR change points scaled proportionally.
• s1x ("stretched 1x"): This schedule scales the 1x schedule by roughly 1.44x, but also extends the duration of the first learning rate. With a minibatch size of 16, it reduces the LR by * 0.1 at 100k and 120k iterations, finally ending after 130k iterations.

All training schedules also use a 500 iteration linear learning rate warm up. When changing the minibatch size between 8 and 16 images, we adjust the number of SGD iterations and the base learning rate according to the principles outlined in our paper Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour.

License

All models available for download through this document are licensed under the Creative Commons Attribution-ShareAlike 3.0 license.

ImageNet Pretrained Models

The backbone models pretrained on ImageNet are available in the format used by Detectron. Unless otherwise noted, these models are trained on the standard ImageNet-1k dataset.

• R-50.pkl: converted copy of MSRA's original ResNet-50 model
• R-101.pkl: converted copy of MSRA's original ResNet-101 model
• X-101-64x4d.pkl: converted copy of FB's original ResNeXt-101-64x4d model trained with Torch7
• X-101-32x8d.pkl: ResNeXt-101-32x8d model trained with Caffe2 at FB
• X-152-32x8d-IN5k.pkl: ResNeXt-152-32x8d model trained on ImageNet-5k with Caffe2 at FB (see our ResNeXt paper for details on ImageNet-5k)

Log Files

Training and inference logs are available for most models in the model zoo.

Proposal, Box, and Mask Detection Baselines

RPN Proposal Baselines

        backbone         type lr
schd im/
gpu train
mem
(GB) train
time
(s/iter) train
time
total
(hr) inference
time
(s/im) box
AP mask
AP kp
AP prop.
AR model id download
links R-50-C4 RPN 1x 2 4.3 0.187 4.7 0.113 - - - 51.6 35998355 model | props: 1, 2, 3 R-50-FPN RPN 1x 2 6.4 0.416 10.4 0.080 - - - 57.2 35998814 model | props: 1, 2, 3 R-101-FPN RPN 1x 2 8.1 0.503 12.6 0.108 - - - 58.2 35998887 model | props: 1, 2, 3 X-101-64x4d-FPN RPN 1x 2 11.5 1.395 34.9 0.292 - - - 59.4 35998956 model | props: 1, 2, 3 X-101-32x8d-FPN RPN 1x 2 11.6 1.102 27.6 0.222 - - - 59.5 36760102 model | props: 1, 2, 3

Notes:

• Inference time only includes RPN proposal generation.
• "prop. AR" is proposal average recall at 1000 proposals per image.
• Proposal download links ("props"): "1" is coco_2014_train; "2" is coco_2014_valminusminival; and "3" is coco_2014_minival.

Fast & Mask R-CNN Baselines Using Precomputed RPN Proposals

        backbone         type lr
schd im/
gpu train
mem
(GB) train
time
(s/iter) train
time
total
(hr) inference
time
(s/im) box
AP mask
AP kp
AP prop.
AR model id download
links R-50-C4 Fast 1x 1 6.0 0.456 22.8 0.241 + 0.003 34.4 - - - 36224013 model | boxes R-50-C4 Fast 2x 1 6.0 0.453 45.3 0.241 + 0.003 35.6 - - - 36224046 model | boxes R-50-FPN Fast 1x 2 6.0 0.285 7.1 0.076 + 0.004 36.4 - - - 36225147 model | boxes R-50-FPN Fast 2x 2 6.0 0.287 14.4 0.077 + 0.004 36.8 - - - 36225249 model | boxes R-101-FPN Fast 1x 2 7.7 0.448 11.2 0.102 + 0.003 38.5 - - - 36228880 model | boxes R-101-FPN Fast 2x 2 7.7 0.449 22.5 0.103 + 0.004 39.0 - - - 36228933 model | boxes X-101-64x4d-FPN Fast 1x 1 6.3 0.994 49.7 0.292 + 0.003 40.4 - - - 36226250 model | boxes X-101-64x4d-FPN Fast 2x 1 6.3 0.980 98.0 0.291 + 0.003 39.8 - - - 36226326 model | boxes X-101-32x8d-FPN Fast 1x 1 6.4 0.721 36.1 0.217 + 0.003 40.6 - - - 37119777 model | boxes X-101-32x8d-FPN Fast 2x 1 6.4 0.720 72.0 0.217 + 0.003 39.7 - - - 37121469 model | boxes R-50-C4 Mask 1x 1 6.4 0.466 23.3 0.252 + 0.020 35.5 31.3 - - 36224121 model | boxes | masks R-50-C4 Mask 2x 1 6.4 0.464 46.4 0.253 + 0.019 36.9 32.5 - - 36224151 model | boxes | masks R-50-FPN Mask 1x 2 7.9 0.377 9.4 0.082 + 0.019 37.3 33.7 - - 36225401 model | boxes | masks R-50-FPN Mask 2x 2 7.9 0.377 18.9 0.083 + 0.018 37.7 34.0 - - 36225732 model | boxes | masks R-101-FPN Mask 1x 2 9.6 0.539 13.5 0.111 + 0.018 39.4 35.6 - - 36229407 model | boxes | masks R-101-FPN Mask 2x 2 9.6 0.537 26.9 0.109 + 0.016 40.0 35.9 - - 36229740 model | boxes | masks X-101-64x4d-FPN Mask 1x 1 7.3 1.036 51.8 0.292 + 0.016 41.3 37.0 - - 36226382 model | boxes | [masks](https://dl.fbaipublicfiles.com/detectron/36226382/12_2017_baselines/mask_rcnn_X-101-64x4d-FPN_1x.yaml.08_56_59.rUCejrBN/output/test/coco_2014_minival/generalized_rcnn/segment ...

摘要

这个网页展示了Detectron模型库和基线，主要记录了2017年12月训练的大量基线模型。所有配置文件位于configs/12_2017_baselines目录。网页详细介绍了训练和推理的通用设置，包括使用8个NVIDIA Tesla P100 GPU进行训练，采用8 GPU数据并行同步SGD，批量大小为8或16图像。文档提供了三种训练计划（1x、2x和s1x），并列出了预训练的ImageNet骨干网络模型，包括ResNet-50/101和ResNeXt变体。网页主要展示了RPN提议基线、使用预计算RPN提议的Fast & Mask R-CNN基线的详细性能数据，包括训练内存、时间、推理时间和各种性能指标（box AP、mask AP等）。所有模型都提供了下载链接和相应的输出结果。

与问题相关的信息

根据网页内容，我没有找到任何使用ResNeXt作为骨干网络的RetinaNet模型。网页中展示了多种模型类型，包括RPN、Fast R-CNN和Mask R-CNN，并且确实有使用ResNeXt作为骨干网络的模型（如X-101-64x4d-FPN和X-101-32x8d-FPN），但没有任何RetinaNet模型与ResNeXt骨干网络的组合。

网页中列出的预训练ImageNet模型中包含了ResNeXt变体：

• X-101-64x4d.pkl：转换自FB原始的使用Torch7训练的ResNeXt-101-64x4d模型
• X-101-32x8d.pkl：在FB使用Caffe2训练的ResNeXt-101-32x8d模型
• X-152-32x8d-IN5k.pkl：在FB使用Caffe2训练的ResNeXt-152-32x8d模型（在ImageNet-5k上训练）

但在展示的模型表格中，没有任何将这些ResNeXt骨干网络用于RetinaNet架构的模型。表格中只展示了RPN、Fast R-CNN和Mask R-CNN类型的模型。

相关网页链接

1. Detectron Model Zoo and Baselines - 当前网页的主链接
2. R-50.pkl - MSRA的原始ResNet-50模型的转换副本
3. R-101.pkl - MSRA的原始ResNet-101模型的转换副本
4. X-101-64x4d.pkl - FB原始ResNeXt-101-64x4d模型的转换副本
5. X-101-32x8d.pkl - FB使用Caffe2训练的ResNeXt-101-32x8d模型
6. X-152-32x8d-IN5k.pkl - FB使用Caffe2训练的ResNeXt-152-32x8d模型
7. Training and inference logs - 大多数模型库中模型的训练和推理日志
8. ResNeXt paper - 关于ImageNet-5k的详细信息
9. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour - 关于改变批量大小时调整SGD迭代次数和基础学习率的原则

相关图片

网页中没有提供与RetinaNet或ResNeXt相关的图片，只有文本内容和链接。

摘要

《Focal Loss for Dense Object Detection》论文介绍了一种新型损失函数——Focal Loss，用于解决一阶段目标检测器中的类别不平衡问题。传统上，两阶段检测器（如R-CNN系列）在准确性上优于一阶段检测器，主要因为两阶段方法能通过级联过程和采样启发式方法处理前景-背景类别极度不平衡的问题。论文提出的Focal Loss通过动态缩放因子(1-pt)^γ修改标准交叉熵损失，自动降低易分类样本的权重，使训练聚焦于难分类样本。基于此损失函数，作者设计了名为RetinaNet的一阶段检测器，它采用ResNet-FPN作为骨干网络，结合分类和边界框回归子网络。实验表明，RetinaNet在COCO数据集上达到39.1 AP，以5 FPS的速度运行，首次使一阶段检测器在准确性上超越了所有现有的两阶段检测器，同时保持较高的检测速度。

与问题相关的信息

根据网页内容，原始RetinaNet模型使用的骨干网络是Feature Pyramid Network (FPN)，它建立在ResNet架构之上。具体来说，论文中提到："Following [20], we build FPN on top of the ResNet architecture [16]"（我们按照[20]的方法，在ResNet架构上构建FPN）。

论文中明确指出他们的最佳模型是基于ResNet-101-FPN骨干网络的，该模型在COCO test-dev上达到了39.1 AP，运行速度为5 fps。此外，论文图2还展示了使用ResNet-50-FPN和ResNet-101-FPN作为骨干网络的RetinaNet变体在不同输入尺寸(400-800像素)下的性能对比。

RetinaNet的设计特点包括高效的网络内特征金字塔和锚框的使用，借鉴了多种最新的想法，包括[22, 6, 28, 20]等工作。论文强调，RetinaNet取得优异结果不是基于网络设计的创新，而是归功于他们提出的新型Focal Loss损失函数。

相关网页链接

网页中提到的链接：

1. https://github.com/facebookresearch/Detectron - 论文代码库链接，上下文是"Code is at: https://github.com/facebookresearch/Detectron."

相关图片

1. 图片1

   • Title: Focal Loss与标准交叉熵损失对比图
   • Content: 展示了Focal Loss添加(1-pt)^γ因子到标准交叉熵损失的效果，不同γ值(0, 0.5, 1, 2, 5)下损失函数的变化曲线
   • Source: 论文作者(Lin et al.)
   • Link: 未知

2. 图片2

   • Title: 速度(ms)与准确性(AP)在COCO test-dev上的对比
   • Content: 展示了RetinaNet与其他检测器在速度和准确性上的对比，RetinaNet使用ResNet-50-FPN和ResNet-101-FPN在五种尺度(400-800像素)下的性能
   • Source: 论文作者(Lin et al.)
   • Link: 未知

3. 图片3

   • Title: RetinaNet架构图
   • Content: 显示RetinaNet的组成部分，包括FPN骨干网络和两个任务特定子网络(分类和边界框回归)
   • Source: 论文作者(Lin et al.)
   • Link: 未知