J/ApJ/747/L41 Solar flares probabilities (Bloomfield+, 2012)
Toward reliable benchmarking of solar flare forecasting methods.
Bloomfield D.S., Higgins P.A., McAteer R.T.J., Gallagher P.T.
<Astrophys. J., 747, L41 (2012)>
=2012ApJ...747L..41B 2012ApJ...747L..41B
ADC_Keywords: Sun
Keywords: magnetic fields - Sun: activity - Sun: flares - sunspots
Abstract:
Solar flares occur in complex sunspot groups, but it remains unclear
how the probability of producing a flare of a given magnitude relates
to the characteristics of the sunspot group. Here, we use
Geostationary Operational Environmental Satellite X-ray flares and
McIntosh group classifications from solar cycles 21 and 22 to
calculate average flare rates for each McIntosh class and use these to
determine Poisson probabilities for different flare magnitudes.
Forecast verification measures are studied to find optimum thresholds
to convert Poisson flare probabilities into yes/no predictions of
cycle 23 flares. A case is presented to adopt the true skill statistic
(TSS) as a standard for forecast comparison over the commonly used
Heidke skill score (HSS). In predicting flares over 24 hr, the maximum
values of TSS achieved are 0.44 (C-class), 0.53 (M-class), 0.74
(X-class), 0.54 (≥M1.0), and 0.46 (≥C1.0). The maximum values of HSS
are 0.38 (C-class), 0.27 (M-class), 0.14 (X-class), 0.28 (≥M1.0), and
0.41 (≥C1.0). These show that Poisson probabilities perform
comparably to some more complex prediction systems, but the overall
inaccuracy highlights the problem with using average values to
represent flaring rate distributions.
Description:
In order to facilitate the calculation of flare probabilities, we
obtained historical flare rates for each McIntosh class from two
locations that share the same data source. The National Oceanic and
Atmospheric Administration (NOAA) Space Weather Prediction Center
(SWPC) provided total numbers of Geostationary Operational
Environmental Satellite (GOES) C-, M-, and X-class flares and the
originating active regions (ARs) for each McIntosh classification over
1988 December 1 to 1996 June 30 (C. C. Balch 2011, private
communication). Additional M- and X-class flares and McIntosh class
numbers were taken from Kildahl (1980, Solar-Terrestrial Predictions
Proceedings, Vol. 3, ed. R. F. Donnelly (Boulder: U.S. Dept. of
Commerce), 166) over 1969-1976, but relate to the same data source
(i.e., NOAA-collated ground-based AR observations and GOES flare
events). These were included to increase the rare M- and X-class
samples so that the rates were more statistically significant.
The AR and flare data that are used for testing were gathered from the
online archives of NOAA/SWPC
(http://www.swpc.noaa.gov/ftpdir/warehouse/). McIntosh classes of
regions that have predictions issued and tested were taken from the
daily NOAA Solar Region Summary files over 1996 August 1 to 2010
December 31. In this work, each daily record of an NOAA region was
treated as an individual measurement, yielding 22276 AR samples.
File Summary:
--------------------------------------------------------------------------------
FileName Lrecl Records Explanations
--------------------------------------------------------------------------------
ReadMe 80 . This file
table3.dat 198 101 Flare forecast contingency table and skill score
dependence on threshold Poisson probability
--------------------------------------------------------------------------------
See also:
J/A+A/304/563 : Cool X-ray flares of Sun with GOES (Phillips+, 1995)
Byte-by-byte Description of file: table3.dat
--------------------------------------------------------------------------------
Bytes Format Units Label Explanations
--------------------------------------------------------------------------------
1- 3 I3 % Tpr [0/100] Threshold probability (percent) (2)
5- 8 I4 --- C-TP [0/3667] C-class true positives
10- 13 I4 --- C-FN [0/3667] C-class false negatives
15- 19 I5 --- C-FP [0/18609] C-class false positives
21- 25 I5 --- C-TN [0/18609] C-class true negatives
27- 32 F6.3 --- C-HSS [-0.002/0.384] C-class Heidke skill score (1)
34- 39 F6.3 --- C-TSS [-0.007/0.443] C-class true skill statistic (1)
41- 47 F7.2 --- C-FN/FP ? C-class failure ratio FN/FP (3)
49- 51 I3 --- M-TP [0/810] M-class true positives
53- 55 I3 --- M-FN [0/810] M-class false negatives
57- 61 I5 --- M-FP [0/21466] M-class false positives
63- 67 I5 --- M-TN [0/21466] M-class true negatives
69- 73 F5.3 --- M-HSS [0/0.273] M-class Heidke skill score (1)
75- 79 F5.3 --- M-TSS [0/0.526] M-class true skill statistic (1)
81- 84 F4.2 --- M-FN/FP ? M-class failure ratio FN/FP (3)
86- 87 I2 --- X-TP [0/92] X-class true positives
89- 90 I2 --- X-FN [0/92] X-class false negatives
92- 96 I5 --- X-FP [0/22184] X-class false positives
98-102 I5 --- X-TN [0/22184] X-class true negatives
104-108 F5.3 --- X-HSS [0/0.142] X-class Heidke skill score (1)
110-114 F5.3 --- X-TSS [0/0.74] X-class true skill statistic (1)
116-119 F4.2 --- X-FN/FP ? X-class failure ratio FN/FP (3)
121-123 I3 --- MX-TP [0/858] Class ≥M1.0 true positives
125-127 I3 --- MX-FN [0/858] Class ≥M1.0 false negatives
129-133 I5 --- MX-FP [0/21418] Class ≥M1.0 false positives
135-139 I5 --- MX-TN [0/21418] Class ≥M1.0 true negatives
141-145 F5.3 --- MX-HSS [0/0.28] Class ≥M1.0 Heidke skill score (1)
147-151 F5.3 --- MX-TSS [0/0.539] Class ≥M1.0 true skill statistic (1)
153-156 F4.2 --- MX-FN/FP ? Class ≥M1.0 failure ratio FN/FP (3)
158-161 I4 --- CX-TP [0/3912] Class ≥C1.0 true positives
163-166 I4 --- CX-FN [0/3912] Class ≥C1.0 false negatives
168-172 I5 --- CX-FP [0/18364] Class ≥C1.0 false positives
174-178 I5 --- CX-TN [0/18364] Class ≥C1.0 true negatives
180-185 F6.3 --- CX-HSS [-0.002/0.407] Class ≥C1.0 HSS (1)
187-192 F6.3 --- CX-TSS [-0.005/0.456] Class ≥C1.0 TSS (1)
194-198 F5.2 --- CX-FN/FP ? Class ≥C1.0 failure ratio FN/FP (3)
--------------------------------------------------------------------------------
Note (1): Numerous skill scores exist to quantify the performance of forecasts,
computed from the numbers of flares observed and predicted, classified in:
TP = number of "true positives" (flare predicted and observed),
FP = number of "false positives" (flare predicted and none observed),
TN = number of "true negatives" (no flare predicted and none observed),
FN = number of "false negatives" (no flare predicted but one oberved).
* the Heidke (1926, Geogr. Ann. Stockh., 8, 301) skill score (HSS),
is most frequently used in flare forecasting (e.g., Barnes & Leka
2008ApJ...688L.107B 2008ApJ...688L.107B), defined by Equantion (1):
HSS=(2[(TPxTN)-(FNxFP)])/((TP+FN)(FN+TN)+(TP+FP)(FP+TN))
* the Hanssen & Kuipers (1965, Meded. Verh., 81, 2) discriminant,
known as the true skill statistic (TSS):
TSS=(TP/(TP+FN))-(FP/(FP+TN)). See equation (2).
Note (2): the threshold of the prediction varies from 0 to 100%, predicting
"no flare" for values below the threshold and "flare" for those at or
above the threshold.
Note (3): A blank field indicates an infinite ratio.
--------------------------------------------------------------------------------
History:
From electronic version of the journal
(End) Greg Schwarz [AAS], Emmanuelle Perret [CDS] 07-Oct-2013