From brideout at haystack.mit.edu Fri Oct 20 11:53:15 2006 From: brideout at haystack.mit.edu (Bill Rideout) Date: Mon Oct 23 17:53:13 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal Message-ID: <4538F0EB.40003@haystack.mit.edu> Phil Erickson has suggested we modify out definition of the F10.7 Multiday average (FBAR). Right now this is an 81 day average, where the 81 days the center day, 40 days in the past, and 40 days in the future. The problem with this definition is that this parameter is never available for any day less than 40 days in the past. For example, since MSIS uses FBAR as an input, this means that MSIS also cannot be run for any day less than 40 days in the past. Phil has suggested that for any day less than 40 days in the past, we calculate FBAR as an average over the most recent 81 days. From the Madrigal point-of-view, this means that every time we reload geophysical data, the FBAR values for the latest 40 days might change slightly. At present, FBAR is not loaded into Madrigal for days less than 40 days ago. We would document this FBAR definition on the Madrigal help pages. Any thoughts? Bill Rideout From shunrong at haystack.mit.edu Fri Oct 20 12:30:33 2006 From: shunrong at haystack.mit.edu (Shunrong Zhang) Date: Mon Oct 23 17:53:13 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal In-Reply-To: <4538F0EB.40003@haystack.mit.edu> References: <4538F0EB.40003@haystack.mit.edu> Message-ID: <4538F9A9.7070408@haystack.mit.edu> I think Fbar definition is somewhat universally adopted. If we change the definition as described, the Fbar for the last 40 days will actually repeat the true Fbar values from 80 days before to 40 days before. Today's Fbar will be the true value for 41 days back. As there is a 28-day (not 41 days!) periodicity in the sloar flux, there might be some problems. There might be also a big jump in the time series. In my opinion, there are two options: (1) when data are not available for the Fbar calculation, use the data for 28 (or so) days back. (2) invent a new F107 index, for example, Fdbar, that always give F107 averages in the past 81 days. But we'll need to change our codes for calling MSIS model so that when Fbar isn't available we use Fdbar. I think option (2) is better. Shunrong Bill Rideout wrote: > Phil Erickson has suggested we modify out definition of the F10.7 > Multiday average (FBAR). Right now this is an 81 day average, where > the 81 days the center day, 40 days in the past, and 40 days in the > future. The problem with this definition is that this parameter is > never available for any day less than 40 days in the past. For > example, since MSIS uses FBAR as an input, this means that MSIS also > cannot be run for any day less than 40 days in the past. > Phil has suggested that for any day less than 40 days in the past, we > calculate FBAR as an average over the most recent 81 days. From the > Madrigal point-of-view, this means that every time we reload > geophysical data, the FBAR values for the latest 40 days might change > slightly. At present, FBAR is not loaded into Madrigal for days less > than 40 days ago. > > We would document this FBAR definition on the Madrigal help pages. > Any thoughts? > > Bill Rideout > _______________________________________________ > OpenMadrigal-developers mailing list > OpenMadrigal-developers@openmadrigal.org > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > !DSPAM:4538f129202422102912529! -- _____________________________________________________________ Shunrong Zhang (shunrong@haystack.mit.edu) http://www.haystack.mit.edu/~shunrong/ MIT Haystack Observatory, Route 40, Westford, MA 01886, USA Phone: 781-981-5725 FAX: 781-981-5766 From jmh at haystack.mit.edu Fri Oct 20 14:38:53 2006 From: jmh at haystack.mit.edu (John Holt) Date: Mon Oct 23 17:53:14 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal Message-ID: <200610201838.k9KIcrI01199@hyperion.haystack.edu> Phil's suggestion seems reasonable. In the context of a model, the definition of F10.7bar as a centered average is an absurdity, obviously. In the "old" days we always used the 81 days ending with the day of interest, but at some point it apparently became more common to forget causality and use a centered average (I have seen 27 days, 54 days, 81 days 90 days and one year) and a variety of filter functions causal and not. Unfortunately, in my view, causal filters fell out of favor. Had they not, we would not have had this problem and would have been consistent with the laws of physics as well, which is no small thing. I suspect, by the way, that using the 81 days ending with the day of interest will tend to give a somewhat better prediction than using a centered average, even though the model was computed using a centered average. John ________________________________________________________________________ | | | John M. Holt tel: 781-981-5625 | | Principal Research Scientist fax: 781-981-5766 | | MIT Haystack Observatory email: jmh@haystack.mit.edu | | Route 40 WWW: http://www.haystack.edu/ | | Westford, MA 01886 | | USA | |________________ Ni h-eibhneas gan chlainn domhnaill _________________| > Date: Fri, 20 Oct 2006 11:53:15 -0400 > From: Bill Rideout > User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060417 > X-Accept-Language: en-us, en > To: openmadrigal-developers@openmadrigal.org > Content-Transfer-Encoding: 7bit > Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal > X-BeenThere: openmadrigal-developers@openmadrigal.org > X-Mailman-Version: 2.1.5 > List-Id: Open Madrigal developers list > List-Unsubscribe: , > List-Archive: > List-Post: > List-Help: > List-Subscribe: , > X-DSPAM-Result: Innocent > X-DSPAM-Probability: 0.000000 > X-DSPAM-Signature: 4538f138202497825867 > > Phil Erickson has suggested we modify out definition of the F10.7 > Multiday average (FBAR). Right now this is an 81 day average, where the > 81 days the center day, 40 days in the past, and 40 days in the future. > The problem with this definition is that this parameter is never > available for any day less than 40 days in the past. For example, since > MSIS uses FBAR as an input, this means that MSIS also cannot be run for > any day less than 40 days in the past. > > Phil has suggested that for any day less than 40 days in the past, we > calculate FBAR as an average over the most recent 81 days. From the > Madrigal point-of-view, this means that every time we reload geophysical > data, the FBAR values for the latest 40 days might change slightly. At > present, FBAR is not loaded into Madrigal for days less than 40 days ago. > > We would document this FBAR definition on the Madrigal help pages. Any > thoughts? > > Bill Rideout > _______________________________________________ > OpenMadrigal-developers mailing list > OpenMadrigal-developers@openmadrigal.org > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > !DSPAM:4538f138202497825867! From amg at haystack.mit.edu Fri Oct 20 14:50:42 2006 From: amg at haystack.mit.edu (Anne Gorczyca) Date: Mon Oct 23 17:53:14 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal In-Reply-To: <200610201838.k9KIcrI01199@hyperion.haystack.edu> References: <200610201838.k9KIcrI01199@hyperion.haystack.edu> Message-ID: <200610201450.42016.amg@haystack.mit.edu> On Friday 20 October 2006 02:38 pm, John Holt wrote: > |________________ Ni h-eibhneas gan chlainn domhnaill _________________| OK ... I understood everything up to this line ... Anne From shunrong at haystack.mit.edu Fri Oct 20 15:07:17 2006 From: shunrong at haystack.mit.edu (Shunrong Zhang) Date: Mon Oct 23 17:53:15 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal In-Reply-To: <200610201838.k9KIcrI01199@hyperion.haystack.edu> References: <200610201838.k9KIcrI01199@hyperion.haystack.edu> Message-ID: <45391E65.8060709@haystack.mit.edu> If we totally redefine Fbar to be for the last 81 days, that is fine; but I think what Phil suggested was "for any day less than 40 days in the past, we calculate FBAR as an average over the most recent 81 days. " That sounds to me that you still use the old definition for any day greater than day 40 days in the past. I wanted to say that there should not be two definitions for one parameter. From modeling point of view, I agree with John that "ending with the day of interest will tend to give a somewhat better prediction". That was why I suggested my option (2) for a new F107 index that is an average over the past 81 days and is always available. Shunrong John Holt wrote: > Phil's suggestion seems reasonable. In the context of a model, the > definition of F10.7bar as a centered average is an absurdity, > obviously. In the "old" days we always used the 81 days ending with the > day of interest, but at some point it apparently became more common to > forget causality and use a centered average (I have seen 27 days, 54 > days, 81 days 90 days and one year) and a variety of filter functions > causal and not. Unfortunately, in my view, causal filters fell out of > favor. Had they not, we would not have had this problem and would have > been consistent with the laws of physics as well, which is no small > thing. I suspect, by the way, that using the 81 days ending with the > day of interest will tend to give a somewhat better prediction than > using a centered average, even though the model was computed using a > centered average. > > John > > ________________________________________________________________________ > | | > | John M. Holt tel: 781-981-5625 | > | Principal Research Scientist fax: 781-981-5766 | > | MIT Haystack Observatory email: jmh@haystack.mit.edu | > | Route 40 WWW: http://www.haystack.edu/ | > | Westford, MA 01886 | > | USA | > |________________ Ni h-eibhneas gan chlainn domhnaill _________________| > > > >> Date: Fri, 20 Oct 2006 11:53:15 -0400 >> From: Bill Rideout >> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060417 >> X-Accept-Language: en-us, en >> To: openmadrigal-developers@openmadrigal.org >> Content-Transfer-Encoding: 7bit >> Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal >> X-BeenThere: openmadrigal-developers@openmadrigal.org >> X-Mailman-Version: 2.1.5 >> List-Id: Open Madrigal developers list >> > > >> List-Unsubscribe: >> > , > > >> List-Archive: >> List-Post: >> List-Help: >> > > >> List-Subscribe: >> > , > > >> X-DSPAM-Result: Innocent >> X-DSPAM-Probability: 0.000000 >> X-DSPAM-Signature: 4538f138202497825867 >> >> Phil Erickson has suggested we modify out definition of the F10.7 >> Multiday average (FBAR). Right now this is an 81 day average, where the >> 81 days the center day, 40 days in the past, and 40 days in the future. >> The problem with this definition is that this parameter is never >> available for any day less than 40 days in the past. For example, since >> MSIS uses FBAR as an input, this means that MSIS also cannot be run for >> any day less than 40 days in the past. >> >> Phil has suggested that for any day less than 40 days in the past, we >> calculate FBAR as an average over the most recent 81 days. From the >> Madrigal point-of-view, this means that every time we reload geophysical >> data, the FBAR values for the latest 40 days might change slightly. At >> present, FBAR is not loaded into Madrigal for days less than 40 days ago. >> >> We would document this FBAR definition on the Madrigal help pages. Any >> thoughts? >> >> Bill Rideout >> _______________________________________________ >> OpenMadrigal-developers mailing list >> OpenMadrigal-developers@openmadrigal.org >> http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers >> >> >> > > _______________________________________________ > OpenMadrigal-developers mailing list > OpenMadrigal-developers@openmadrigal.org > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > !DSPAM:453917f312891216122679! > -- _____________________________________________________________ Shunrong Zhang (shunrong@haystack.mit.edu) http://www.haystack.mit.edu/~shunrong/ MIT Haystack Observatory, Route 40, Westford, MA 01886, USA Phone: 781-981-5725 FAX: 781-981-5766 From jmh at haystack.mit.edu Fri Oct 20 15:17:22 2006 From: jmh at haystack.mit.edu (John Holt) Date: Mon Oct 23 17:53:15 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal Message-ID: <200610201917.k9KJHMI04428@hyperion.haystack.edu> I agree with Sunrong's reasoning but, unfortunately, the acausal definition of F10.7bar is now the one in common use. Phil's definition, as I understand it, retains that definition for all but the most recent 40 days and converges on the common definition after 40 days. The only CEDAR definiton I could find is not explicit. John > Date: Fri, 20 Oct 2006 15:07:17 -0400 > From: Shunrong Zhang > User-Agent: Thunderbird 1.5.0.7 (X11/20060909) > To: Open Madrigal developers list > Subject: Re: [OpenMadrigal-developers] F10.7 Multiday average proposal > Content-Transfer-Encoding: 7bit > X-BeenThere: openmadrigal-developers@openmadrigal.org > X-Mailman-Version: 2.1.5 > List-Id: Open Madrigal developers list > List-Unsubscribe: , > List-Archive: > List-Post: > List-Help: > List-Subscribe: , > X-DSPAM-Result: Innocent > X-DSPAM-Probability: 0.000000 > X-DSPAM-Signature: 45391edb37962061126357 > > If we totally redefine Fbar to be for the last 81 days, that is fine; > but I think what Phil suggested was "for any day less than 40 days in > the past, we calculate FBAR as an average over the most recent 81 days. > " That sounds to me that you still use the old definition for any day > greater than day 40 days in the past. I wanted to say that there should > not be two definitions for one parameter. From modeling point of view, I > agree with John that "ending with the day of interest will tend to give > a somewhat better prediction". That was why I suggested my option (2) > for a new F107 index that is an average over the past 81 days and is > always available. > > Shunrong > > > John Holt wrote: > > Phil's suggestion seems reasonable. In the context of a model, the > > definition of F10.7bar as a centered average is an absurdity, > > obviously. In the "old" days we always used the 81 days ending with the > > day of interest, but at some point it apparently became more common to > > forget causality and use a centered average (I have seen 27 days, 54 > > days, 81 days 90 days and one year) and a variety of filter functions > > causal and not. Unfortunately, in my view, causal filters fell out of > > favor. Had they not, we would not have had this problem and would have > > been consistent with the laws of physics as well, which is no small > > thing. I suspect, by the way, that using the 81 days ending with the > > day of interest will tend to give a somewhat better prediction than > > using a centered average, even though the model was computed using a > > centered average. > > > > John > > > > ________________________________________________________________________ > > | | > > | John M. Holt tel: 781-981-5625 | > > | Principal Research Scientist fax: 781-981-5766 | > > | MIT Haystack Observatory email: jmh@haystack.mit.edu | > > | Route 40 WWW: http://www.haystack.edu/ | > > | Westford, MA 01886 | > > | USA | > > |________________ Ni h-eibhneas gan chlainn domhnaill _________________| > > > > > > > >> Date: Fri, 20 Oct 2006 11:53:15 -0400 > >> From: Bill Rideout > >> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) Gecko/20060417 > >> X-Accept-Language: en-us, en > >> To: openmadrigal-developers@openmadrigal.org > >> Content-Transfer-Encoding: 7bit > >> Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal > >> X-BeenThere: openmadrigal-developers@openmadrigal.org > >> X-Mailman-Version: 2.1.5 > >> List-Id: Open Madrigal developers list > >> > > > > > >> List-Unsubscribe: > >> > > , > > > > > >> List-Archive: > >> List-Post: > >> List-Help: > >> > > > > > >> List-Subscribe: > >> > > , > > > > > >> X-DSPAM-Result: Innocent > >> X-DSPAM-Probability: 0.000000 > >> X-DSPAM-Signature: 4538f138202497825867 > >> > >> Phil Erickson has suggested we modify out definition of the F10.7 > >> Multiday average (FBAR). Right now this is an 81 day average, where the > >> 81 days the center day, 40 days in the past, and 40 days in the future. > >> The problem with this definition is that this parameter is never > >> available for any day less than 40 days in the past. For example, since > >> MSIS uses FBAR as an input, this means that MSIS also cannot be run for > >> any day less than 40 days in the past. > >> > >> Phil has suggested that for any day less than 40 days in the past, we > >> calculate FBAR as an average over the most recent 81 days. From the > >> Madrigal point-of-view, this means that every time we reload geophysical > >> data, the FBAR values for the latest 40 days might change slightly. At > >> present, FBAR is not loaded into Madrigal for days less than 40 days ago. > >> > >> We would document this FBAR definition on the Madrigal help pages. Any > >> thoughts? > >> > >> Bill Rideout > >> _______________________________________________ > >> OpenMadrigal-developers mailing list > >> OpenMadrigal-developers@openmadrigal.org > >> http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > >> > >> > >> > > > > _______________________________________________ > > OpenMadrigal-developers mailing list > > OpenMadrigal-developers@openmadrigal.org > > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > > > > > > > > -- > > _____________________________________________________________ > Shunrong Zhang (shunrong@haystack.mit.edu) > http://www.haystack.mit.edu/~shunrong/ > MIT Haystack Observatory, Route 40, Westford, MA 01886, USA > Phone: 781-981-5725 FAX: 781-981-5766 > > _______________________________________________ > OpenMadrigal-developers mailing list > OpenMadrigal-developers@openmadrigal.org > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > !DSPAM:45391edb37962061126357! From pje at haystack.mit.edu Fri Oct 20 15:36:24 2006 From: pje at haystack.mit.edu (Phil Erickson) Date: Mon Oct 23 17:53:15 2006 Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal In-Reply-To: <200610201917.k9KJHMI04428@hyperion.haystack.edu> References: <200610201917.k9KJHMI04428@hyperion.haystack.edu> Message-ID: <45392538.8040301@haystack.mit.edu> Yes, that's what I intended. It seems the CEDAR definition really never covered the close-to-realtime (at least < 40 day) case, so I figured we could document that and use it to fill in the gap. As it is now, we have a 'missing' value in the < 40 day case. ---- phil John Holt wrote: > I agree with Sunrong's reasoning but, unfortunately, the acausal > definition of F10.7bar is now the one in common use. Phil's definition, > as I understand it, retains that definition for all but the most recent > 40 days and converges on the common definition after 40 days. The only > CEDAR definiton I could find is not explicit. > > John > >> Date: Fri, 20 Oct 2006 15:07:17 -0400 >> From: Shunrong Zhang >> User-Agent: Thunderbird 1.5.0.7 (X11/20060909) >> To: Open Madrigal developers list >> Subject: Re: [OpenMadrigal-developers] F10.7 Multiday average proposal >> Content-Transfer-Encoding: 7bit >> X-BeenThere: openmadrigal-developers@openmadrigal.org >> X-Mailman-Version: 2.1.5 >> List-Id: Open Madrigal developers list > >> List-Unsubscribe: > , > >> List-Archive: >> List-Post: >> List-Help: > >> List-Subscribe: > , > >> X-DSPAM-Result: Innocent >> X-DSPAM-Probability: 0.000000 >> X-DSPAM-Signature: 45391edb37962061126357 >> >> If we totally redefine Fbar to be for the last 81 days, that is fine; >> but I think what Phil suggested was "for any day less than 40 days in >> the past, we calculate FBAR as an average over the most recent 81 days. >> " That sounds to me that you still use the old definition for any day >> greater than day 40 days in the past. I wanted to say that there should >> not be two definitions for one parameter. From modeling point of view, I >> agree with John that "ending with the day of interest will tend to give >> a somewhat better prediction". That was why I suggested my option (2) >> for a new F107 index that is an average over the past 81 days and is >> always available. >> >> Shunrong >> >> >> John Holt wrote: >>> Phil's suggestion seems reasonable. In the context of a model, the >>> definition of F10.7bar as a centered average is an absurdity, >>> obviously. In the "old" days we always used the 81 days ending with the >>> day of interest, but at some point it apparently became more common to >>> forget causality and use a centered average (I have seen 27 days, 54 >>> days, 81 days 90 days and one year) and a variety of filter functions >>> causal and not. Unfortunately, in my view, causal filters fell out of >>> favor. Had they not, we would not have had this problem and would have >>> been consistent with the laws of physics as well, which is no small >>> thing. I suspect, by the way, that using the 81 days ending with the >>> day of interest will tend to give a somewhat better prediction than >>> using a centered average, even though the model was computed using a >>> centered average. >>> >>> John >>> >>> ________________________________________________________________________ >>> | | >>> | John M. Holt tel: 781-981-5625 | >>> | Principal Research Scientist fax: 781-981-5766 | >>> | MIT Haystack Observatory email: jmh@haystack.mit.edu | >>> | Route 40 WWW: http://www.haystack.edu/ | >>> | Westford, MA 01886 | >>> | USA | >>> |________________ Ni h-eibhneas gan chlainn domhnaill _________________| >>> >>> >>> >>>> Date: Fri, 20 Oct 2006 11:53:15 -0400 >>>> From: Bill Rideout >>>> User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.13) > Gecko/20060417 >>>> X-Accept-Language: en-us, en >>>> To: openmadrigal-developers@openmadrigal.org >>>> Content-Transfer-Encoding: 7bit >>>> Subject: [OpenMadrigal-developers] F10.7 Multiday average proposal >>>> X-BeenThere: openmadrigal-developers@openmadrigal.org >>>> X-Mailman-Version: 2.1.5 >>>> List-Id: Open Madrigal developers list >>>> >>> >>> >>>> List-Unsubscribe: >>>> >>> , >>> > >>> >>>> List-Archive: > >>>> List-Post: >>>> List-Help: >>>> >>> >>> >>>> List-Subscribe: >>>> >>> , >>> >>> >>>> X-DSPAM-Result: Innocent >>>> X-DSPAM-Probability: 0.000000 >>>> X-DSPAM-Signature: 4538f138202497825867 >>>> >>>> Phil Erickson has suggested we modify out definition of the F10.7 >>>> Multiday average (FBAR). Right now this is an 81 day average, where the >>>> 81 days the center day, 40 days in the past, and 40 days in the future. >>>> The problem with this definition is that this parameter is never >>>> available for any day less than 40 days in the past. For example, since >>>> MSIS uses FBAR as an input, this means that MSIS also cannot be run for >>>> any day less than 40 days in the past. >>>> >>>> Phil has suggested that for any day less than 40 days in the past, we >>>> calculate FBAR as an average over the most recent 81 days. From the >>>> Madrigal point-of-view, this means that every time we reload geophysical >>>> data, the FBAR values for the latest 40 days might change slightly. At >>>> present, FBAR is not loaded into Madrigal for days less than 40 days ago. >>>> >>>> We would document this FBAR definition on the Madrigal help pages. Any >>>> thoughts? >>>> >>>> Bill Rideout >>>> _______________________________________________ >>>> OpenMadrigal-developers mailing list >>>> OpenMadrigal-developers@openmadrigal.org >>>> http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers >>>> >>>> >>>> >>> _______________________________________________ >>> OpenMadrigal-developers mailing list >>> OpenMadrigal-developers@openmadrigal.org >>> http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers >>> >>> >>> >> >> -- >> >> _____________________________________________________________ >> Shunrong Zhang (shunrong@haystack.mit.edu) >> http://www.haystack.mit.edu/~shunrong/ >> MIT Haystack Observatory, Route 40, Westford, MA 01886, USA >> Phone: 781-981-5725 FAX: 781-981-5766 >> >> _______________________________________________ >> OpenMadrigal-developers mailing list >> OpenMadrigal-developers@openmadrigal.org >> http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers >> >> > > _______________________________________________ > OpenMadrigal-developers mailing list > OpenMadrigal-developers@openmadrigal.org > http://www.openmadrigal.org/mailman/listinfo/openmadrigal-developers > > !DSPAM:453920fc45371318818334!