1 / 19

Resources on forecast evaluation/verification

Resources on forecast evaluation/verification. Barbara Brown. Joint Numerical Testbed. NCAR, Boulder, CO. July 2011. WMO Verification web page. Developed and maintained by. working group on. forecast verification research. Includes FAQs,. references and links to

bob
Download Presentation

Resources on forecast evaluation/verification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Resourcesonforecast evaluation/verification BarbaraBrown JointNumericalTestbed NCAR,Boulder,CO July2011

  2. WMOVerificationwebpage Developedand maintainedby workinggroupon forecastverification research IncludesFAQs, referencesandlinksto tutorialpresentations Verificationdiscussion group: vx- discuss@rap.ucar.edu http://www.cawcr.gov.au/projects/verification/

  3. EUMETCALOnlineLearningModule Freeon-line tutorialwith exercises Includesmodules on Categorical Continuous Probabilistic Verification methods http://www.eumetcal.org/resources/ukmeteocal/verific ation/www/english/courses/msgcrs/index.htm

  4. Books Jolliffeand Stephenson: Forecast Verification-A Practitioner’s Guidein Atmospheric Sciences PublishedbyWiley (newversionin 2011) Weatherand Climate Prediction Publishedby Cambridge UniverstityPress Includeschapter Warner(2010): Wilks(2011):Statistical MethodsintheAtmospheric Sciences PublishedbyElsevier Includesextensivechapteron verificationmethods Numerical

  5. ForecastEvaluationTools RStatisticsPackage: Includesmanystatisticaltools, includingtheRVerification Library METisfreelyavailableand supportedtothecommunity Mainfocus:Model verification Includestoolsforpointand griddeddata,ensemble forecasts,spatialmethods http://www.dtcenter.org/met/users/

  6. IntroductiontoR TaraJensen NCAR/RAL/JNT

  7. WhatisR? •Astatisticalprogramminglanguage •Inpart,developedfromtheSProgramming LanguagefromBellLabs(JohnChambers) •Createdtoallowrapiddevelopmentof methodsforuseindifferenttypesofdata. •Createnewgraphics.Manydefault parametersarechosen,butusersretain completecontrol.

  8. WhyR? •Rhasbecomethedominantlanguageinthestatistical researchcommunity. •RisOpenSourceandfree. •Runsonalloperatingsystems •Nearly2,400packagescontributed. •Packagesandapplicationsinnearlyeveryfieldof science,businessandeconomics. •SeeRNotes,RJournalandJournalofStatistical Software.www.jstatsoft.org •Morethan100bookswithaccompanyingcode •Verylarge,activeuserbase.

  9. WhynotR? •NCL,IDL,Matlab,SAS,…areallviable alternativestoR.Ifyouareapartofanactive communityofresearchersusinganother language,dolikewise. •Ifwewerebiostatisticianswewouldbeusing SAS.BookTitle:“AnalyzingReceiverOperatingCharacteristicCurveswithSAS” •Considerbuildingverificationfunctionsand utilitiesaspartofcodedevelopment. Verificationneednotbeanexternalprocessto forecasting.

  10. TheRCommunity •Developers –RCoreGroup(17members),only2haveleftsince 1997 –MajorupdateinApril/October(freezedates,beta versions,bugtracking,...) •Mailinglists –Helplist~150messages/day,archived, searchable. •5InternationalConferences,2US,1China

  11. EverythingaboutRisatwww.r-project.org •Sourcecode •Binarycompilations(Windows,MacOS,Linux •Documentation(Maindocuments,plusnumerous contributed.Someinforeignlanguages.) •Newsletter(replacedbyRJournal.) •Mailinglist(Severalsearchengines) •Packagesoneverytopicimaginable •Wikiwithexamples •ReferencelistofbooksusingR.(morethan100) •TaskManager

  12. UseRwithscripts •InLinux-EmacsSpeaksStatistics –Providessyntax-based –Objectnamecompletion –Keystrokeshortcuts –Commandhistory –Alt-xRtoinvokeRwithXemacs. •InWindows,useeditor –AddedGUIfeatures –<control>RsendsalineorhighlightedsectionintoR. –InstallpackagewithGUIs –Savegraphicsbypointandclick. •MacOS –SimilartoWindowswithadvantagesofsystemcalls.

  13. Codingprinciples •Makeverificationcodetransparentandeasy toread •Commentanddocumentliberally •Archiveyourcode •Shareyourcode •Labelandsaveyourdata •Shareyourdata

  14. PackagesinR •Contributedbypeopleworldwide. •Allowscientistsorstatisticianstopushtheir ideas. •ApplyandextendRcapabilitiestomeetthe needsofspecificcommunities. •Accompanymanystatisticaltextbooks •Accompanyappliedarticles(AdrianRaftery, DougNychka,TilmanGneiting,BarbaraCasati, MattBriggs)

  15. Asampleofusefulpackages • • • • • •Rgis,spatstat(GIS) •ncdf(supportfor netcdffiles) •Rcolorbrewer •randomForests verification fields(spatialstats) radiosondes extRemes BMA(BayesianModel Averaging) •BMAensemble •circular •Rsqlite

  16. Packages •Packagesmustbeinstalledtocall. •Packagesmustbecalledtouse. •Basepackagesareinstalledbydefault.

  17. 10mostusefulfunctioninR •aggregate-appliesafunctiontogroupsof datasubsetbycategories. •apply-incrediblyefficientinavoidingloops. Appliesfunctionsacrossdimensionsofarrays. •layout-creativelydivideaprintregion. •xyplot(inthelatticepackage)slightlyadvance graphictechniques •%in%returnslogicalshowingwhichelements inAareinB.(e.gA%in%B)

  18. Moretop10 • • • • • table–createcontingencytabelcounts. boot–applybootstrapfunctioncorrectly read.fwf–readfixedwidthformatdata par–controleverythinginagraph system()–allowsyoutocallsystem commandfromR •pairs–themostunderutilizedplot–plotsa matrixof4columnsina4x4plotlayout

  19. RExercises • • • • • Choosegroupsof3-4–findacomputer Logontomachines Bringupatleast2xterms >cp/wrfhelp/MET/R-packages/example/*. >viintro2R.2011comet.R or >kwriteintro2R.2011comet.R&

More Related