1 / 69

Intr o duc t i on

Intr o duc t i on. Computatio n in fields. Accelerating fields. Conclusion s and F uture W o rk. References. Ap p endix. Accelerating the ‘ fields ’ P ac k age in R : The o ry and Application John P aige 1 ( paigejo@gmail.com )

skyla
Download Presentation

Intr o duc t i on

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingthe‘fields’PackageinR: TheoryandApplication JohnPaige1(paigejo@gmail.com) IsaacLyngaas2(irlyngaas@gmail.com) SrinathVadlamani3(srinathv@ucar.edu) DougNychka3(nychka@ucar.edu) 1MacalesterCollege 2FloridaStateUniversity 3NationalCenterofAtmosphericResearch July31,2014 tudentswhoare

  2. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Introduction IntroductiontoProblemGoalsandMotivation Computationinfields MathematicalFoundation Acceleratingfields EigenandCholeskyDecompositionsParameterOptimization ConclusionsandFutureWork Appendix

  3. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix AnalyzingSpatialDatawithfields:Kriging Colorado Average Spring High Temperature (Celsius) 41 20 40 15 Latitude 39 10 38 5 37 −108 −106 −104 −102 Longitude Datasetfromfieldspackage(Nychkaetal.,2014b)

  4. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix AnalyzingSpatialDatawithfields:Kriging Colorado Average Spring High Temperature (Celsius) 41 20 40 ? 15 Latitude 39 10 38 5 37 −108 −106 −104 −102 Longitude Datasetfromfieldspackage(Nychkaetal.,2014b)

  5. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix AnalyzingSpatialDatawithfields:Kriging Datasetfromfieldspackage(Nychkaetal.,2014b)Krigingestimateforλ=.103,θ=5.45

  6. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050 100 150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  7. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050 100 150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  8. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations • • O(n3) Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050 100 150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  9. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations • • O(n3) • 100,000observations,fixedparameters:∼8hours Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050 100 150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  10. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations • • O(n3) • 100,000observations,fixedparameters:∼8hours • 100,000observations,10 parametersamples:∼80hours Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050100150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  11. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations • • O(n3) • 100,000observations,fixedparameters:∼8hours • 100,000observations,10 parametersamples:∼80hours • Krigingovertimeformanyparametersets: Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050100150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  12. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix DifficulttoAnalyzeLargeDatasets • DifficultiesinKriging: • Krigingwithmanyobservations • • O(n3) • 100,000observations,fixedparameters:∼8hours • 100,000observations,10 parametersamples:∼80hours • Krigingovertimeformanyparametersets: • 5,000observations,30years(monthlydata),20parametersamples:nearly14hours Fields' CO2Dataset 50 Latitude 0 −50 −150 −100−50050100150 Longitude CO2datasetinfieldspackage(Nychkaetal.,2014b)

  13. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ResearchGoals • LearnKrigingtheory • AcceleratethefieldspackageinR • Choleskyandeigendecompositions • Spatialparameterestimation(maximumlikelihoodestimation)

  14. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Why Focusonfields? • Spatial Problem Workflow Times • goeR • fields(withmKrig) • MadeinR • (Nychkaetal.,2014b) • Free,opensource • Popularwithstatisticians • Easytouse 100 Time(Minutes) 10 1 0.1 0 2000 4000 6000 NumberofObservations 8000 10000 geoRandfieldspackagesusedinthisplot:DiggleandRibeiro(2007);Nychkaetal.(2014b);RibeiroJrandDiggle(2001)

  15. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Why Focusonfields? • Spatial Problem Workflow Times • goeR • fields(withmKrig) • MadeinR • (Nychkaetal.,2014b) • Free,opensource • Popularwithstatisticians • Easytouse • Fast(forR) 100 Time(Minutes) 10 1 0.1 0 2000 4000 6000 NumberofObservations 8000 10000 geoRandfieldspackagesusedinthisplot:DiggleandRibeiro(2007);Nychkaetal.(2014b);RibeiroJrandDiggle(2001)

  16. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • Why FocusonMatrixDecompositionsandOptimization? • EigenandCholeskydecompositionstakealongtime • Over10,000observationsmeansover40%ofthecomputationtime Spatial Surface Estimation Computation Time Complete Workflow Workflow CholeskyDecompositions 0 1 2 3 4 5 6 Time(Minutes) • Spatialparameteroptimization requiresmatrixdecompositions • Betteroptimizationmeans fewermatrixdecompositions 0 2000 4000 6000 Number of Observations 8000 10000 • Note: runningonaCalderanodeon Jellystone • GPUs:2TeslaM2070-Q • CPUs:28-core2.6-GhzIntelXeonE5-2670(SandyBridge)

  17. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • Why FocusonMatrixDecompositionsandOptimization? • EigenandCholeskydecompositionstakealongtime • Over10,000observationsmeansover40%ofthecomputationtime Percent Workflow Time Taken by Cholesky Decompositions 20 40 60 80 100 • Spatialparameteroptimization requiresmatrixdecompositions • Betteroptimizationmeans fewermatrixdecompositions Percent 0 0 2000 4000 6000 Number of Observations 8000 10000 • Note: runningonaCalderanodeon Jellystone • GPUs:2TeslaM2070-Q • CPUs:28-core2.6-GhzIntelXeonE5-2670(SandyBridge)

  18. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • Why FocusonMatrixDecompositionsandOptimization? • EigenandCholeskydecompositionstakealongtime • Over10,000observationsmeansover40%ofthecomputationtime Percent Workflow Time Taken by Cholesky Decompositions 20 40 60 80 100 • Spatialparameteroptimization requiresmatrixdecompositions • Betteroptimizationmeans fewermatrixdecompositions Percent 0 0 2000 4000 6000 Number of Observations 8000 10000 • Note: runningonaCalderanodeon Jellystone • GPUs:2TeslaM2070-Q • CPUs:28-core2.6-GhzIntelXeonE5-2670(SandyBridge)

  19. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • Why FocusonMatrixDecompositionsandOptimization? • EigenandCholeskydecompositionstakealongtime • Over10,000observationsmeansover40%ofthecomputationtime Percent Workflow Time Taken by Cholesky Decompositions 20 40 60 80 100 • Spatialparameteroptimization requiresmatrixdecompositions • Betteroptimizationmeans fewermatrixdecompositions Percent 0 0 2000 4000 6000 Number of Observations 8000 10000 • Note: runningonaCalderanodeon Jellystone • GPUs:2TeslaM2070-Q • CPUs:28-core2.6-GhzIntelXeonE5-2670(SandyBridge)

  20. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • Why FocusonMatrixDecompositionsandOptimization? • EigenandCholeskydecompositionstakealongtime • Over10,000observationsmeansover40%ofthecomputationtime Percent Workflow Time Taken by Cholesky Decompositions 20 40 60 80 100 • Spatialparameteroptimization requiresmatrixdecompositions • Betteroptimizationmeans fewermatrixdecompositions Percent 0 0 2000 4000 6000 Number of Observations 8000 10000 • Note: runningonaCalderanodeon Jellystone • GPUs:2TeslaM2070-Q • CPUs:28-core2.6-GhzIntelXeonE5-2670(SandyBridge)

  21. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix KrigingSet-Up(WithNoTrendComponent) yi =g(xi)+ei yi:observations xi:locations g:Krigingsurface ei:errorfollowingN(0,σ2) distribution,independent

  22. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix KrigingSet-Up(WithNoTrendComponent) yi =g(xi)+ei yi:observations xi:locations g:Krigingsurface ei:errorfollowingN(0,σ2) distribution,independent

  23. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix KrigingSet-Up(WithNoTrendComponent) yi =g(xi)+ei yi:observations xi:locations g:Krigingsurface ei:errorfollowingN(0,σ2) distribution,independent

  24. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix KrigingSet-Up(WithNoTrendComponent) yi =g(xi)+ei yi:observations xi:locations g:Krigingsurface ei:errorfollowingN(0,σ2) distribution,independent

  25. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix KrigingSet-Up(WithNoTrendComponent) yi =g(xi)+ei yi:observations xi:locations g:Krigingsurface ei:errorfollowingN(0,σ2) distribution,independent

  26. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • KrigingSet-Up(WithNoTrendComponent) • yi =g(xi)+ei • yi:observations • xi:locations • g:Krigingsurface • ei:errorfollowingN(0,σ2) distribution,independentAssumptions(Nychkaetal.,2014a): • • E(g(x))=0 • Krigingsurfacehaszeromean • • E[g(x)g(xt)]=ρ·k(x,xt): • k:correlationfunction (e.g.e−||x−xt||/θ) • ρ:correlationstrength(orsignalstrength)

  27. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • KrigingSet-Up(WithNoTrendComponent) • yi =g(xi)+ei • yi:observations • xi:locations • g:Krigingsurface • ei:errorfollowingN(0,σ2) distribution,independentAssumptions(Nychkaetal.,2014a): • • E(g(x))=0 • Krigingsurfacehaszeromean • • E[g(x)g(xt)]=ρ·k(x,xt): • k:correlationfunction (e.g.e−||x−xt||/θ) • ρ:correlationstrength(orsignalstrength)

  28. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • KrigingSet-Up(WithNoTrendComponent) • yi =g(xi)+ei • yi:observations • xi:locations • g:Krigingsurface • ei:errorfollowingN(0,σ2) distribution,independentAssumptions(Nychkaetal.,2014a): • • E(g(x))=0 • Krigingsurfacehaszeromean • • E[g(x)g(xt)]=ρ·k(x,xt): • k:correlationfunction (e.g.e−||x−xt||/θ) • ρ:correlationstrength(orsignalstrength)

  29. IntroductionComputationinfields Acceleratingfields ConclusionsandFutureWork References Appendix • KrigingSet-Up(WithNoTrendComponent) • yi =g(xi)+ei • yi:observations • xi:locations • g:Krigingsurface • ei:errorfollowingN(0,σ2) distribution,independentAssumptions(Nychkaetal.,2014a): • • E(g(x))=0 • Krigingsurfacehaszeromean • • E[g(x)g(xt)]=ρ·k(x,xt): • k:correlationfunction (e.g.e−||x−xt||/θ) • ρ:correlationstrength(orsignalstrength) • Reparameterization: λ=σ2/ρ:smoothingparameter (1/λ: signal/noiseratio)

  30. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimization:MaximumLikelihoodEstimation Definition Alikelihoodfunction,L,givesthechancethatasetofobservationsoccurinamodelgiventhemodelparameters. Definition αˆMLEisamaximumlikelihoodestimateofαifαˆMLEmaximizes thedatalikelihood: L(αˆMLE)≥L(α),∀α

  31. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimization:MaximumLikelihoodEstimation Definition Alikelihoodfunction,L,givesthechancethatasetofobservationsoccurinamodelgiventhemodelparameters. Definition αˆMLEisamaximumlikelihoodestimateofαifαˆMLEmaximizes thedatalikelihood: L(αˆMLE)≥L(α),∀α

  32. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimization:MaximumLikelihoodEstimation Definition Alikelihoodfunction,L,givesthechancethatasetofobservationsoccurinamodelgiventhemodelparameters. Definition αˆMLEisamaximumlikelihoodestimateofαifαˆMLEmaximizes thedatalikelihood: L(αˆMLE)≥L(α),∀α

  33. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimization:MaximumLikelihoodEstimation Definition Alikelihoodfunction,L,givesthechancethatasetofobservationsoccurinamodelgiventhemodelparameters. Definition αˆMLEisamaximumlikelihoodestimateofαifαˆMLEmaximizes thedatalikelihood: L(αˆMLE)≥L(α),∀α Thedatalog-likelihoodgiveninNychkaetal.(2014a)foragiven θandλis: yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2

  34. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant

  35. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant

  36. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant

  37. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant

  38. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant • Howcanthislikelihoodbecomputedquickly?

  39. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant • Howcanthislikelihoodbecomputedquickly? • Kriguseseigendecomposition • mKrigusesCholeskydecomposition

  40. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix ParameterOptimizationinfields yT(Σθ+λI)−1y1 lnL(θ,λ)=− + ln|Σθ+λI|+C 2 2 • y:vectorofobservationvalues • Σθ:covariancematrix,where: • (Σθ)ij=ρk(xi,xj)=ρe−||xi−xj||/θ • C:constant • Howcanthislikelihoodbecomputedquickly? • Kriguseseigendecomposition • mKrigusesCholeskydecomposition

  41. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingfields:EigenandCholeskyDecompositions • MAGMA(Agulloetal.,2009) • Freelyavailablelibrarywithmulti-GPUcomputingcapability • MuchfasterthandefaultR

  42. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix AcceleratedWorkflowTime Spatial Problem Workflow Times Default R One GPU Two GPUs 8 Time (Minutes) 6 4 2 0 0 5000 10000 15000 Number of Observations • >2500observations⇒ acceleratedworkflowisfaster • >10000observations⇒ ≥1.55×speedup(for1or2GPUs)

  43. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix AcceleratedWorkflowTime Spatial Problem Workflow Times Default R One GPU Two GPUs 8 Time (Minutes) 6 4 2 0 0 5000 10000 15000 Number of Observations • >13000observations⇒ • twoGPUworkflowfasterthanoneGPUby≥1sec

  44. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingfields:MaximumLikelihoodEstimation • Goal: • Quicklyfindsetofmodel parametersmaximizingdata likelihood • Questions: • IsEigenorCholeskydecompositionfasterformaximizinglikelihood? • Howdodifferentwaysofsplitting upthesedecompositionsamongcoresandGPUsaffectlikelihood maximizationtime? ObservationsfromCO2datasetinfieldspackage(Nychkaetal., 2014b)

  45. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingfields:MaximumLikelihoodEstimation • Goal: • Quicklyfindsetofmodel parametersmaximizingdata likelihood • Questions: • IsEigenorCholeskydecompositionfasterformaximizinglikelihood? • Howdodifferentwaysofsplitting upthesedecompositionsamongcoresandGPUsaffectlikelihood maximizationtime? ObservationsfromCO2datasetinfieldspackage(Nychkaetal., 2014b)

  46. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingfields:MaximumLikelihoodEstimation • Goal: • Quicklyfindsetofmodel parametersmaximizingdata likelihood • Questions: • IsEigenorCholeskydecompositionfasterformaximizinglikelihood? • Howdodifferentwaysofsplitting upthesedecompositionsamongcoresandGPUsaffectlikelihood maximizationtime? ObservationsfromCO2datasetinfieldspackage(Nychkaetal., 2014b)

  47. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix Acceleratingfields:MaximumLikelihoodEstimation • Goal: • Quicklyfindsetofmodel parametersmaximizingdata likelihood • Questions: • IsEigenorCholeskydecompositionfasterformaximizinglikelihood? • Howdodifferentwaysofsplitting upthesedecompositionsamongcoresandGPUsaffectlikelihood maximizationtime? ObservationsfromCO2datasetinfieldspackage(Nychkaetal., 2014b)

  48. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix CholeskyWinsOverEigendecomposition Cholesky Decomposition Speedup Over Eigendecomposition (Default Implementations) 0 15 40 60 80 100 120 140 Speedup 0 5000 10000 15000 NumberofObservations • Optimizingλforfixedθ: • 10to15Choleskydecompositions • Oneeigendecomposition

  49. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix CholeskyWinsOverEigendecomposition Cholesky Decomposition Speedup Over Eigendecomposition (Default Implementations) 0 15 40 60 80 100 120 140 Speedup 0 5000 10000 15000 NumberofObservations • <20000observations⇒ Choleskyisatleast18×faster

  50. Introduction Computationinfields Acceleratingfields ConclusionsandFutureWork References Appendix CholeskyWinsOverEigendecomposition Cholesky Decomposition Speedup Over Eigendecomposition (Default Implementations) 0 15 40 60 80 100 120 140 Speedup 0 5000 10000 15000 NumberofObservations • CholeskydecompositionachievedbetterspeedupswithGPUs

More Related