TECHNICAL APPENDIX

1. TECHNICAL APPENDIX METHODOLOGY SUMMARY FOR CPM RESEARCH Missions International PO Box 681299 – Franklin, TN 37068 – www.missions.com

2. Human Resourcing • Design of CPM assessment instrument and analysis support • Hired Statistician • Ph.D. from Vanderbilt in Psychology • VP of Methodology for a Market Research Firm • Over 15 years corporate research experience with Fortune 500 companies • Previous analytic work for MI on (SG)2

3. OBJECTIVES • Discover elements (DNA) that drive conversion and planting growth • Measure conversions and planting growth • Measure comprehensive set of potential “drivers” (DNA) • Build predictive models to uncover which elements from comprehensive set are most “important” in driving conversions/planting • Substantive CPM interpretations across predictive models

4. Assessment Design • Qualitative content inputs from key expert leaders in Church Planting Movement - Watson, Sergeant, Sanchez, etc. • What are “DNA” elements of church planting movement? • Developed initial battery of items to capture that domain • Subjected to review by experts • Iterated to list of 31 items designed to capture domain broadly • All elements measured on five-point frequency-of-occurrence scale: “Very Often” to “Almost Never”

5. Other Variables Measured • Also included several descriptive / demographic measures • age, gender, education, size of community, leader/member, formal training, how long ago church started, how many members attend, etc. • Also included “Dependent” Variables • Self-report on frequency of starting new churches • On same “Very Often” to “Almost Never” scale • Raw numeric dependent variables: • How many converted through your church? • How many converted through your church have been baptized? • How many churches branched off from your church?

6. Samples • Sample sizes by contributing countries: • (n=542) • (n=507) • (n=858) • (n=61) • One sample showed extremely limited variability (interpreted as most elements from expert inputs fully in operation in this movement, thus “topping out” the scale – i.e., all aspects tend to be happening very often in this movement). • Analysis conducted on other countries where more variability existed - pooling of other nations (starting n=1110)

7. More About Dependent Variables • Self-report of Church planting activity • “Our church starts new churches…” • “Very Often” • “Often” • “Sometimes” • “Rarely” • “Almost Never” • Coded as quasi-interval 1-5 ratings

8. Numeric Dependent Variables • Number of conversions, baptisms, new churches started • All influenced by (a) length of existence • 24 conversions from 2 year old church, vs. 24 conversions from 2 month old church • 1 conversion per month vs. 12 per month • All influenced by (b) number of members • 50 conversions from church of 10, vs. 50 conversions from church of 25 • 5 conversion per member vs. 2 per member • Must control for length of existence and number of members to make raw numbers sensible and comparable

9. “Per Member Per Month”- pmpm • Implemented per member per month approach • 100 conversions from 10 member church in existence for 1 month = 10 conversions per member per month • 200 conversions from 20 members in existence 5 months = 2 conversion per member per month • Raw numbers would have made 200 look “better” than 100, when clearly in this scenario 200 was less productive relative conversion growth than 100 • Thus created metric – per member per month (pmpm) – to take care of this issue • Approach taken with number of conversions, number of baptisms, number of churches planted

10. Further Treatment of Dependent Variables - indexing • Baptism often associated with conversion, thus reported numbers of baptisms highly correlated with numbers of conversions • Created a single index of conversions and baptisms • This index is a simple average of the two quantities, expressed in pmpm units

11. Further Treatment of Numeric Dependent Variables- normalization • Distributions of variables in original and pmpm forms “positively” / “right” skewed • Multivariate procedure - assumptions call for transformations to normality • Cube root transformation chosen

12. Further Treatment of Dependent Variables - outliers • Examined distributions on numeric pmpm variables in terms of “standardized” / “z-scores” • This procedure helps identify “outliers” – cases that are quite different from most other cases in the sample • A typical cut-off would be z-scores greater than 3.0 in absolute value (+/- 3.0) • After inspection, allowing retention of more original sample, the cut-off for this study was set at +/- 3.5 • Typically less than 1% of cases eliminated

13. Final Variables For Predictive Models after all Treatments • Dependent Variables • Index of conversions and baptisms, pmpm (cube-root transformed) • Number of churches planted, pmpm (cube-root transformed) • Self-reported frequency of church starts • Independent Variables • 31 items capturing domain of “DNA of CPM”

14. Model Building - regression • Many available approaches to predictive model building • Typical statistical tool is multiple regression • Predict a dependent variable from a set of independent variables • Find a weight for each independent variable such that the weighted combination of independent variable values produces an output that is “as close as possible” to the actual/observed dependent variable values

15. Model Building - Regression • Weights combine independent variables to produce an output – a predicted dv value • (w1*iv1 + w2*iv2 + w3*iv3…) = output / predicted dv • Output/predicted dv as close as possible to observed dependent variable scores • Technically, squared errors between predicted dv and observed dv are minimized, hence the name “least squares” regression • When standardized, regression weights can be interpreted as “relative impact” – how much unique impact each iv has on the dv

16. Model Building Procedure • There are different approaches to regression depending on the research objectives • Many options for variable selection in building a predictive multiple regression equation • Backward entry • Forward entry • Stepwise entry • Simultaneous/ forced entry • A hybrid “procedure” was developed here to arrive at the final models

17. Hybrid Model Building Procedure • Not logical to accept a variable as a significant driver if it shows no meaningful bivariate correlation • Thus, screened independent variable set to include only starting variables with meaningful correlations • Organized meaningfully correlated independent variables into “tiers”, e.g., • .4s and above • .3s • .2s • .1s

18. Hybrid Model Building Procedure • Used forward entry within each tier, starting with strongest correlation tier first • Forward entry added variables one at a time based on biggest improvement in R-square • Forward entry thus identified subset of significant predictors in strongest correlational tier • That subset became starting basis for consideration of next tier

19. Hybrid Model Building Procedure • Forward entry within next tier added additional significant predictors • Significant predictors from first two tiers became starting basis, in moving to the next correlational tier • Tested that next tier for any additional variables adding predictive power • Procedure continued until all tiers were exhausted • (note – not all models had all four tiers of correlational strengths. If only .3s, .2s, .1s existed, general procedure was still same, progressing from highest strength tier to lowest strength tier) • After no additional variables added significant predictive power, tried to re-enter any previously eliminated variables • When no additional variables add anything, final predictive set has been discovered

20. Hybrid Model Building Procedure • Traditional regression diagnostics checked at each step along the way (e.g., outliers, collinearity, normality of errors of prediction, quality of prediction as evidenced in adjusted R-squared, checks for sign reversal/ suppressor variables) • Note 1 – for treatment of “missing” data, large total sample size allowed use of “listwise” procedure – modeling used only those individual respondents with complete data on all variables in any given model • Note 2 – a few negative regression coefficients were accepted as valid. This was an acceptable finding only when: • Original bivariate correlation was negative, thus indicating not a sign-reversal / suppressor problem • Interpretation of negative association was plausible and logical given understanding of domain of study

21. Overall Model Summaries

22. Self-Report Planting Drivers = negative association

23. Conversion/ Baptism Drivers = negative association

24. Numeric Planting Drivers = negative association

25. Interpretative Note • Substantive interpretation is a synthesis across predictive models, also made in light of original qualitative research with experts • This synthesis draws out key areas of focus found to be predictive of Church Planting success • If generalizability holds beyond the measured samples, based on the data collected for this study, deeper implementation of positively related DNA elements (and avoidance of negatively related ones) should lead to more conversion growth and baptisms, and increased levels of church planting activity