NOW THAT I ’ VE COLLECTED SOME DATA, HOW DO I WRITE UP MY FINDINGS? Prepared by

1. NOW THAT I’VE COLLECTED SOME DATA, HOW DO I WRITE UP MY FINDINGS? Prepared by Dr. Diane Tickton Schuster BMR Congregational Inquiry Coordinator and Rabbi Stacy Eskovitz Rigler Reform Congregation Keneseth Israel Elkins Park, PA February 28, 2013

2. The Webinar Broadcast Has Begun • All attendees are in “Listen Only Mode” and will remain this way for the duration of the webinar. Before the presentation begins, we would like to review a few of the technical aspects of your webinar experience. • You should all see a panel on your screen similar to our examples on the following slides. This control panel presents each attendee with various options related to our webinar presentation.

3. Helpful Tips Open and close your Panel

4. Helpful Tips • Asking Questions • Raise hand • Questions Log • Note any technical issues in the Questions Log as well

5. Helpful Tips • Audio Mode • View, Select, and Test your audio

6. Today’s Agenda… • How Your Research Fits into the BMR Theory of Change • Why Research Reports are Beneficial • Parts of a Research Report • Collecting Quantitative Data • Collecting Qualitative Data • Analyzing Data • Reporting What You’ve Learned

7. BMR Theory of Change

8. BMR Theory of Change Key Concepts Impetus for Change: This is what led your congregation to participate in the B’nai Mitzvah Revolution. Visioning: Exploring (and possibly rethinking) your beliefs about the role bar/bat mitzvah play in the life of the child, the family, and the congregation. Planning for Experimentation: The process you will go through to plan the innovations that will enable you to fulfill your vision. Experimentation:Implementation of these innovative programs and approaches. Documentation: Collecting systematic data that enable you to refine your experiments and evaluate their success. Reflection: Analyzing the data you have collected, and considering what you have learned from each round of experimentation. Community Engagement: Involving successive layers of the congregation in the BMR, so that a large part of the congregation comes to share your vision and appreciate the changes you’ve instituted.

9. The BMR Theory of Change assumes that there will be an ongoing interaction among the different aspects of the cycle.For example: • Visioning and Planning for Experimentation anticipate the research questions that drive the experiment(s) • Documentation activities trace back to the Impetus for Change and Visioning • Reflection builds on what’s been learned from Experimentation and leads to additional experiments • All of these processes lead to some form of Community Engagement Reminder: As you proceed with developing and documenting your experiment, be sure to regularly let your various stakeholders know about ongoing BMR activity and get their feedback.

10. Why Research Reports areBeneficial Research reports help us to… • Continually loop back to our vision and the research questions that derived from that vision • Systematically “fill in the blanks” about what we are doing and the impact it’s having • Organize findings in ways that will be helpful for explaining—even justifying—”change” to others • Give us solid information on which to build future plans and increase community engagement

11. The Parts of a BMR Documenter’sResearch Report • Introduction • Your congregation’s BMR overall Research Question • Literature Review [optional] • Short-term Action Research Question & Rationale

12. Parts of a BMR Documenter’sResearch Report (continued) • The Sample [who was studied] • The Method [how the research was conducted] • The Findings • Discussion/Implications/ Future Research/Community Engagement

13. Introduction What is Keneseth Israel’s BMR project? How did it come about?

14. At KI, b’nai mitzvah students have long undertaken independent mitzvah projects. • The impact of these experiences has not been studied. From the perspective of the KI BMR team, typically these projects do not help the children or their families to develop a deep connection with the synagogue community as a whole. • Moreover, it is not clear how the mitzvah project is part of the child’s understanding of what it means to become a Jewish adult.

15. To explore these issues… KI has decided to introduce an alternative mitzvah project experience—a BMR experiment—inwhich a new model of participation in social justice activity will be offered.

16. Called The Journey to Jewish Adulthood, this alternative mitzvah project participation will: • give both parents and children an opportunity to become more connected to the congregation  • enable students to gain an understanding of the connection between their upcoming b’naI mitzvah and the core mitzvot of Jewish adulthood

17. What Will Happen in 2013-2014 Select pairs of 6th grade religious school children and parents will participate in a monthly service learning activity in one of KI’s existing core congregational activities: Broad Street Ministry KI Mitzvah Garden Rydal Park Shabbat Service KI Caring Committee One-on-One Torah Study

18. KI’s Research Questions 1. What impact does participation in KI’s Journey to Jewish Adulthood track have on 6th grade families’ relationship with the congregation? 2. What impact does participation in the Journey To Jewish Adulthood track have on 6th graders’ parents perception of the connection between their children’s mitzvah work and the bar/bat mitzvah?

19. Literature Review (optional!) What research or writing by others might have bearing on the Journey to Jewish Adulthood project? Note: The “literature” may be from both formal (e.g,, research journals) and informal (e.g., magazines and blogs) resources.

20. Potentially Relevant Literature • Articles about the history of the mitzvah project and its use in congregations • Research about parents of teens and their involvement in community • Research about the impact of social justice work on adolescent identity • Research about early adolescence as a time of learning about the relationship between self and community

21. Short-Term Action Research Question + Rationale As part of KI’s overall BMR research process, what did KI’s BMR team decide to study first? In what ways did they think the findings from this inquiry might inform future behavior?

22. Short-Term Research Question What can current 8th grade parents tell us about the experience of the mitzvah project for themselves and their children?

23. Rationale for this Inquiry Gaining 8th grade parent feedback will provide a basis for explaining to 5th grade families why we are restructuring the KI mitzvah project process to become more closely aligned with: a) b’nai mitzvah as a whole b) the practice of core adult mitzvot in particular

24. The Short-Term Research Sample Who was studied? How big was the whole group? How many/what percent of the whole group did this research focus on? What characterized the sample that was studied?  (e.g., gender, age, grade level)

25. The Short-Term Research Sample • 8th grade parents whose children became b’nai mitzvah from May 2011 - December 2012 • 53 parents (from 31 families) received email invitations to give feedback • More mothers than fathers were contacted due to the availability of their email addresses • 21 parents (40%) responded to the survey • 15 parents (12 families) attended 2 focus group meetings chaired by a parent volunteer

26. Quantitative Methods • What quantitative methods were used to collect data? • How were the quantitative data documented? • How were the quantitative data organized? • How were the quantitative findings analyzed?

27. E-mail SurveyQuantitative Items Checklists • Type of mitzvah project • # of hours devoted to mitzvah project • Duration of the project Ratings • 5-point scale about impact of the mitzvah project experience on the child • 5-point scale about impact of the mitzvah project experience on the parent

29. Things happen…. A similar item asking parents to rate the impact of the b’nai mitzvah project on the parents themselves inadvertently “disappeared” from the email survey.* *Welcome to the world of research

30. Analyzing Quantitative Data What was learned from the quantitative data?

31. Type of mitzvah project (multiple answers) Hands-On Service 16 Fundraising 8 Education/Raising Awareness 10 Collections 3 Other 2 # of hours devoted to mitzvah project 11-15 hours 6 16-20 hours 4 20-25 3 More than 25 hours 5 Other 2 Duration of the Project 3-5 individual days 4 Period of 2-3 Months 3 Period of 3-6 Months 4 Beyond 6 Months 4 Other 5

32. Descriptive Statistics A set of numbers (sometimes referred to as coefficients) that provide a brief description of a given data set. These statistics may describe an entire population and/or a sample. The measures used by researchers to describe data sets are • Measures of central tendency (e.g., mean, median, mode) • Measures of variability or dispersion (e.g., standard deviation, skewness).

33. Mean 1. This project gave my child an opportunity to spend time doing something meaningful with a parent. 4.00 2. This project helped my child understand the Jewish value of Tikkun Olam (repairing the world). 3.75 3. This project helped my child learn what it means to be a Jewish adult in preparation for B'nai Mitzvah. 3.60 4. This project gave my child an opportunity to engage in conversations about morals and values. 3.40 5. This project strengthened my child's connection to the synagogue. 2.60

34. Qualitative Methods What qualitative methods were used to collect data? How were the qualitative data documented? How were the qualitative data organized? How were the qualitative findings analyzed?

35. E-Mail SurveyQualitative items • In your opinion, what seemed to stand out to your child about the experience with the project? • What stood out to you, as a parent, about your experience with the project? • What other impacts, if any, do you think the mitzvah project had on you or your child? Were there negative impacts, as well as positive ones?

36. Focus Groups Qualitative Items • What stands out about the B’nai Mitzvah project as part of the overall experience at KI? • What impact do you think the Mitzvah project had on your child’s journey towards becoming a Jewish adult? • What discussions did you and your child have around the Mitzvah project? • Should the Mitzvah project move towards more familyinvolvement? • What suggestions do you have about the role the synagogue should/could play?

37. Analyzing Qualitative Data How do you “code” answers to open-ended questions?

38. Simple Steps for Coding Responses • Assemble all responses • Identify themes, repeated concepts and words/phrases • Generate categories • Code responses according to categories • Hold onto “uncodeable” responses for future analysis

39. What stood out about your child’s mitzvah project experience?

42. What about the Outliers —Responses that Can’t Readily Be Coded Example: “Our son is a high functioning autistic spectrum child (Asperger’s). He had fun tossing the boxes to the people in line and eating the snacks. He did connect with the bigger picture but in such a more subtle way than he can express.”

43. Findings • What have we learned from the data? • What has this research revealed so far?

44. Findings About KI’s 8th Grade Parents’ Perceptions of Their Children’s Mitzvah Projects1. Kids liked doing the projects. Parents liked watching the kids take satisfaction from doing this type of meaningful activity. 2. The projects enabled the students to give back, to see how they could make a difference, to help others less fortunate They had personal connections with the projects which made the projects richer. 3. The projects opened the kids’ eyes; they came in contact with people they would not have otherwise, situations they did not know about. 4. The projects gave students an opportunity to demonstrate leadership. It was a real experience, they had to be responsible, realized the need to make a difference.

45. More Findings about KI’s Parents’ Perceptions…. • In the initial mitzvah project meeting the assistant rabbi talked to the students about what they are good at and passionate about. Because of this, projects tended to focus on a passion of the student’s rather than the true need of the community. • Some parents reported that their families struggled with what project to do and felt they did not receive sufficient guidance after the initial meeting for making a decision. • Some parents expressed a need for “closure” or “debriefing” about the experience and how it connected to B’nai Mitzvah. • Some parents valued that the child chose a project that • related to something that was important to their family • (e.g., raising money for ALS because family member suffered • from this) • Some families were able to identify a project and see it to • completion while others struggled with the process.

46. Discussion & Implications • What do we make of the findings? • What are some of implications from what’s been learned so far? • What are some questions for future research? • How can these findings be shared with the community?

47. Some Implications from the Findings • The mitzvah projects had little to do with the synagogue and did • not enhance the students connection to the synagogue or Judaism. • Accordingly, the new structure will need to involve a change in • parents’ expectations and how they need to be educated about the BMR agenda. • The kids did not understand that their work was Tikkun Olam; • accordingly the new project will need to have a more significant teaching, planning and reflection process. • The process of asking students what they are passionate about • creates a focus on the child rather than a focus on the world and • what is needed. Accordingly, we will offer projects that the • congregation has already identified as ideas for the students. • We will use these findings to explain to the 5th grade • families why we are giving them an alternative b’nai mitzvah • process called Journey to Jewish Adulthood.

48. Future Research What questions now arise that might lead to future research? In what ways do these questions tie back to your institutional vision and to the original research question?

49. Sample Questions for Future Research • Will a change in the process lead to a better • connection between students mitzvah work • and their B’nai Mitzvah? • Will a change in the process alter what parents • report about the the significance of the work or • the level of responsibility felt by the students • and their families? • Will doing the projects with other KI families and • KI congregants who lead the projects create a • better connection to the synagogue in specific • and to Judaism overall?

50. Community Engagement • How can the research findings be used to help people in your congregation begin to share your vision and to appreciate the changes already instituted? • How might you broadcast what you have learned so far? • How will you get feedback from the congregation as you move along?