How to Make a Valid and Unbiased Survey

1. How to Make a Valid and Unbiased Survey

2. First Step • Brainstorm… • What is the topic of your survey • What is the objective, or what information are you trying to obtain • Make sure your questions specifically address the objectives you are trying to learn, for ex: If you are trying to make a survey on drug usage for high school students, you might ask, “Have you ever tried Marijuana?” and if so, “How often do you smoke marijuana?”

3. Who is the audience? • If you are going to ask a small group you can ask everybody (called a Census) • If you want to survey a large group, you may not be able to ask everybody so you should ask a sample of the population (this is called Sampling)

4. What does bias have to do with it? • If you are Sampling you should be careful who you ask, for example: • If you only ask people who look friendly, you will only know what friendly people think! • If you went to the swimming pool and asked people "Can you swim?" you will get a biased answer ... maybe even 100% will say "Yes" • The surveys where people are asked to ring a number to vote are not very accurate, because only certain types of people actually ring up! So be careful not to bias your survey. • Example: You want to know the favorite colors for people at your school, but don't have the time to ask everyone. • Solution: Choose 50 people at random: • stand at the gate and choose "the next person to arrive" each time • or choose people randomly from a list and then go and find them! • or you could choose every 5th person Your results will hopefully be nearly as good as if you asked everyone.

5. Now that you have your audience, Create the survey… • Types of Questions • A survey question can be: • Open-ended (the person can answer in any way they want), or • Closed-ended (the person chooses from one of several options) • Closed ended questions are much easier to total up later on, but may stop people giving an answer they really want. • Example: "What is your favorite color?" • Open-ended: Someone may answer "dark fuchsia", in which case you will need to have a category "dark fuchsia" in your results. • Closed-ended: With a choice of only 12 colors your work will be easier, but they may not be able to pick their exact favorite color.

6. Question Sequence • It is important that the questions don't "lead" people to the answer • Example: people may say "yes" to donate money if you ask the questions this way • Do you love nature? Will you donate money to help the river? • But probably will say "no" if you ask the questions this way: • Is lack of money a problem for you? Will you donate money to help the river? • To avoid this kind of thing, try to have your questions go: • from the least sensitive to the most sensitive • from the more general to the more specific • from questions about facts to questions about opinions • Go through your questions and put them in the best sequence possible • Example: I will ask people how often they visit the river (a fact) before I ask them what they feel about pollution (an opinion) • I will ask people their general feelings about the environment before I ask them their feelings about the river.

7. Make your questions Neutral • Your questions should also be neutral ... allowing the person to think their own thoughts about the question. • In the example before I had the question "Do you love nature?" ... that is a bad question because it is almost forcing the person to say "Yes, of course." • Try rewording it to be more neutral, for example: • Example: "How important is the natural environment to you?" • Not Important • Some Importance • Very Important

8. Tally up the data or the results…. • Tally the Results • Now you have finished asking questions it is time to tally the results. • By "tally" I mean add up. This usually involves lots of paperwork and computer work (spreadsheets are useful!) • Example: For "favorite colors of my class" you can simply write tally marks like this (every fifth mark crosses the previous 4 marks, so you can easily see groups of 5):

9. The Results…. • Look at your survey results for trends and patterns. • What conclusions can you draw from the survey results? • Write a conclusion, and include any tables or graphs that help you present your survey results

10. Ways to present your results • Tables • Sometimes, you can simply report the information in a table.  • A table is a very simple way to show others the results.  A table should have a title, so those looking at it understand what it shows: • Statistics • You can also summarize the results using statistics, such as mean or standard deviation • Example: you have lots of information about how long it takes people to get to school but it may be simpler just to present a summary such as: • Shortest Journey: 3 minutesAverage Journey: 22 minutesLongest Journey: 58 minutes

11. Graphs • Nothing makes a report look better than a nice graph or chart • There are many different types of graphs.  Three of the most common are: • Line Graph  - Used to show information that is somehow connected (such as change over time) • Pie Chart - Used most often to show survey data that is to be reported in percentages.

12. People's Comments • If people have given their opinions or comments in the survey, you can present the more interesting ones: • Example: In response to the question "How can we best clean up the river?" we received these interesting replies: • "The government has a special fund for this" • "The local gardening group has seedlings you could plant"

13. Resources • http://www.microsoft.com/education/DesignSurvey.aspx?pf=true • http://www.mathsisfun.com/data/survey-conducting.html