CE 00901 -3 fyp !

1. CE 00901 -3 fyp ! Research & Domain Analysis Diane Bishton – K229 (d.k.bishton@staffs.ac.uk) Part 1 1) Project Management, and 2) Documentation

2. 1 Project Management Time management - one of the commonest problems faced at level 3 (so students’ say) The key to success at level 3 Don’t just use it for your fyp How can you hope to Control what you’re doing without a Plan ? If you’re not in control, who (or what) is ? Planning - and, more importantly USING the plan AS A TOOL - is indicative of a professional approach

3. So little time, so much to do… How many weeks ? Float ? We’re now in the third week of your first semester If each semester is 12 weeks long, that gives you 20 weeks left including this one (maximum) but… • You will hand your final report in BEFORE the end of the second semester • There’s always the holidays (really ? ) revision ? actually taking a a break ? finishing other assignments ? further reading for your other modules ? (yes, you are expected to do these things as well))

4. Making a Plan Gantt chart but NOT MS ProjectTM - it’s not appropriate, takes up a lot of your time to set up, so probably won’t get altered (or even looked at ?). This is YOUR project, make the Plan WORK for you.

5. On Having a Time Machine Unfortunately, they don’t exist … as yet. You can’t get back days that have gone. What evidence will you have for the time you’ve used ? You NEED products to : • make you feel better, that you’re achieving • to show to supervisors or assessors & give them confidence in your progress • to cut down your work later on …

6. Regularly Review your Plan ! Frequent reviews & FOCUS • With 4 Modules per semester, that’s a day and a bit (in your working week) for each module. That’s ~ 12 fyp days in total left in semester 1, and (because fyp is 2 modules in semester 2), another 25 days in semester 2 (to hand in ??). Hmmm ! • So - if you spend 2 days (not much) reading a book, with nothing to show for it, that’s the equivalent of nearly 2 weeks work … GONE • Every time you overrun, it affects something else.

7. Closing thoughts Although this is YOUR project, there are other people involved occasionally too, so… • Make appointments well in advance -plan ahead ! Don’t forget - maybe the people you’ll need work somewhere that may be CLOSED or VERY BUSY when you think you need them - plan ahead ! You’re bound to find something that you’d wish you’d done, seen earlier etc. etc. >>Don’t panic<< • You still have your Critical Review section (because, despite how much you plan, no-one can predict the future)

8. 2 Documentation Imagine you're reading about 15 - 20 of these ! Keep your Documentation up-to-date from the beginning (remember Product Focus ?) You need to ‘distance’ yourself from the project if you’re going to appear ‘scientific’ in your approach No “I did…” or “…myself…” etc. , writelike an observer of the project Makeconnections between the Problem & your Solution EXPLICIT through Analysis & Design

9. Once Upon a Time … • Help your reader (s) • Treat them like intelligent, interested people who have a genuine desire to find out what you did, why, how, when… • Don’t forget that although you should know your project intimately, they (at best) only have an overview (and, they’re only human !) • Tell a story - have ‘signposts’ to join the different partstogether. These should be meaningful links between your Sections / Chapters • Organise. Arrange. • Ask yourself - Does it make sense ?

10. Writing a Report is a Process • Understand what your writing is for • Plan • Gather appropriate content • Select and Organise the Content • Prepare a Draft Copy • Review and Proof Read the Draft (after a break) • Final version

11. Planning (what, more !?) • Take the time to think about what you need to Produce • Add extra time to what you expect, for proof reading and review (and running out of paper, ink, PC collapsing under the strain etc.) • Have you got the binding sorted ? Do you REALLY want to be up all night panicking before the hand-in ?

12. Content • There are so many information sources now • Information Overload • Store your References as you go, in a spreadsheet - you won’t forget or lose them, and they’re easier to sort into order later. Include Chapter, page numbers & even paragraph - you may need to go back to the reference. • Make Lists, ‘Spider Diagrams’, Mind Maps of the possible content of your report • Your own preferred style will affect how you approach this

13. Mind Map From “The Brain Book” Russell, Peter

14. Good Practice • A clear Title helps you to focus • Make sure you’re using the right words - look up words you’re not sure of. Make a Glossary for us ! • Use a Thesaurus to make the writing more interesting and varied • One idea per paragraph • Try and sequence and link the Main Points • Shorter sentences are usually easier to read, but longer ones, with longer words, are considered to be more ‘academic’

15. FLOW ! Draft Sitting near an open window cools your brain, but that’s the wrong sort of draught To make a ‘rough’ copy, • Start with some possible Headings, include at least : • a form of Introduction, • then Main Points with Examples (evidence) • (one Main Point per paragraph) • then a Summary or Conclusion (Remember the ‘signposts’ bit earlier ?) • Your aim is to get the ‘flow’ right, so leave ‘fiddly’ (and really difficult) bits until later

16. Take a Break • You’ll find it much more effective to Review your work after you’ve had a break (maybe a couple of days) • Because you’ve written it, you’re very close to it, so it’s harder to spot errors • At least with a PC, it should be easier to make changes, but you need to keep this in perspective. You have got other things you need to do. Start good habits now.

17. Review • Really try and concentrate on what you’ve written • Read it out loud - does it make sense ? • Make changes (but make sure you’ve got a copy first - you might make it worse - eeek !) • Put in the fiddly bits, and illustrations • Check that the writing has a clear focus on the Main Points. Aim for Quality, not Quantity. • Go back to the requirements and make sure that you’ve covered everything

18. Proof Read • Preferably, get someone who you can rely on to do it ! • The chances of you finding anyone ? • (and anyway, they’ll claim it sounds great, when they don’t want to, or can’t, understand it) • If you don’t proof read it, whoever assesses it probably will, but it’s too late then isn’t it ? • Check spelling, grammar, page numbering, references, figure and table labelling & captions etc.

19. Final Version • Check the Contents Page • Make sure all the Appendices are in, and properly referenced • Make sure your list of References is in the Bibliography • When it’s printed, make sure that it’s right justified enough,so that when it is bound, what’s on the left can still be read ! Personally, I prefer Appendices separately bound - that way I can refer to them without having to keep turning back & forth through the main report. But that’s me.

20. A typical (general) Report Structure A Report is more formally structured than most other forms of writing. There are Conventions for breaking reports down into manageable chunks. • Contents Page • 1.0 Introduction / Terms of Reference • 2.0 Methods or Procedures • 3.0 Findings • 3.1 • 3.2 • etc • 4.0 Summary or Conclusions • 5.0 Recommendations • Bibliography • Appendices

21. Part of a Technical (academic) paper From “Semiologic From Sending Messages to Managing Meetings” Stewart, Liz Director of Semiologic Ltd

22. Some real student ‘bloopers’ “In the Olympic Games, Greeks ran races, jumped, hurled the biscuits, and threw the java” “Joan of Arc was burnt to a steak” “In midevil times, most people were alliterate” “Sir Francis Drake circumcised the world with a 100 foot clipper” “Christopher Columbus discovered America while cursing about the Atlantic” “Queen Victoria sat on a thorn for 63 years” “Louis Pasteur discovered a cure for rabbis”

23. Punctuation ! ! ! and scope Virgin Rail trains will be running normally, late this afternoon Virgin Rail trains will be running normally late, this afternoon Serve hot or cold with cream

24. The nitty gritty You’re given a Font style, size etc. so use it ! Reading 15-20 tiny font, poorly spaced reports makes ones eyes hurt. This may translate to ones impression of the overall quality of the product & hence to ones judgement. Hmmm. Spell check & Proof Read. Do NOT rely on a computer to pick out all the errors “… there was a need to discus…” !

25. Summary We’ve seen that : It is important to Plan your writing, carefully considering the purpose of it and the target audience Producing a Draft Copy can get your ideas organised and ultimately help you to feel better Reviewing and Proof Reading are really important things to do to improve quality

26. CE 00901 -3 fyp ! Research & Domain Analysis Diane Bishton – K229 (d.k.bishton@staffs.ac.uk) Part 2 1) Literature Searching, 2) Data Gathering & Recording

27. 1 Literature Searching Literature searching - keep focused Scan for likely sources, then USE the Precis of a Paper & the Contents of a Book - they summarise the main points. Don’t get sidetracked (easy for me to say !) Use HARVARD referencing (advice leaflets in the library) Quotes ?- well, a few are alright , but most of the writing MUST be your own. Guess what - it’s fairly easy to spot changes in writing style…so… Plagiarism- DON'T. It’s MUCH better to acknowledge your sources - you won’t be penalised for that

28. 2 Data Gathering & Recording What do you want that data for anyway ? What Purpose will it serve ? Categories ? - when you include, by definition you exclude. If numbers model the world, what numbers do you choose ? What’s so important about the different groupings you’ve chosen - what features differentiate or characterise them ? Make sure you keep proper records of the context of your data capture - dates, units etc.

29. Research design - essentially a 5 stage process : 1) Problem formulation 2) Case selection (e.g. by experiment, survey, case study) 3) Data production (e.g. by observation, questionnaires) 4) Data analysis 5) Report writing

30. These 5 stages translate into : • Set a Hypothesis • Decide how to test the Hypothesis • Design a Task • Decide on Data Collection • Identify Variables & Sources of Bias • Determine Independent Variables (that will be manipulated) • Determine Dependent Variables (that will be ‘observed’ for change) • Identify a suitable Pilot & Test population (these are the same in a Longitudinal Study) • Choose Result Analysis Method (s) • Choose Result Presentation Method (s) • Pilot the experiment • Review the Pilot & • Conduct the Actual experiment

31. Some definitions Independent variables : a factor being manipulated (changed) to produce different results for comparison The Hypothesis predicts that changing the Independent variable will affect the Dependent variable e.g. different Icon designs affecting Recognition time. Dependent variables : a factor being observed & measured examples – the number of errors made ; the Task ‘stages’ at which errors are made Control Group : This group’s ‘behaviour’ is compared against the experiment group

32. Where things go wrong In determination of the hypothesis to be tested (problem formulation), In selection of variables, measurement instruments, test populations (case selection and data production), In sampling and analysis of data and elicitation of findings (data analysis), In presentation of the results (report writing)

33. Things to remember All data rely on theoretical presuppositions. Results and conclusions need to be at least : Credible – capable of being believed / trustworthy, and Plausible – apparently reasonable. It is necessary to give the recipients of research findings adequate detail about the process, the context and the decisions made. ‘Triangulation’ of data by going to different sources has the potential to ‘cross correlate’ findingsfor verification. So: measurewhat can be measured, observe (carefully – ‘presented’ behaviour is only one possible manifestation) , and ask users for their assessment.

34. Your ‘experiments’ May reveal a number of issues, for example of : Lack of ‘Control’ – field based versus lab based - ‘unnatural’, ‘artificial’ Bias – practice or learning effects - ‘order’ effects Qualitative versus Quantitative ‘scientific’ measurement (In many ways, both quantitative and qualitative studies are tentative)

35. Your ‘experiments’ cont. Who is doing the ‘measuring’ ? People • Highly variable, adaptive and subject to context effects. They will SELECT and PREFER, perhaps subconsciously, and can ‘over’ or ‘under’ emphasise features. • Often unable to describe or ‘rationalise’ their particular preferences / observations – professional judgement – since ‘skill’ is cognitively a highly organised and SUBCONSCIOUS behaviour. • A different experimenter, or even the same one on different days, may ‘code’ an observation under a different category. Rating scales ?

36. Some real sample questions From a questionnaire for evaluating computer usability - NPL DITC 169/90 (Wong and Rengger, 1990) “Please remember : there are no right or wrong answers, no “trick” questions, and your answers are completely confidential” “To which one of the following age groups do you belong ? Under 25 25-45 Over 45” what do you think ? “How do you rate your knowledge of computing ?” “Low Medium High” what about this one ?

37. Some real sample questions (cont.) “On this page you will find 10 statements of opinion… Would you please rate each statement to show your agreement or disagreement with it. You should use the following rating scale : 1 Strongly disagree… 3 Have no opinion either way… 5 Strongly agree…” “I find EDIT unnecessarily complex 1 2 3 4 5” and then later “I think EDIT is easy to use 1 2 3 4 5” Look how they’ve included statements that are ‘opposite’ one another to draw out inconsistencies in respondents’ choices

38. Some real sample questions (cont.) “Each section consists of several statements which can be rated on a scale of 0 to 9. Your ratings will therefore indicate your impressions of certain characteristics of each aspect of EDIT use. For instance : Organisation of information on computer screen Confusing 0 1 2 3 4 5 6 7 8 9 Very clear … Tasks can be performed in a straightforward manner Never 0 1 2 3 4 5 6 7 8 9 Always” Look how they’ve put all the ‘negative’ responses on the left & phrased the statements in such a way that they all come out like that - people don’t always read what’s there !

39. Some real sample contexts From a test context document that assesses whether users’ skills / competence may affect the judgement of a product’s usability- NPL DITC 213/92 (Bowden & Kirakowski, 1992) “Two types of user will be tested, scientific staff… and trained task experts…Potential subjects will be questioned about their usage of…Anyone who has received any formal training…, does not use at least the basic functions on a regular basis…,is not fluent in English…, or has significant sight or hearing deficiencies…, will be rejected as a subject”

40. Some real sample contexts (cont.) “The evaluation tasks for all the subjects will be to…” “…evaluators …will use a script to ensure that spoken communications are kept to a minimum” “Prior to carrying out the tasks the subjects will be asked how frequently they do these tasks in their normal job…” “The tasks will be carried out at… between 9am and 5pm from Monday to Friday…” “The tests will take place in the Usability Laboratory in building 93… with the test room set up as an office…” “A telephone… and other conventional office furniture will be installed in the test room…The telephone will be located on a desk…at which the subjects will be seated…”

41. Some real sample contexts (cont.) “Context of Use & Measurement of the Product” “Context of UseLikely to Affect Usability Context of Measurement “User TypesFix U1 Scientific Staff… Yes only type 1 U2 Admin Staff… tested Skills & Knowledge Experience with the productMonitor Regular Daily Usage Yes ask subjects Typing skill varies… No Training None Yes Select only subjects who have received no formal training”

42. Summary We’ve looked at • The typical stages in the process of designing an experiment • Definitions of terms, such as ‘Independent Variable’ etc. • Points where things go wrong in experiment design • Things you should keep in mind when designing your evaluation ‘experiments’, if you want the results to be considered ‘Valid’