Usability test unesco org full report june 8 2006
Download
1 / 82

Usability Test UNESCO Full Report - PowerPoint PPT Presentation


  • 249 Views
  • Updated On :

Usability Test – UNESCO.org Full Report June 8 2006. Laetitia Giannettini [[email protected]] Usability Specialist Cyril Alegre [[email protected]] Usability Specialist. Table of Contents. Study Background and Objectives Methodology Executive Summary Detail of Results

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Usability Test UNESCO Full Report' - Sharon_Dale


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Usability test unesco org full report june 8 2006 l.jpg

Usability Test – UNESCO.org Full ReportJune 8 2006

Laetitia Giannettini [[email protected]] Usability SpecialistCyril Alegre [[email protected]] Usability Specialist


Table of contents l.jpg
Table of Contents

  • Study Background and Objectives

  • Methodology

  • Executive Summary

  • Detail of Results

  • Main Recommendations



Background of the study and objectives l.jpg
Background of the Study and Objectives

  • The United Nations Educational, Scientific and Cultural Organization (UNESCO) is a specialized United Nations agency that promotes peace through Education, Social and Natural Science, Culture and Communication programmes across the world.

  • UNESCO’s Bureau of Public Information (BPI) is the service in charge, among other functions, of public relations and providing audiovisual media. Its also in charge ofthe administration and editorial coordination of the UNESCO web portal: www.unesco.org.

  • The web portal aggregates a great amount of distinct UNESCO websites with different contents and look and feel. BPI wishes to create astandard setting and a capacity building process to harmonize standardize these websites.

  • A framework has been developed to migrate all the websites’ pages into a new look and feel essentially based on usability and accessibility criteria and previous user tests. The home page and the Worldwide and Organisation sections have been migrated so far.

  • In addition, the UNESCO portal now provides content in 6 languages: Arabic, Chinese, Russian and Spanish (the four other official languages) were added to French and English (the Organization’s working languages languages).


Background of the study and objectives5 l.jpg
Background of the Study and Objectives

  • BPI wished to assess the perception and understanding of the portal by the representatives of the UNESCO Executive Board

  • The study’s objectives were to assess:

    • Perception and understanding of the home page and its main items (access, tools, navigation…)

    • Perception and understanding of pages in the main sections (Themes, Worldwide, Organisation, Services)

    • Opinions toward the online version of the UNESCO Courier

    • Opinions toward the availability of multilingual contents

    • Opinions toward the legitimacy of BPI in coordinating the portal

  • The applied methodology was user testing (see section 2. Methodology)


Background of the study and objectives6 l.jpg
Background of the Study and Objectives

  • This actual user testing follows a previous one, which took place in February 2004

  • The main objectives of the first study were to assess the utility (usefulness) and usability of the portal by UNESCO’s target public.

  • 18 users divided into 6 profiles (3 participants per profile) were consulted for the first study:

    • Researchers/Scientists/Academics

    • Media

    • Teachers/Educators/Trainers

    • Decision-makers/Civil Servants in Supervising Ministry/Governing bodies

    • Development agents (NGOs, IGOs, UN)

    • Students/Young people



Background of the study and objectives8 l.jpg
Background of the Study and Objectives

  • User testing is mostly a qualitative method based on observational and oral data

  • It assesses users’ experience through two major criteria:

    • usability (usefulness) of the website: is the website easily and efficiently usable?

    • utility of the website: does the website meets users’ needs and expectations (contents, tools, etc.)?


Methodology usability testing l.jpg
Methodology > Usability Testing

  • Individual sessions in three steps:

    • Step 1: Pre-test questionnaire

      • User’s profile and their use of the UNESCO web portal

    • Step 2: Website browsing

      • perception and understanding of the home page

      • perception and understanding of pages in the main sections

      • opinions toward the online version of the UNESCO Courier

    • Step 3: Post-test questionnaire

      • General opinion of the website

      • Performance and satisfaction rating (questionnaire for performance indicators)

  • Duration of Sessions : variable – between 10 and 45 minutes according to the participants availability


Methodology user profiles l.jpg
Methodology > User Profiles

  • User profiles (cf. Excel spreadsheet for details) :

    • 55 participantsto sessions of the UNESCO’s Executive Board

    • 22 female and 33 male

    • From 39 Permanent Delegations (cf. list on next slide)

    • Age 22 to 66

    • 30 users saw the portal in English and 25 users saw the portal in French


Methodology user profiles11 l.jpg
Methodology > User Profiles

  • Participants to this test are from the following Countries:



Strong points l.jpg
Strong Points

  • Plenty of information available – the portal is a useful working tool

  • Some information is available in several languages

  • The home page layout is clear

  • The work done in order to add some consistency across the portal (the Framework) was noticed and appreciated

  • The “Worldwide” section is found useful (the possibility to find information about members states, its programs)

  • The online UNESCO Courier was appreciated


Weak points l.jpg
Weak Points

  • Guidance needs some improvement

  • The portal is rather expert-oriented – outsiders and new comers need some time to understand it

  • Some contents are outdated or missing

  • Users have difficulty finding certain information and documents and the search engines are not efficient enough

  • There is still a lack of consistency between the different sections – this hurts understanding and learnability

  • Users expect more multilingual content



Results details severity ratings l.jpg
Results Details – Severity ratings

The results are classified according to their severity. This classification should also be seen as an emergency rating – the higher the number the sooner rework should be done:

1

Good point

Graphical issue

2

Minor usability issue

3

Medium usability issue

4

Major usability issue


Results details table of contents l.jpg
Results Details – Table of contents

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


A pre test questionnaire l.jpg
A. Pre-test questionnaire

  • The age spread was as shown in the graphic opposite

  • The average age was around 41

  • Half of the participants had spent more than 2 years working for UNESCO

  • The average seniority was around 3.5 years


A pre test questionnaire19 l.jpg
A. Pre-test questionnaire

  • Most participants were web experts: 83% of them have been using for more than 5 years – on average for 10 years and for 16 hours per week nowadays

  • Spreads as shown in the graphic opposite


A pre test questionnaire20 l.jpg
A. Pre-test questionnaire

  • Almost all participants had already browsed UNESCO.org.

  • Most often, participants started using the portal as they started working at UNESCO. On average, they have been using it pretty often (on a daily basis for most respondents) for 4.5 years

  • Spread is as shown in the graphic opposite


A pre test questionnaire21 l.jpg
A. Pre-test questionnaire

  • Mostly, the participants browse the portal to find information and documents. The most visited sections are:

    • Themes (mentioned by 22 participants) to get information about UNESCO’s programs and activities

    • Organization (22) to collect information and documents from the Executive Board, the General Conference (meetings, resolutions, speeches…)

    • Services (20) to find some documents and publications (conventions, resolutions…) and information about legal instruments

    • News (14) and Events (11)to check what’s happening (UNESCO’s activities, exhibitions, conferences…)

    • Communities (7) to find contacts


A pre test questionnaire22 l.jpg
A. Pre-test questionnaire

Strong points

  • The main strong points reported by the participants according to their own experience were:

    • The great amount of information availableThe portal is seen as a useful source of information to keep up to date about UNESCO news, about “what’s currently going on at UNESCO”. Furthermore, the UNESCO portal is often seen and used as a regular working tool: some users, for example, use it to collect information in order to prepare Executive Board meetings

    • The overall ease of use and easy access to the information

    • The contents available in 6 languages


A pre test questionnaire23 l.jpg
A. Pre-test questionnaire

Weak points

  • The main weak points reported by the participants were:

    • Overall complexity, especially for outsiders or newcomers – the portal was often said to take some understanding of UNESCO structure to be efficiently used

    • Difficulty in finding certain information and documents: all the participants said they could find most of what they were looking for but many also reported some problems in finding some information

    • Outdated contents: many users deplored the slowness – or lack, at worst – of updates

    • Lack of multilingual contents


A pre test questionnaire24 l.jpg
A. Pre-test questionnaire

  • Most participants were aware that BPI was in charge of the UNESCO website

  • Most of them also had some general knowledge of BPI activities:

    “Press information, Public Relations” (Holy See)

    “Publishing, Communications” (China)

    “Public information about UNESCO’s activities” (Italy)

    “Promotion of UNESCO’s image” (Sri Lanka)

  • Other main activities reported: website administration, content feed, updates and coordination


Results details table of contents25 l.jpg
Results Details – Table of contents

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


B perception and understanding of the home page l.jpg
B. Perception and understanding of the home page

The home page is clear

  • Most users found the home page layout clear and user friendly

  • They understood the 3-column layout and the left and right menus

  • The sections’ organisation and the wordings were well understood

    “It’s clear, one can find almost everything he needs” (China)

    “It’s practical, I can find what I want” (Indonesia)


B perception and understanding of the home page27 l.jpg
B. Perception and understanding of the home page

The home page is clear

  • The languages available on the home page were well perceived and the work done to provide multilingual contents highly appreciated

    “They are very useful” (Italy)

  • The tools are well perceived – the search engine is the most attractive

    “Practical, I find what I want” (Indonesia)


B perception and understanding of the home page28 l.jpg
B. Perception and understanding of the home page

The home page is regularly kept up to date

  • The home page is seen as being kept up to date (with changing pictures, themes, news and events), which gives the website a dynamic look that was appreciated


B perception and understanding of the home page29 l.jpg
B. Perception and understanding of the home page

3

High informational density

  • Users found the home page had too much information and links on it, especially in the right menu

    “Too many things – I can’t get any information” (Holy See)

    “It’s compact, there are too many things on it, which distracts one’s attention – even more for someone browsing it for the first time” (India)

    “The right menu offers too many things – nothing shows up” (Switzerland)


B perception and understanding of the home page30 l.jpg
B. Perception and understanding of the home page

3

High informational density

  • A few users did not understand the wording “Communities” nor anticipate the content of this section

  • The right menu is too heavy for non-UNESCO users – it takes some good knowledge of the UNESCO organisation to understand the menu

  • The high density in the right menu restrains the visibility of the links of interest – they do not attract attention (e.g. “General Conference” and “Executive Board”)


B perception and understanding of the home page31 l.jpg
B. Perception and understanding of the home page

3

High informational density

  • Hide the submenus (under “Secretariat”) – and make it available by a click on “Secretariat”?

  • By default, the Organisation menu should only offer “About UNESCO”, “General Conference”, “Executive Board”, “Director-General” and “Secretariat”

    “The home page should focus: it should only give access to UNESCO’s main sectors and organs” (India)

    “The ‘General Conference’ and ‘Executive Board’ links should show up more” (Switzerland)

  • This 3 column layout is more adapted for 1024 resolution.


B perception and understanding of the home page32 l.jpg
B. Perception and understanding of the home page

The home page does not communicate UNESCO’s spirit enough

3

  • A few users deplored the fact that the home page like the whole website did not convey efficiently UNESCO’s mission, values and messages

    “I would prefer a global picture in the central area, a more universal look to it” (Kazakhstan)

    “The home page fails at showing what UNESCO is” (Canada)


B perception and understanding of the home page33 l.jpg
B. Perception and understanding of the home page

The home page does not communicate UNESCO’s spirit enough

3

  • The home page content is rather insider-oriented: non-UNESCO users may not get a good insight of what UNESCO is and does

  • A baseline on top of the page could provide a good introductory overview of UNESCO’s definition missions:“UNESCO - the United Nations Educational, Scientific and Cultural Organization (UNESCO) was founded on 16 November 1945. For this specialized United Nations agency, Education, Social and Natural Science, Culture and Communication are the means to a far more ambitious goal : to build peace in the minds of men”


B perception and understanding of the home page34 l.jpg
B. Perception and understanding of the home page

The home page does not communicate UNESCO’s spirit enough

3

  • The central area does not give an overview of UNESCO’s core activities (i.e. key international programs)

  • The central area is not well perceived, it doesn’t catch UNESCO people’s attention a lot – insiders are more attracted to the menus

    “I don’t browse the central area” (Holy See)

    “It doesn’t catch my attention” (Kazakhstan)


B perception and understanding of the home page35 l.jpg
B. Perception and understanding of the home page

The central boxes’ layout is confusing

3

  • The central area layout is not clear:

    • The articles in the “News” and “Events” boxes are separate with white lines and look like they’re discrete boxes – therefore, the second article, for example, does not seem to be part of the news

    • The boxes should be united


B perception and understanding of the home page36 l.jpg
B. Perception and understanding of the home page

Some accesses are not perceived

2

  • The themes tabs on top of the page were not well perceived by some users – the lack of consistency across the portal restrains learnability of the navigation:

    • Different graphical aspect

    • Different tabs – some sites offer an “The Organisation” tab or a “Site map” tab, which could be extended to the whole portal


B perception and understanding of the home page37 l.jpg
B. Perception and understanding of the home page

The home page does not mention some current events

2

  • Some users deplored the fact that the home page did not mention the ongoing Executive Board (Palestine, European Commission)


B perception and understanding of the home page38 l.jpg
B. Perception and understanding of the home page

The French “Plan” tab is not well designed

2

  • The tab’s icon and label are so far from each other that they don’t seem to be linked together

  • The site map icon needs some graphical readjustment


Results details table of contents39 l.jpg
Results Details – Table of contents

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


C perception and understanding of the worldwide section l.jpg
C. Perception and understanding of the “Worldwide” section

The “Worldwide” access was well perceived and appreciated

  • The “Worldwide” access was well perceived by most participants

  • It was also well understood: most participants had a good insight of what the section offers – local information about UNESCO organisation and programmes

  • The Worldwide section available from the left menu was generally appreciated as a useful source of local information


C perception and understanding of the worldwide section41 l.jpg
C. Perception and understanding of the “Worldwide” section

The “Worldwide” access is mostly clear

  • The “Worldwide” accesswas easily used by most users – they had no difficulties accessing the page dedicated to their country


C perception and understanding of the worldwide section42 l.jpg
C. Perception and understanding of the “Worldwide” section

The “Worldwide” access is also confusing

2

  • A few users got confused by the“Worldwide” access, which follows UNESCO’s own logic in aggregating regions of the world, which does not match the common logic about world’s continents and therefore may confuse non-UNESCO people – e.g. Europe and North America form a single region and the Russian Federation and Turkey are listed twice, both in the “Europe and North America” and the “Asia and the Pacific” regions.


C perception and understanding of the worldwide section43 l.jpg
C. Perception and understanding of the “Worldwide” section

The “Worldwide” access is also confusing

2

  • Several interpretations were given as for UNESCO’s own logic in aggregating regions of the world:

    • Regions are voting groups

    • Regions follow a “political” logic


C perception and understanding of the worldwide section44 l.jpg
C. Perception and understanding of the “Worldwide” section

The cross-referencing of information principle was appreciated

  • The availability of country-specific information (themes) was mostly appreciated


C perception and understanding of the worldwide section45 l.jpg
C. Perception and understanding of the “Worldwide” section

The themes were not easily perceived by users

3

  • Some users did not notice that some more content was available in the lower area of the page

  • The page layout should allow users to notice the content below – the browser should “cut” the page where some content is partly visible in order to suggest some more content


C perception and understanding of the worldwide section46 l.jpg
C. Perception and understanding of the “Worldwide” section

The themes’ content was not easily anticipated by users

2

  • A few users deplored some difficulties anticipating the contents “behind” the themes links on the country-specific pages

  • The links are not clear enough, not understood as being contextual, country-specific

  • There is no clear introduction

  • Explanatory sentences should introduce the themes’ content (what about the type of links is)


C perception and understanding of the worldwide section47 l.jpg
C. Perception and understanding of the “Worldwide” section

The “Organisation” menu lacks visibility

3

  • Many users did not see the “Organisation” menu on the country-specific pages

    • The background colour and the search engine and “UNESCO at 60” picture above the menu make it “secondary” (Austria)

    • The “organisation” menu lacks consistency

    • “The Organization” label is not country-specific and the “UNESCO in this Country” label is not well located – it applies only to the 2 links underneath it


C perception and understanding of the worldwide section48 l.jpg
C. Perception and understanding of the “Worldwide” section

The “Organisation” menu lacks visibility

3

  • The menu should be moved upward in the place of the search engine and picture and gain consistency from one page/section to another

  • The menu label should be more specific


C perception and understanding of the worldwide section49 l.jpg
C. Perception and understanding of the “Worldwide” section

Content was sometimes assessed not to be full-accomplished

2

  • According to the users, some contents are poor or information is missing or not enough emphasised :

    • The countries’ main sectors of activity and the key programmes (Luxembourg, Greece, Holy See, Poland, Slovenia, Ethiopia, etc.)

    • The statistics should come along with interpretations (Indonesia)

    • The National Commission programme (Netherlands)

    • A link to the country website from the National Commission page (Argentina, Kazakhstan)


C perception and understanding of the worldwide section50 l.jpg
C. Perception and understanding of the “Worldwide” section

Some country-specific pages content is outdated

2

  • Some contents are not up to date – and some pages are still empty (Slovenia)

  • Some users (Slovenia, Luxembourg) suggested the National Commissions should be involved and provide BPI with information – it could be through a form –, given the National Commissions are in charge of defining their status, composition principles and main sectors of activities


C perception and understanding of the worldwide section51 l.jpg
C. Perception and understanding of the “Worldwide” section

Some country-specific pages pictures are disliked

2

  • Some users found the pictures could be more attractive (India, Luxembourg) and linked to the content


C perception and understanding of the worldwide section52 l.jpg
C. Perception and understanding of the “Worldwide” section

2

Text format is not adapted

  • The text is compact – its format should be improved in order to ease the reading:

    • The main words and sentences should be written in bold font

    • Text should be divided into different paragraphs

    • There should be more pictures on both side of the central area


Results details table of contents53 l.jpg
Results Details – Table of contents section

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


D perception and understanding of the unesco courier l.jpg
D. Perception and understanding of the UNESCO Courier section

The UNESCO Courier was appreciated

  • Most users would likely read the UNESCO Courier online

  • Most of them would like to print it out – be it in PDF format or directly from their Internet browser


D perception and understanding of the unesco courier55 l.jpg
D. Perception and understanding of the UNESCO Courier section

Users would diffuse the UNESCO Courier

  • All the users would likely participate in the diffusion of the UNESCO Courier, mostly via email – as a link, as a PDF document or using a “send this article to a friend” feature


D perception and understanding of the unesco courier56 l.jpg
D. Perception and understanding of the UNESCO Courier section

Several ways to access the UNESCO Courier

  • The website offers several accesses to the UNESCO Courier


D perception and understanding of the unesco courier57 l.jpg
D. Perception and understanding of the UNESCO Courier section

The UNESCO Courier central access is not well perceived

3

  • The UNESCO Courier box was not perceived: many users missed it although they had scrolled the page down to the bottom

  • The UNESCO Courier access should be easier: it should show up more, either visually (a specific colour, layout) and/or by its location (to be moved upward, in a menu?)

  • The title “The UNESCO Courier” should be clickable


D perception and understanding of the unesco courier58 l.jpg
D. Perception and understanding of the UNESCO Courier section

The UNESCO Courier layout needs improvement

2

  • The first paragraph is a general introductory mention to the online version of the UNESCO Courier

  • As such, it should be located on top of the page, above the first picture and first article

  • The “Free subscription” link should be located right under the first paragraph

  • The right menu is not clear enough: the first item is not clearly related to the Courier’s main theme – water

  • Put an introductory mention like “Read more (about this theme)” instead of “Features”


Results details table of contents59 l.jpg
Results Details – Table of contents section

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


B transversal issues l.jpg
B. Transversal issues section

The Framework was appreciated

  • The standardization work done so far and to be continued was appreciated for adding consistency to the portal and bringing better ease of use to the user experience: layout and navigational principles


B transversal issues61 l.jpg
B. Transversal issues section

3

Search issues

  • Many users reported difficulties finding what they want (mostly documents):

    • The website was often reported as requiring some learning – the content is seen as huge and the structure as complex

    • Searches by keywords and themes were reported to give poor results

    • The search engines don’t give acceptable results often enough – some participants use Google to find documents in the UNESCO portal

    • A French-speaking participant (Algeria) deplored the fact that the advanced search feature is only available in English


F perception and understanding of other main sections l.jpg
F. Perception and understanding of other main sections section

3

Document-finding difficulties

  • The “Executive Board” search engine suggests that only searches by document codes can be run here – some users weren’t aware that a search by keywords was also possible here

  • A few users wished they could run searches by themes


B transversal issues63 l.jpg
B. Transversal issues section

3

Late updates

  • Some information on the website is outdated – especially in the “Worldwide” and “Themes” sections

  • Some documents are uploaded and available on the website too late vis-à-vis UNESCO users’ needs during the Executive Board – these late updates add some difficulties in finding the desired information or documents

    “The information is brought out to late” (Algeria)

    “The meetings’ outcomes are not published right after the meeting is over” (Portugal)


B transversal issues64 l.jpg
B. Transversal issues section

Lack of consistency across the portal

3

  • The lack of consistency across the portal (different layouts) restrains learnability and understanding from one section to another


B transversal issues65 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • Page titles and layout fail to provide efficient guidance due to a lack of a logical and consistent principles in building titles:

    • The page title building principle should be the same through the whole portal

    • The page title building principle should always refer to the relevant menu building principle – i.e. the items’ organisation and order

    • The title of a page should always refer to the link that brought to the page and vice versa


B transversal issues66 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • On the first page shown on the left the subtitle doesn’t refer to any item in the menu on the right

  • On the second page, “About UNESCO” should remain visible as the page title while “The Organisation’s History” should be the subtitle

  • The titles and subtitles should be the same format – titles either bigger or smaller than the subtitles – through the whole portal


B transversal issues67 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • The last page of the breadcrumbs does not always refers to the last clicked link and the current page title

    • The last page of the breadcrumbs should always match the last clicked link and the current page title

    • Therefore the last page of the breadcrumbs should not be clickable


B transversal issues68 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • The breadcrumbs are usually located on top of the central area rather than above the left menu

  • Moreover, the breadcrumbs format (font type and size, background colour) is the same as the menu below, which reduces its visibility

    • Breadcrumbs should start reading above the central area in order to offer the best visibility possible

    • The background should be white


B transversal issues69 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • Some UNESCO sites do not permit going back to the portal home page

    • e.g. the World Heritage web site has the same UNESCO logo as the portal although this one leads to the World Heritage home page

    • The UIS web site offers no link to the portal


B transversal issues70 l.jpg
B. Transversal issues section

Lack of consistent and efficient guidance principles

3

  • The “Organisation” menu changes (content, format, location) from one page or section to another, which is confusing


B transversal issues71 l.jpg
B. Transversal issues section

Home page links not well perceived

3

  • The home page links – top tab, breadcrumbs and top left logo – were not well perceived

  • Most participants used the browser’s “Back” button


B transversal issues72 l.jpg
B. Transversal issues section

2

Acronyms are not clear

  • A few users deplored the lack of explanations about the acronyms used on the website


B transversal issues73 l.jpg
B. Transversal issues section

Expectations – Themes landing page

  • A few users wished the themes landing pages would provide users with a baseline mentioning UNESCO’s key goals and programmes in the relevant field


B transversal issues74 l.jpg
B. Transversal issues section

Expectations – Some extra content

  • Some users wished the website offered:

    • Yearly and monthly meetings schedules

    • Daily meetings summary


Results details table of contents75 l.jpg
Results Details – Table of contents section

  • Pre-test questionnaire – Users’ profiles and web habits

  • Browsing observation – Perception and understanding of the home page

  • Browsing observation – Perception and understanding of the “Worldwide” section

  • Browsing observation – Perception and understanding of the UNESCO Courier

  • Browsing observation – Transversal issues

  • Post-test questionnaire – General opinions

  • Post-test questionnaire – Performance and satisfaction rating


G general opinions l.jpg
G. General opinions section

Strong points

  • The main strong points reported by the participants were:

    • A great amount of useful information available

    • Numerous accesses on the home page

    • The standardization process

    • Country-specific information

    • 6 languages


G general opinions77 l.jpg
G. General opinions section

Weak points

  • The main weak points were:

    • Overall complexity, especially for outsiders or newcomers

    • Problems to find certain information and documents

    • Outdated contents

    • Heavy home page


H post test questionnaire performance and satisfaction rating l.jpg
H. Post-test questionnaire – Performance and satisfaction rating

  • Performance indicators: 5 criteria rated from 1 to 5

    • Attractiveness / Website utility

    • Browsing/Locating throughout the site (guidance)

    • Efficiency / Ease of access to information

    • Content structure (Logic in content organization)

    • Understanding / Learnability

  • Criteria assessed through a questionnaire (inspired from the Wammi questionnaire): participants are asked to tell whether they agree or not (from 0 – not at all – to 5 – totally) with 20 statements (4 statements per criterion)

    • E.g.: “This website has much that is of interest to me”

      Fully disagree •••• • Fully agree


G general opinions79 l.jpg
G. General opinions rating

Scores for the performance indicators were pretty good

  • Utility has a very good score (4.2) – it might refer to the rich content (compared to 3.5 for the first study)

  • Browsing and Understanding/Learnability could be better (resp. 3.3 and 3.2) – it might refer to the difficulties in finding certain contents (compared to 2.6 for the first study)

  • Efficiency/Ease of access has a good score (3.8) – maybe due to the expertise of most of the participants (compared to 3.3 for the first study)

  • Content structure has also a good score (3.6) (compared to 2.9 for the first study)



Main recommendations l.jpg
Main recommendations rating

  • The standardization work must be continued to offer a better overall user experience

    • Strong and efficient guidance principles should be applied across the portal

  • The portal targets need to be clearly defined and the website be adapted to them: is it a portal for everyone or a working tool for the insiders?

    • The right column offers too many links, specially for the general public

  • The home page needs more clarity, especially the layout of the central area

  • The work done to offer more multilingual contents should also be continued as it was very appreciated

  • The updates need to be done more regularly – especially for a working tool

  • The “Worldwide” section should offer more contents and be updated more regularly

  • The search engines need improvement


Slide82 l.jpg
Axance « User Experience Design » rating5, rue Régnault93 697 Pantin Cedex FranceTel : 01 48 91 99 93Axance© Juin 2006


ad