Experience of automated valuation modelling avm in england
Download
1 / 48

Experience of Automated Valuation Modelling AVM in England - PowerPoint PPT Presentation


  • 172 Views
  • Uploaded on

Experience of Automated Valuation Modelling (AVM) in England. Tim Eden BSc MRICS IRRV Deputy Director of Council Tax Valuation Office Agency. Presentation overview. A “fly through” of the VOA AVM experience including: Background The challenge The extent of data capture

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Experience of Automated Valuation Modelling AVM in England' - katina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Experience of automated valuation modelling avm in england l.jpg

Experience of Automated Valuation Modelling (AVM)in England

Tim Eden BSc MRICS IRRV

Deputy Director of Council Tax

Valuation Office Agency


Presentation overview l.jpg
Presentation overview

  • A “fly through” of the VOA AVM experience including:

    • Background

    • The challenge

    • The extent of data capture

    • Changes in data management

    • Process improvement and change

    • Model development

    • Training

    • Achievements


To be a world class organisation providing valuation and property services for the public sector l.jpg

To be a world class organisation providing valuation and property services for the public sector


Voa our purpose l.jpg
VOA - Our purpose property services for the public sector

  • To provide a fair and robust basis for taxes, which help to pay for public services; and to help drive better use of property in the public sector by:

    • compiling and maintaining accurate and comprehensive valuation lists for local taxation

    • providing accurate valuations for national taxes

    • delivering expert advice on valuations and strategic property management

    • developing and maintaining a comprehensiveand up to date property database

    • advising policy makers on valuation andproperty issues


Voa our structure l.jpg
VOA - Our structure property services for the public sector


Recent taxation history l.jpg
“Recent” Taxation History property services for the public sector


Council tax ct context l.jpg
Council Tax (CT) - Context property services for the public sector

  • CT + Non-Domestic Rates together raise c. £40bn pa

  • CT raises £18.5bn pa

  • 22m domestic properties in England

  • 1.3m domestic properties in Wales

  • Clients

    • Department for Communities & Local Gov’t (DCLG) in England

    • Welsh Assembly Gov’t (WAG) in Wales


Ct 1993 list bands england l.jpg
CT – 1993 List Bands (England) property services for the public sector

  • A - up to £40,000

  • B - £40,001 to £52,000

  • C - £52,001 to £68,000

  • D - £68,001 to £88,000

  • E - £88,001 to £120,000

  • F - £120,001 to £160,000

  • G - £160,001 to £320,000

  • H – over £320,001


Ct revaluation l.jpg
CT - Revaluation property services for the public sector

  • Local Government Act 2003 included requirement to undertake revaluations in Wales (2005) and England (2007)

  • In Wales AVD 1 April 2005; list came into force on 1 April 2005

  • In England AVD 1 April 2005; list was to come into force on 1 April 2007

  • Draft lists were to be published 1 September 2006


The challenge l.jpg
The Challenge property services for the public sector

  • New banding scheme not known

  • Necessary to produce individual valuations & then overlay new bands

  • Scale of task

  • Complexity of housing types

  • Vagaries of the market

  • Inconsistent, paper-based records


The challenge market vagaries l.jpg
The Challenge – Market Vagaries property services for the public sector

  • Market analysis across England

    • Free flow of money

    • Historically low interest rates

    • Fluctuating volumes in sales


The challenge housing types l.jpg
The Challenge – Housing Types property services for the public sector

  • Profile of Housing Stock

    • 80% Houses/Bungalows

    • 20% Flats/Maisonettes

    • Over 1/3rd of all flats in Greater London

    • Around 1/6th of properties at least 100 years old

    • BUT high proportion of new properties in sales evidence!


The solution l.jpg
The Solution! property services for the public sector

  • Develop an AVM

  • Skill-up staff

  • Learn from others

    • Cole Layer Trumble (CLT) – computer modelling expertise, AVM software

    • KPMG – programme & project management support

    • IAAO – statistical training

  • Digitise data


Voa avm journey l.jpg

  • business case

  • political issues

  • staff planning and allocation

  • AVM design

  • Improving Business Decisions

  • value reviews

  • data collection

  • 22 million values

  • data cleansing

  • MRA models

VOA - AVM Journey

  • Began in 2002; worldwide research

  • Mass data capture/enhancement

  • Model development

  • Many innovations and lessons

  • These aspects have been very important to the success of the project


Attribute data investigation l.jpg
Attribute Data Investigation property services for the public sector

  • Investigation commences in 2002 considering:

    • Use of existing “attribute” codes already available on VOA IT database (“Group”, “Type”, “Area”)

    • Valuer and caseworker engagement to determine useful codes

    • Data availability & maintainability

      • CT List Maintenance work

      • External standards e.g. IAAO (6 year cycle)

      • Impact of differing local practices

    • Ability to undertake sales investigation activity

  • Pre-contractor appointment; so limited AVM expertise


Main attributes to be digitised l.jpg
Main Attributes to be digitised property services for the public sector

  • Architectural Style and construction quality – “Group”

  • Property Type e.g. House Semi-Detached – “Type”

  • Age (coded as era e.g. G – 1965-72)

  • Area m² - external area (houses); internal area (flats)

  • No. of rooms

  • No. of bedrooms

  • No of bathrooms

  • Number of floors (houses); floor level (flats)

  • Parking

  • Conservatory (type and size)

  • Outbuilding details

  • Value Significant Codes


What about sales data l.jpg
What about Sales Data? property services for the public sector

  • Access to Land Registry and Stamp Duty Land Tax information

  • Address irregularities – creates matching problems and manual effort

  • Need to establish whether transaction at “Market Value”, special circumstances etc.

  • Data understanding significantly assists

  • AVM geared to support this process

  • “Value” understanding working with AVM is key to the process


Process development l.jpg
Process Development property services for the public sector

  • Consider existing process

  • The impact of AVM – scale and profile

  • Skills

    • Existing skills and abilities of targeted staff

    • The need to re-train certain staff

  • Resource availability

  • New process to reflect Revaluation needs and business as usual

  • IT constraints and capabilities

  • Understand relationship between sales and model performance


The analysis process l.jpg

After property services for the public sector

Locality Definition

(area from which comparables derived)

AVM analysis output reviewed by qualified staff

AVM

Sales Data

1)New Sales continually updating analysis set

2) Value significance considered

Models

(MRA, algorithms and variables)

Property Data

(Group, type, floor area, age, bedrooms, rooms, garage)

Decision to change, retain or include current data and coding

The Analysis Process

Before

Sales received

Suspect sales coded

Sales searched to support each valuation case “on paper”

At a Revaluation “Beacon Sales” reviewed and recorded “on paper”


Initial issues with modelling process l.jpg
Initial Issues with Modelling Process property services for the public sector

  • No first hand knowledge of sale

  • Data time lag

  • Data gaps due to

    • “Permissible development” – the public need not inform the Billing Authority (BA)

    • BAs internally not acting in a joined-up way – billing, planning & building control

    • BAs not acting in joined-up way with VOA

    • Are we maintaining lists or data or both?

  • “Condition” not an attribute


Integrating the avm l.jpg
Integrating the AVM property services for the public sector

  • Investigation work commenced late 2002

  • Procurement process 2003

  • Collaboration of in-house team and successful contractors/partners - Cole Layer Trumble (CLT)

  • Working on BA areas where data capture is advanced (note had only started in 2003!)

  • Challenge of integrating AVM technology with existing VOA IT systems


Getting started l.jpg
Getting Started property services for the public sector

  • Started with a simple additive model

    • £ = B0 + B1 * size + B2 * Detached * Size + B3 * Terraced * Size + … + Bn * Date of Sale * Size

  • 1993 Band initially used to support analysis, but quickly stripped out of valuation models

  • Postcode sectors used as proxy for location – VOA developed “localities”

    • Postal areas are too crude

    • Designed to support mail delivery, not to reflect influences on the property market!


Performance improvement 1 l.jpg
Performance Improvement (1) property services for the public sector

  • Better understanding of subject/sale data helped to:

    • Determine usability rules for raw data

    • Determine usability rules for a sale

  • This understanding fed

    • Comparable selection

    • Model specification

    • Consideration of rarely occurring variables

    • Consideration of locality relationships


High correlations l.jpg
High Correlations property services for the public sector

Floors/

Floor Level

Type

32 separate codes

Parking

Group

55 separate codes

Age

11 separate codes

Correlation among variables

High correlation compromises modelling stability

Area

Bathrooms

Rooms

Bedrooms


Localities illustrative photograph l.jpg

Privately built housing estate property services for the public sector

Local Authority housing estate

Localities – illustrative photograph

  • Created bespoke localities (neighbourhoods)


Mapping localities l.jpg
Mapping & Localities property services for the public sector

  • In excess of 10,000 localities

  • Regular boundary review required

  • Thematic mapping as part of process

  • X-Y co-ordinate becomes necessary data

    • X-Y suitable for comparable selection

    • Issue of who maintains the data


Mapping example l.jpg
Mapping Example property services for the public sector


Performance improvement 2 l.jpg
Performance Improvement (2) property services for the public sector

  • Move to multiplicative (log linear) model structure

    Log (£) = B0 + B1 * log (size) + B2 * Detached + B3 * Terraced + …

    + Bn * Date of Sale + Bp * Log (Locality Adjustment Factor)

    where Bi determined from sales set using MRA

  • This enabled improvements in: -

    • Locality Adjustment Factor (time adjusted median price per square metre)

    • Locality Grouping – support for comparable selection

    • Central Modelling across the whole country


Benefits of central modelling l.jpg
Benefits of Central Modelling property services for the public sector

  • Central Modelling enabled:

    • Central recognition of national modelling patterns and data issues

    • Modelling “constraints” could be imposed by the centre ensuring model coefficients are consistently applied

    • Effective direction of effort

    • Calculation of market trend information to support VOA modelling and wider government market appreciation


Performance improvement 3 l.jpg
Performance Improvement (3) property services for the public sector

  • Integrating X-Y & Mapping allowed mass calculation of “plot size”

  • Issues with map plots are:

    • Plot bleed

    • Non-alignment with the property transferred

    • Maintenance of data


Model development lessons l.jpg
Model Development - Lessons property services for the public sector

  • Create a multi-skilled and focused R&D team

  • Select several representative areas to test

  • Ensure proper debate on proposals

  • Predict overall likely modelling gains

  • Sense check & gather feedback from local staff

  • Business model to relate model performance and cost/benefit

  • Produce individual cost/benefit for each proposal

  • Promote external verification e.g. IAAO


Supporting decision making 1 l.jpg
Supporting Decision Making (1) property services for the public sector

  • VOA recognised management of a national valuation delivery required consistency in valuation decisions

  • Property level confidence score required

    • COD/COV inadequate existing banding too remote

  • Score needed to support decisions at several levels:

    • Strategic:

      • Business case development

      • Process definition

      • Data collection & enhancement

    • Tactical:

      • Resource planning

    • Operational:

      • Resource allocation

      • Value review or band review


Supporting decision making 2 l.jpg
Supporting Decision Making (2) property services for the public sector

  • Confidence estimate for each subject property needed to reflect the aspects which would reduce accuracy:

    • Data quality

    • Data availability

    • Market variability

    • Model accuracy

      • Adequate coverage for accuracy in the MRA

      • Adequate comparables


Confidence model l.jpg
Confidence Model property services for the public sector

  • Confidence model based upon indicators from MRA and comparables

  • Related actual errors to the dispersion amongst the comparables

  • Comparable sales approach provides a number of other indicators for confidences:

    • comparability distance

    • weighted estimate (average adjusted) vs. MRA estimate

    • the overall COV for the model on the sales set which tells us something about the underlying uniformity in the market and sales base

  • MRA modelling includes a control model which considers current CT band

  • So Confidence Model is:

    Likely Error = A

    + B  dispersion of comparables

    + C x average distance between comparables

    + D x absolute value of (ln (mkt est. / control model est.) )

    + E x model standard error

    + F x absolute value of (ln (weighted est. / MRA est.) )


Confidence model maintenance l.jpg
Confidence Model Maintenance property services for the public sector

  • Re-calibrated with every calibration iteration

  • Periodically calibrated to valuer judgement of estimate output

  • Can be used individually or aggregated for decisions at varying levels

  • Developed between VOA, John Thompson (Cole Layer Trumble), Dr Jim Abbott (EDS)

  • Presented at IAAO CAMA/GIS conference February 2006


Summary of models 1 l.jpg
Summary of Models (1) property services for the public sector

  • Multiplicative MRA and comparable selection

    • Operational delivery

    • Undertaken locally

    • Specified centrally

  • Broad Based Model

    • Market Trends

    • Linearisations e.g. Group and Type variables to allow wider comparability

    • Develop constraints for local models

    • Quality assurance on operational delivery


Summary of models 2 l.jpg
Summary of Models (2) property services for the public sector

  • Control Model

    • Operates in background of local model

    • Highlights data irregularities

  • Confidence Model

    • Uses data from all models

    • Calibrated to valuer opinion

    • Provides information for use at all levels


Managing business change l.jpg
Managing Business Change property services for the public sector

  • Addressing the time lag

    • Review whole data process

    • Use electronic transfer to bring in sales data

    • Consider how to engage with taxpayer

  • New skills

    • Control modelling and train professionals

    • Develop support and maintenance staff

  • Rigorous application of project management methodology


It development l.jpg
IT Development property services for the public sector

  • VOA has successfully integrated AVM technology with existing IT; this required:

    • Improving data flow and management

    • Enhanced mapping tools

    • “Customising” 3rd party “off-the-shelf” product

    • Introducing workflow to manage delivery

  • Lead-in time has created tensions

  • IT supplier’s understanding of business and new technology does not happen overnight!


Conclusions lessons learned 1 l.jpg
Conclusions/Lessons Learned (1) property services for the public sector

  • Other data sources create issues:

    • Often aggregated

    • Currency – frequency of update

    • Niche provider with limited depth

    • Definition – what creates the data

    • Cost – Is it worth it to you?

    • Your records may be adequate and better than elsewhere anyway!


Conclusions lessons learned 2 l.jpg
Conclusions/Lessons Learned (2) property services for the public sector

  • How do you measure data quality

    • Sale data quality

      • Mostly measured by model

      • Dependent upon proper sales review process

    • Subject data quality

      • Adoption of external standards not always possible e.g. IAAO 6 year cycle

      • Data sampling using existing maintenance activities can support the view


Conclusions lessons learned 3 l.jpg
Conclusions/Lessons Learned (3) property services for the public sector

  • Time lag

    • Understand the delivery requirement

    • Address steps in the data process

    • Work closer with data “partners”

    • Central modelling can identify areas of concern

  • Proximity to data

    • You cannot easily replace local knowledge of market and meaning of sales.


Conclusions lessons learned 4 l.jpg
Conclusions/Lessons Learned (4) property services for the public sector

  • Modelling

    • Define “attributes” with knowledge of AVM techniques

    • Balance modellers desire for more data of imperfect market and cost to complete/maintain

    • Stabilise data prior to commencement of modelling

    • Valuing to “Band” does not loosen the rigour of modelling and data management


Conclusions lessons learned 5 l.jpg
Conclusions/Lessons Learned (5) property services for the public sector

  • Use AVM to direct effort

    • Raw data analysis

    • COD & COV are not the only measures

    • “Frequently used comparables”

    • Thematic mapping

  • Use formal modelling hierarchy

  • Don’t rush to deliver. Inefficiencies result

    • Stabilise Model approach to ensure consistency

    • Data stability and consistency


Conclusions lessons learned 6 l.jpg
Conclusions/Lessons Learned (6) property services for the public sector

  • Create clear lines of communication

    • Local Management

    • Local technical/modelling

  • Training

    • Make timely

    • Consider delivery mechanism (e-learning, workshop)

  • Project Management Structure

  • A quality model can be let down by a poorly defined and delivered process!


Some voa achievements l.jpg
Some VOA achievements property services for the public sector

  • Digitisation of over 22 Million records

  • Data completeness approaching 100%

  • Over 120 surveyors trained in AVM techniques

  • Market analysis nationwide, including mapping localities across the whole country

  • Model performance well within recognised standards e.g. median COD of 353 BAs is 9.97

  • Proven IT platform for mass appraisal at national level.


Slide47 l.jpg

Questions? property services for the public sector

Tim Eden BSc MRICS IRRV

Deputy Director of Council Tax

[email protected]


Slide48 l.jpg

Achieving property services for the public sectorWorld Class


ad