Accounting and Allocation Management with Gold
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Accounting and Allocation Management with Gold Scott Jackson Moab·Con 2010 PowerPoint PPT Presentation


  • 83 Views
  • Uploaded on
  • Presentation posted in: General

Accounting and Allocation Management with Gold Scott Jackson Moab·Con 2010. Gold is an open source allocation system that tracks and charges for resource usage on High Performance Computers. Allocation Manager. Ensures resources are used according to mission plan Tracks resource utilization

Download Presentation

Accounting and Allocation Management with Gold Scott Jackson Moab·Con 2010

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Accounting and allocation management with gold scott jackson moab con 2010

Accounting and Allocation Management with Gold

Scott Jackson

Moab·Con 2010


Allocation manager

Gold is an open source allocation system that tracks and charges for resource usage on High Performance Computers.

Allocation Manager

  • Ensures resources are used according to mission plan

  • Tracks resource utilization

  • Allows for insightful capacity planning

  • Facilitates resource sharing between organizations

  • Used in production at dozens of sites


Gold in a nutshell

Gold in a nutshell:

  • A resource billing system

    Gold tracks and manages resource usage on High Performance Computers. It behaves like a bank in which deposits are made into accounts and charges occur at the moment of job completion. Reservations prevent oversubscription.

  • An allocation manager

    Credits can be allocated to specific projects, users and machines. Timeframes can be given for the expenditure of the credits enabling allocation cycles, capacity planning and use-it-or-lose it.

  • An accounting system

    Gold maintains past state and a complete transaction history. It can track and maintain arbitrary custom accounting statistics.

  • An information service

    Custom objects and attributes can be dynamically defined and manipulated.


Accounting and allocation management with gold scott jackson moab con 2010

Setting up Gold

  • Define Users, Projects, Machines

  • Add Users to Projects

  • Setup Default Projects (optional)

  • Make Deposits into Accounts


Accounts projects users machines allocations

Universe

of

Projects

Universe

of

Users

AccountsProjectsUsersMachinesAllocations

Account: 1234

Name:Chemistry

Account: 1234

Name:Chemistry

Universe

of

Machines


Allocations

Allocations

  • An allocation is an allotment of computational resource credits for use by specific projects or users during a specified time period.

  • When jobs complete, projects are charged and resource usage recorded. Jobs are prevented from starting unless they have an active allocation with a sufficient balance.

  • Limitations: Does manage short-term usage, does not impact scheduling priority.


Reservations and dynamic charging

0

6

3

2

4

1

5

Charge

Reservations and Dynamic Charging

Accounting Manager (Gold)

0 Deposits are made in Account

1 A Job is Submitted

2 A Quote is Requested

3 A Reservation is Made

4 The Job is allowed to Start

Scheduler

(Moab)

5 The Job Completes

6 The Reservation is Removed

and a Charge is Issued

Resource Manager

(Torque, LL, LSF)

Deposit

Reservation

Quote

Reservation


Capacity workload planning the brick approach

Capacity & Workload Planning:The Brick Approach

100 % Capacity

  • Controlling project usage is key to capacity and workload planning.

  • The use of regularly expiring allocations allows you to establish a project cycle and use the brick approach to allocation management.

Now

- 3 Qtr

- 2 Qtr

- 1 Qtr

+ 1 Qtr

+ 2 Qtr

+ 3 Qtr

+ 4 Qtr


Year end resource exhaustion

Year-end Resource Exhaustion

  • It is common to see projects get off to a slow start and then have a surge of activity at the end when results are due.

  • Without careful management of expectations, this can be a cause of considerable anxiety.

100% Machine Capacity

Project A’s Allotment

Project A’s Demand

Project Period


Expiring allocations

Expiring Allocations


Chargerates

ChargeRates

  • Consumable Resource Charge Rates — Consumable Resource Charge Rates define how much it costs per unit of time to use a consumable resource like processors, memory, etc.

  • Usage Charge Rates — Usage Charge Rates define how much to charge for metrics of total resource usage such as CpuTime, Licenses used, Power consumed (in Watts), etc.

  • Name-Based Multiplier Rates — Name-Based Multiplier Rates are quality based charge rates which apply a multiplicative charge factor based on a quality of the job such as QOS, nodetype, queue, user, time of day, etc.

  • Value-Based Multiplier Rates — Value-Based Multiplier are scaled multipliers which apply a multiplicative charge factor based on a numeric scaling factor.


Other gold features

Other Gold Features

Guaranteed Quotes

Credit and Debit Models

CPU

Memory

Disk

QOS

Powerful Querying

Flexible Charging


Other gold features1

Other Gold Features

Transparency Features

Historical Journaling

Customizable

Accounting

Dynamically Customizable

Strong Security


Web based gui

Web-based GUI


Nested accounts

Nested Accounts

University

25

25

25

Biology

Chemistry

Physics

40

50

Chem 101

Chem 201

40

40

Actinides

Workshop


End of life spending spree

End of Life Spending Spree

  • Offline banks cannot strictly enforce allocations

  • Without reservations, more jobs can get started than allocation balance can support.

Debit

Accounts

Debit

Accounts

Job

Job


Sneak peek chargerates

Sneak Peek -- ChargeRates

Type Name Instance Rate Description

---------- ---------- --------- ---- ------------------------------

Resource Processors 10 Value Based Resource (VBR)

Service License Abacus 10 Name Based Resource (NBR)

Usage Power 500 Value Based Usage (VBU)

Feature Feature Telescope 5 Name Based Usage (NBU)

Quality QOS Premium 2 Name Based Multiplier (NBM)

Multiplier Discount 0.5 Value Based Multiplier (VBM)

Fee Shipping 100 external Value Based Usage (xVBU)

Token Zone Asia 400 external Name Based Usage (xNBU)

*(Disk) User scottmo 10 Multi-dimensional VBR (MVBR)

The multi-dimensional Resource is a resource value that changes on a per-attribute basis, and is assumed if the Type is anything other than the known types. For example, the above line illustrates a disk resource fee of 10 for User=scottmo, whereas a different value might be applied for User=fred.

Total Charge = ((((sum of all NBR and (M)VBR*value) times walltime) + (sum of all NBU and VBU*value)) times (product of all NBM and VBM*value)) + (sum of all xNBU and xVBU*value)


  • Login