Download
1 / 48

The Server Technology Of Eve Online: How To Cope With 300,000 Players In One World Kristj á n Valur Jónsson Senior So - PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on

The Server Technology Of Eve Online: How To Cope With 300,000 Players In One World Kristj á n Valur Jónsson Senior Software Architecht (CCP Games) 

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' The Server Technology Of Eve Online: How To Cope With 300,000 Players In One World Kristj á n Valur Jónsson Senior So' - jerica


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

The Server Technology Of Eve Online: How To Cope With 300,000 Players In One World

Kristján Valur Jónsson

Senior Software Architecht(CCP Games) 

Most MMOGs separate their players onto multiple servers in order to scale; these servers are often called shards. The shards are essentially instances of the game world and the player base is thus broken into segments, based on which shard (server) they are playing on. EVE Online takes a different approach: the technology and the game is developed around the goal of one unified world. All the players log onto the same server cluster and play in the same world. This session will explore the technology developed by CCP to achieve this goal and discuss the technology choices made by CCP, focusing on the software side. Topics include Cluster Technology, Stackless Python, Networking and Database technology.


How to have 360000 players on one server 300,000 Players In One World

The server technology of EVE


About eve
About EVE 300,000 Players In One World

  • One world, one server cluster

  • Launched in May 2003, has been steadily growing ever since

  • Currently at over 360,000 players (~300,000 paying subscribers, trials account for rest)

  • Peak Concurrent Users (PCU) record at 60,453

  • That’s a lot of people!


60,000 people! 300,000 Players In One World


The network effect
The Network Effect 300,000 Players In One World

The value of a service to a potential customer is dependent on the number of customers already using that service

Value of a network equals approximately the square of the number of users in the system

EVE Online is inherently a better game with 300.000subscribers than 50.000 subscribersSize Matters!

Network effect

Metcalfe’s law

Ergo


Eve china
EVE China 300,000 Players In One World

  • Launched in 2006

  • The Serenity cluster is Tranquility’s little brother

  • This talk is about EVE International, running on the Tranquility cluster in London.


More info
More info 300,000 Players In One World

  • http://www.eveonline.com

  • EON Magazine:


I could talk all day
I could talk all day! 300,000 Players In One World

  • Game design

  • Operating systems

  • Multicore

  • Multiprocessing

  • IPC

  • Winsock

  • OLEDB

  • Switches/Routers

  • SQL

  • Introspection

  • Revision control

  • Profiling

  • Racist bike shops

  • Live debugging

  • Compression

  • Session propagation


But we have to be practical
But we have to be practical! 300,000 Players In One World

  • I will focus on a few key aspects

  • These are deep and complex topics. I will skip some of the dangerously complicated details.

  • There will still be technical stuff

  • Demos and videos

  • Additional reference material provided at the end.


The main reasons for eve s scalability
The main reasons for EVE’s scalability 300,000 Players In One World

  • Database system

  • Stackless Python

  • Destiny and space partitioning

  • Hardware configuration


Cluster architecture
Cluster architecture 300,000 Players In One World


Eve s foundation is the database
EVE‘s foundation is the database 300,000 Players In One World

  • Efficient management of data in an un-sharded MMOG is crucial

  • All game-related data is stored in a single database

  • Provides final synchronization of game logic

  • EVE database is currently at 1.3 terabytes

  • Database transactions are usually around 1500 per second, reaching 2000 per second at peak hours

  • ‘Items’ table receives over 9 million insertions in a day (not counting update and select statements)


Eve demands an enterprise level db
EVE demands an 300,000 Players In One Worldenterprise level DB

  • Microsoft SQL 2008 x64 (enterprise edition)

  • IBM X-series 3850 M2 brick server

  • 128 GB RAM

  • 2 x CPU, each six-core at ?? GHz

  • 2x 128 GB RamSan 400 (solid-state drive)

  • 1x 2TB RamSan 500


And enterprise level db engineers
…and enterprise level DB engineers 300,000 Players In One World

  • Tables and procedures are carefully designed

  • Constant monitoring (through DB traces, health checks on DB server) to watch for bottlenecks and optimization opportunities

  • Optimizations sometimes trickle all the way up to game design

  • Recently created robust versioning system for DB content, you will eventually need this.


Application db model
Application DB model 300,000 Players In One World

  • Stored procedures for everything

    • Hand crafted by DB Engineers

    • Lowest possible run-time cost

    • All DB logic in one place

  • Simple application model

    • Integration with Python

    • Looks like regular synchronous function call

    • Return highly optimized Rowset objects


Example code
Example code: 300,000 Players In One World


A more effective way to program
A more effective way to program 300,000 Players In One World


What is python
What is Python 300,000 Players In One World

  • A dynamic programming language

  • Simple intuitive syntax

  • Easily embeddable and extendable

  • Rapid dynamic programming environment with dynamic reloading of code (in-house tech)

  • Open source

    For more info, go to http://python.org


But what is stackless
But what is Stackless? 300,000 Players In One World

  • An alternative version of Python

  • Forked from the Python code

  • Completely backwards compatible

  • Contains additional features:

    • Supports a “tasklet” based application programming model

    • Enables many logical tasks

    • Allows the user to use an imperative, procedural programming model instead of an event driven model to achive parallel IO


Why stackless
Why 300,000 Players In One Worldstackless?

  • Lightweight cooperative “threading” using tasklets

  • Allows you to write robust procedural code, without the boilerplate and machinery of an event-driven model

  • Allows switching of running of both python code and also C code, using stack slicing.

    • Extends to third party libraries, C/C++, etc.


Tasklet essentials
Tasklet 300,000 Players In One World essentials

  • Execution context, not threads!

    • Only one is running at a time

    • An application can have 10000 active tasklets or more.

    • An application can only have relatively few threads

  • Co-operative switching

    • Context switches occur at known point, almost no need for locks!

  • Can even switch C Extensions!

    • Windows overlapped, asynchronous IO looks like synchronous calls.


Example code1
Example code: 300,000 Players In One World


Co operative multitasking demo
Co-operative multitasking (demo) 300,000 Players In One World

import stackless

def func(string):

for i in xrange( 10000 ):

print string

stackless.schedule()

t = stackless.tasklet( func )

s = stackless.tasklet( func )

t(“foo”)

s(“bar”)


Stackless python is cool
Stackless python is cool 300,000 Players In One World

No more this:


Stackless python is cool1
Stackless python is cool 300,000 Players In One World

But this:

www.stackless.com


Destiny
Destiny 300,000 Players In One World


Hierarchical space partitioning
Hierarchical space partitioning 300,000 Players In One World

Binary space partitioning in three dimensions. Each sphere belongs to only one box – the smallest box that is able to contain the spere.

becomes:


But it is more complex than that
But it is more complex than that 300,000 Players In One World

  • The time-extrusion of a sphere determines what space partitions it can interact with (causality bubble)

  • Collision is defined as the intersection of two time-extruded spheres (cylinders with dome shaped end-caps)

  • Collision response is highly elastic (conservation of momentum)


And here the details
And here the details: 300,000 Players In One World

QED


Evolving time
Evolving time 300,000 Players In One World

  • The server solves the system of equations for an entire “system”.

  • The client only solves the equations for the causality bubble that it belongs to.

    Server regularly sends updates to the initial conditions for a time step. The client can work its way backwards to adjust for changes in initial conditions and derive how they affect current state.


This gives defines scaling
This gives defines scaling: 300,000 Players In One World

  • A client scales based on the population in a causality bubble

  • A server scales based on the population in the “system” that we need to solve the differential equations for

  • How are these concepts defined in EVE?

  • Answers after this break!


Movie
Movie 300,000 Players In One World


The eve universe
The EVE Universe 300,000 Players In One World

Solar systems typically consists of a dozen of planets, surrounded by moons and asteroid belts

Routes connect them into constellations that themselves form conglomerates called regions

EVE consists of over 5000 solar systems


Creating solar systems
Creating solar systems 300,000 Players In One World

EVE solar systems are created using the disc accretion model

Gives convincing enough results and is possible to calculate them all in less than 24 hours


Large scale design
Large scale design 300,000 Players In One World

  • The large scale shape of the eve universe is defined first and foremost by Routes.

  • Routes define how pilots can travel between systems.

  • Position of the planets is derived from this.


Routes make it interesting
Routes make it interesting! 300,000 Players In One World

Designing the layout, we wanted interesting properties:

  • Settlements

  • Stratetic value

  • Hierarcical properties


Diffusion limited aggregation
Diffusion Limited Aggregation 300,000 Players In One World

  • Successfully used to model various growth processes

  • Extended to work in 3 dimensions and have multiple seeds.


Implications for scalability
Implications for scalability 300,000 Players In One World

  • The smallest unit of scaling is a solar system

  • Some system are busier than others

    • Pass-through systems

    • Strategic access to trade

  • Occasional tuning of routes necessary


Scaling with increased hardware
Scaling with increased hardware 300,000 Players In One World

  • A unique combination of software and hardware layers

  • Tiered architecture allows software to scale well with increased hardware

  • Make maximum use of worker threads and IO Completion ports underneath micro-threaded Stackless Python logic


Adding more hardware
Adding more hardware 300,000 Players In One World

  • Architecture can be split up and distributed to more hardware:

    • Solar systems

    • Space stations

    • Market regions

    • Corporation services

    • Proxy services


Eve runs on a supercomputer
EVE runs on a supercomputer! 300,000 Players In One World


Summary
Summary 300,000 Players In One World

  • Structure your application in ways that promote scalability

    • Make it as granular as possible, so that load balancing is more efficient, and you can add more hardware.

  • Chose a powerful, expressive programming language

    • Allows you to restructure key areas as needs arise with the least possible engineering effort.

  • Design your database system wisely

    • Be one step ahead, so that it doesn’t become a bottleneck for scaling

    • You are using a DB, aren’t you?


Burt reynolds says
Burt Reynolds says: 300,000 Players In One World

Always use a Database!


That s it
That’s it! 300,000 Players In One World

  • Additional information:

    • http://www.eve-online.com

    • http://stackless.com

    • http://www.slideshare.net/Arbow/stackless-python-in-eve

    • http://us.pycon.org/2009/conference/schedule/event/91/

    • (google for “stackless python in eve”)


ad