1 / 34

OpenAFS Status

OpenAFS Status. new year, new challenges Derrick Brashear The OpenAFS Project 3 June 2009. First things first. If we’re going to have a live debug session, we need to know what to debug. Because after all, OpenAFS is bug-free, right?. This all sounds familiar.

zola
Download Presentation

OpenAFS Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OpenAFS Status • new year, new challenges • Derrick Brashear • The OpenAFS Project • 3 June 2009

  2. First things first • If we’re going to have a live debug session, we need to know what to debug. • Because after all, OpenAFS is bug-free, right?

  3. This all sounds familiar • New stable release coming. (1.4.11) • New development releases continue apace. (1.5.60, 1.5.61 soon)

  4. Recently resolved issues • A race in the namei fileserver’s background sync support, ih_sync_all. • Could cause volumes to go offline during a release until salvage. • Linux post-2.6.29 keyring-to-group PAG copying Oops issue. • Fileserver host tracking/hashing (again). • Most Linux mmap-close-write corruption cases fixed.(*) • Linux byte range (client-only) locking now works!

  5. In the past year • OpenAFS 1.4.8 shipped with a large number of fixes to Rx packet tracking. • Double-frees • Races

  6. New in 1.4.10 • Security fixes (sigh) • Rx in-use packet freeing fix. • Fix for Rx idleness tracking. • Corrections for ubik database recovery process.

  7. New in 1.4.10 • Multiple (more than 2) local realms. • Requires username space to be the same. (“shadow” in any realm is the same person) • List realms one per line in /usr/afs/etc/krb.conf or equivalent.

  8. 1.4.10 open issues • Panic in Linux VFS layer (infrequent). • Page mapping with close followed by write on Linux can lose data where file >> cache. • Fileserver host list can be corrupted.

  9. New in 1.5.60 • A fix for the ih_sync_all bug. • Hartmut pointed us the right way here. • Retry in the event of volserver transaction creation. • Based on Rainer Toebbicke’s patch. • Dragonfly BSD userland support. • Contributed by Matt Benjamin. • Cache type abstraction. • Thanks to Marc Dionne and Simon Wilkinson

  10. 1.5.60 open issues • Demand Attach Fileserver bugs (more on that later) • Otherwise, same issues as 1.4.10.

  11. Unix platform round-up • AIX 5 and 6 • FreeBSD 7 and current • HP-UX 11.0, 11i v1 and v2 • Irix 6.5 • Linux 2.2, 2.4, 2.6 (ia32, ia64, x86_64, ppc, ppc64, arm, sparc, sparc64) • MacOS 10.3, 10.4, 10.5. • OpenBSD 4.4, 4.5. • Solaris 2.6, 7, 8, 9, 10, 11 (and OpenSolaris)

  12. MacOS X • Most of the issues with 10.4 were resolved for 10.5 with help from Apple. • However, getting tokens at login is (now) “hard”. • AFSCommander tool available, integration coming. • Sadly, no kexts on the iPhone (It IS just OSX).

  13. Linux • iget() is dead. • Cache manager opens files by path, as in OSX, to deal. • Cache type abstraction. • Changes to avoid deadlocks due to VM system page writeback. • Dynamic sizing for AFS client vnode pool.

  14. AIX • A LAM plugin for Kerberos 5 based aklog is available and works with CDE Screenlock. • The client properly supports AFSDB. • AIX 6 support was contributed by Niklas Edmundsson. • AIX 5.1 will properly support PAGs again in 1.4.11, 1.5.61

  15. BSD • Previous work on FreeBSD had always stumbled on locking issues. • Matt Benjamin revisited, fixed, and updated the previous work. He found the locking issues still lurked. • Further work on OpenBSD and NetBSD has also been done but clients are not as ready.

  16. Clients • Actually not much exciting on clients. • Numerous interaction issues with GUI environments have been addressed. • Notable exception: Finder (more later) • Far fewer resources are leaked during client operation. We’re not perfect yet. • No more gratuitous token disappearance.

  17. Fileservers • The salvager won’t corrupt directories anymore on 64 bit hosts thanks to good sleuthing by Rainer Toebbicke. • Your larger-than-2tb partitions are now good to go. • Old “vos” clients may report odd numbers for empty partitions.

  18. Playing nicely with other children • Quotas enforced on TellMeAboutYourself /WhoAreYou calls to clients will preclude resource hogging. • No more assert()s when a volume is found in an unexpected state. • The server will never keep clients waiting forever for an answer (nor is the client that patient anymore) starting with 1.4.8.

  19. And who were you, again? • Client tracking turns out to be hard when clients lie (unbeknownst to themselves). • Just because an address is reused does not mean it’s the same client. • The fileserver will work even harder to take client address information with a lump of salt.

  20. Things to test in 1.5 • Cache bypass (Linux-only, new since 1.5.53) • Support for additional platforms is also needed. • Split cache (dedicated portion for read-write data). • Early versions of this had corruption issues likely related to writedcache-enforce-xdcache-writelock-20071208 ; thanks to Stephan Weisand for working with me on that (for quite a while).

  21. Features galore • Linux NFS translator still not globally suportable. • GPL-only changes recently break it. • Mountpointless volume addressing (/afs/.:mount/cell:volumeid/) is available. • Originally done for the Linux NFS translator. • An extension allows any vnode to be used. (/afs/.:mountcell:volumeid:vnodeid:uniquifier/)

  22. But now how much would you pay? • Read-write disconnected AFS support. • Cache read-ahead. (Configurable window size, defaults to off) • Tunable with “fs precache”. • And...

  23. Demand Attach Fileserver • Known issues: • Volume headers for non-existent data on disk can remain in memory in a way that does not allow them to be purged. • Several outstanding bugs in RT. • Please share other issues if you have them!

  24. A common theme • We’re (always) looking for volunteers. • Sometimes we are better about asking than others. • Today I will reiterate: please help us test 1.5 (especially DAFS and Disconnected AFS)

  25. Pending integration • Rx K5 • Being revised to match current build trees. • Rx OSD. • Being committed in manageable chunks. • Testing now possible!

  26. More pending work • Extended callback messages to optimize away unneeded traffic. • Both change “ranges” when data is stored, and metadata bundling when other things cause the callback. • Also possible to get finer-than-whole-volume notifications on releases.

  27. Near term undertakings • Locking enhancements for Unix clients. • A fix for the MacOS “Finder cross-volume drag” issue. • A userspace helper and the ad-hoc “reference any vnode” semantics make this simple to solve.

  28. And on the horizon • Multiple volume snapshots. • RxTCP. • Directory object changes (Unicode, typed streams, more files, better hashing). • Full Kerberos 5 support.

  29. On version control • git is coming. • It should be much easier to track upstream as we work with you on integrating your changes. • And it should be easier for us to merge them, too. • More about this will be discussed.

  30. Fresh blood • Google Summer of Code accepted us again. • Summaries available on the OpenAFS website. • More in a moment. • The UIUC Capstone project still working with us. • Their target project is an improved Windows Server Manager application.

  31. Google Summer of Code • Windows MMC management snapin, Brant Gurganus. • Improved OpenAFS server selection, Jake Thebault-Spieker. • OpenAFS features in the Linux kAFS client, Wang Lei.

  32. Google Summer of Code • Last year’s projects offered varying success. • Write support for disconnected AFS. • kAFS pioctl support. • Read-write replication. • Per-file ACL support.

  33. In other news • More during the roadmap discussion. • Oh yeah, got anything for us to debug?

  34. OpenAFS StatusIf your cell phone rang, you owe me a beer.Oh wait, the beer’s free.

More Related