Cinder On-boarding Education for OpenStack Developers

2. Cinder On-Boarding EducationJay Bryant (Lenovo)April 30, 2019 - Denver

3. Agenda Cinder’s Repos Brief overview of Cinder architecture/organization Cinder’s team Pointers to Upstream Institute education Applying OpenStack Process Cinder’s code Testing changes to Cinder

4. Cinder Repos • Main Cinder repository: • https://github.com/openstack/cinder • Cinderclient: • https://github.com/openstack/python-cinderclient • os-brick shared library: • https://github.com/openstack/os-brick • Brick Cinderclient for Standalone Cinder • https://github.com/openstack/python-brick-cinderclient-ext • cinderlib • https://github.com/openstack/cinderlib • Cinder Specs: • https://github.com/openstack/cinder-specs

5. Cinder Overview • Provides persistent block storage resources for an OpenStack cloud. • Supports multiple back-ends. Approximately 60volume drivers in tree.

6. Vendor Backends • Drivers from many different vendors • Different protocols used depending on the driver • iSCSI • Fibre Channel • RBD • RemoteFS • Etc.

7. Backend Drivers • Note: Supported drivers list. Full list may be seen here: https://docs.openstack.org/cinder/latest/drivers.html

8. Volume Attachment Cinder Nova Note that iSCSI is just an example – several additional protocols are supported (e.g., FC, NFS) LVM VM instance /dev/vda iSCSI target Legend Persistent volume control Persistent volume data KVM iSCSI initiator

9. Cinder’s Team • PTL • Jay Bryant (jungleboyj) • Core Team • Sean McGinnis (smcginnis) • Walt Boring (hemna) • TommyLike Hu (tommylikehu) • Gorka Eguileor (geguileo) • John Griffith (jgriffith) • Eric Harney (eharney) • Ivan Kolodyazhny (e0ne) • Xing Yang (Xyang) • Brian Rosmaita (rosmaita) • Yikun Jiang (yikunkero) • Rajat Dhasmana (whoami-rajat)

10. Upstream Institute • https://docs.openstack.org/upstream-training/ • Day and a half education session to help people new to OpenStack • Lead by OpenStack Foundation members and volunteers • Miss the Denver session … No problem, join us in Shanghai!

11. Upstream Institute - Key Info • VM image to get your development started • OpenStack account setup • https://docs.openstack.org/upstream-training/workflow-reg-and-accounts.html • Launchpad/Storyboard introduction • https://docs.openstack.org/upstream-training/workflow-task-tracking.html • Workflow introduction • https://docs.openstack.org/upstream-training/workflow-training-contribution-process.html • Introduction to gerrit and code reviews • https://docs.openstack.org/upstream-training/workflow-gerrit.html • https://docs.openstack.org/upstream-training/workflow-reviewing.html

12. Cinder in OpenStack’s Processes

13. Cinder Bugs and Blueprints • Still currently using Launchpad • https://launchpad.net/cinder

14. Bug Tags • low-hanging-fruit • doc • i18n • security • ops • <SERIES>-rc-potential • <SERIES>-backport-potential • driver/drivers • gate-failure • ceph • cinder • backup-service • create-volume-from-snapshot

15. Cinder Specs • https://github.com/openstack/cinder-specs • We request specs for more complicated changes • Organized by release (specs/<release>) • Template from which to start: specs/template.rst • Before pushing your spec run ‘tox -e docs,py27,pep8’ • Look for any errors and please correct before pushing for review

16. IRC and Cinder Meetings • #openstack-cinder • Please join us for all things Cinder • Try @? for some (╯°□°)╯︵ ┻━┻ fun • (not working on Cinder but in openstack-meeting  • Weekly Meeting • Wednesdays at 16:00 UTC • #openstack-meeting • https://wiki.openstack.org/wiki/CinderMeetings • Add agenda items with your IRC nickname to the meeting agenda/notes etherpad

17. Typical Development Cycle • Stein release schedule • https://releases.openstack.org/stein/schedule.html • Milestone 2 • Spec freeze • New driver merge deadline • Milestone 3 • New feature freeze • Move focus to testing and bug fixes • Need 3rd party CI’s functional to avoid addition of unsupported flag • RC • Bug fixes only • Some flexibility around dates allowed with core team and PTL’s discretion

18. 3rd Party CI • 3rd Party CI is required for all Cinder drivers • https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers • Purpose • To ensure that a vendor’s driver functions properly as changes merge to the Cinder tree • To show that patches against a vendor’s driver work • Policy (Proposed) • 3rd Party CI must run successfully against a weekly job executed to test that CI’s are reporting • 3rd Party CI must fail a job submitted that should fail all 3rd Party CI testing. • A vendor CI that doesn’t meet these requirements during a release cycle is marked ‘Unsupported’ and consumers have to update their configs to indicate they wish to run an unsupported driver • Continued non-compliance results in driver removal in the next release.

19. 3rd Party CI - Review Implications • If a patch fails all CI’s, something is wrong • Patches against a driver must pass the vendor’s CI before being merged

20. About Reviews • We need your help! • Good way to get involved with Cinder • Learn the code and the project organization • Don’t need to know how Cinder works to catch logic errors and typos • Gets you visibility to the Cinder team • Review inboxes: • Cinder review inbox • os-brick review inbox • python-brick-cinderclient-ex review inbox • Track how we are doing: • http://cinderstats.ivehearditbothways.com/

21. Cinder Code Layout - Top Level • api-ref -- Source documentation for v1,2 and 3 Cinder API • cinder -- Cinder’s source directory with high level implementations at the top level (i.e. exceptions, i18n, policy, utils, etc.) • doc -- Developer documentation that is published to the OpenStack website • etc -- Config files that are not automatically generated • rally-jobs -- Rally tasks and plugins to be run by OpenStack CI • releasenotes -- Contains release notes for patches • tools -- Config generation script, driver list generation, etc.

22. Cinder Code Layout - /cinder • api -- v1, 2, 3 and API extension implementation • backup -- Backup driver API and implementations • brick -- Legacy brick directory, mostly migrated to os-brick library • cmd -- Starter scripts for Cinder’s processes (api, scheduler, volume, etc.) • common -- Shared files for config, sqlalchemy, etc. • compute -- Source for interfacing with Nova • config -- Source files for config file generator • consistencygroup -- Consistency Group API implementation • db -- sqlalchemy migration scripts and DB API implementation

23. Cinder Code Layout - /cinder (cont.) • group -- Generic Group API implementation • hacking -- Hacking check implementation • image -- Source for interfacing with Glance • interface -- Base interface definition for driver validation and reference • keymgr -- Simple security key implementation • locale -- Translation files for log messages • message -- User message API implementation • objects -- Implementation of Cinder’s Oslo Versioned Objects • scheduler -- Scheduling option implementations • tests -- Compliance, functional, tempest and unit test implementation • transfer -- API implementation of volume transfer • volume -- Implementation of API and drivers for volumes • wsgi -- Cinder Web Server Gateway Interface implementation • zonemanager -- Fibre Channel Zonemanager implementation

24. Release Notes • Developer’s way to communicate to the end user • Most important for changes that impact Cinder’s functionality • Documents: • Security issue fixes • New features • Upgrade notes • Known issues • Deprecation notes • Not required for every change, but important for anything that directly impacts packagers or users

25. Upgrade Checks • Helps operators to ensure that their cloud is compatible with the upgrade • Changes that may impact an upgrade should include a check • Option removal • Driver removal • Database changes • Etc. • Not required for every change, but good for catching those cases where one might worry ‘what if the environment were set up this way?’

26. Tox Tests • Tox is a test runner used by most OpenStack projects • Many of the same test targets, but a few specific to Cinder • Before submitting a patch, best to at least run py27 and pep8 targets(tox -e py27,fast8)

27. Q & ATwitter & IRC: JungleboyJE-mail: jsbryant@electronicjungle.net