Apache MXNet AI

1. What Is Apache MXNet ? ● A deep learning framework ● Open source Apache 2.0 license ● Supports distributed gpu cluster training/deployment ● Of deep neural networks ● It supports a variety of language bindings ● Supports hybridize for increased speed/optimization ● Supports near linear scaling on gpu / host clusters ● Provides support for the Horovod framework

2. MXNet Language Bindings ● MXNet has a Python based API ● MXNet also supports the following language bindings – Scala – Julia – Clojure – Java – C++ – R – Perl

3. MXNet Related Terms A distributed deep learning framework from Uber Horovod MMS MXNet Model Server DGL Deep Graph Library ONNX Open Neural Network Exchange Hyperparameter A parameter whose value is used to control the learning process D2l.ai A jupyter notebook based deep learning book for Mxnet ++ KVStore Key-value store interface used by MXNet DMLC Distributed (Deep) Machine Learning Community - GitHub

4. MXNet Eco System Coach RL A Python reinforcement learning framework Deep Graph DGL is a Python pkg for deep learning on graphs GluonFR A community driven toolkit for face detection and recognition InsightFace A face detection and recognition repository Keras-MXNet A back end of high level API Keras MXBoard Logging API's for TensorBoard visualisation MXFusion A modular deep probabilistic programming library MXNet Model A flexible tool for serving models exported from Mxnet Optuna A hyperparameter optimization framework Sockeye A sequence to sequence framework for neural translation

5. MXNet Eco System TensorLY A high level API for tensor methods TVM An open deep learning stack for GPU's, CPU's etc Xfer A library for the transfer of knowledge in deep nets GluonCV A computer vision toolkit with a rich model zoo GluonNLP Deep learning models for natural language processing GluonTS A toolkit for probabilistic time series modelling

6. MXNet User Community

7. MXNet Architecture

8. MXNet Architecture ● Runtime Dependency Engine – Schedules and executes the operations – According to their read/write dependency ● Storage Allocator – Efficiently allocates and recycles memory blocks – On host (CPU) and devices (GPUs) ● Resource Manager – Manages global resources, such as – The random number generator and temporal space ● NDArray – Dynamic, asynchronous n-dimensional arrays

9. MXNet Architecture ● Symbolic Execution – Static symbolic graph executor, which provides – Efficient symbolic graph execution and optimization ● Operator – Operators that define static forward/gradient calc (backprop) ● SimpleOp – Operators that extend NDArray operators and – Symbolic operators in a unified fashion ● Symbol Construction – Symbolic construction, which provides a way to construct – A computation graph (net configuration)

10. MXNet Architecture ● KVStore – Key-value store interface for efficient parameter synchronization ● Data Loading(IO) – Efficient distributed data loading and augmentation

11. MXNet Data Loading ● For large data sets data loading is optimized in MXNet ● Data format –Uses dmlc-core’s binary recordIO implementation ● Data Loading – Reduced IO cost by utilizing the threaded iterator – Provided by dmlc-core ● Interface design – Write MXNet data iterators in just a few lines of Python

12. MXNet Dependency Engine ● Helps to parallelize computation across devices ● Helps to synchronize computation when – We introduce multi-threading ● A run time dependency schedule graph is created ● The graph is then used to – Optimize processing – Optimize memory use – Aid parallelism when using ●GPU / CPU clusters ● For deep learning memory use – Usage during training > during prediction

13. MXNet Forward Vs Backward Graph

14. Available Books ● See “Big Data Made Easy” Apress Jan 2015 – See “Mastering Apache Spark” ● Packt Oct 2015 – See “Complete Guide to Open Source Big Data Stack ● “Apress Jan 2018” – ● Find the author on Amazon www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ – Connect on LinkedIn ● www.linkedin.com/in/mike-frampton-38563020 –

15. Connect ● Feel free to connect on LinkedIn –www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at open-source-systems.blogspot.com/ – ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration