8 Advanced parallelization - Deep Learning with JAX

Using easy-to-revise parallelism with xmap() · Compiling and automatically partitioning functions with pjit() · Using tensor sharding to achieve parallelization with XLA · Running code in multi-host configurations

Dive into Deep Learning — Dive into Deep Learning 1.0.3 documentation

Self-directed online machine learning for topology optimization

Top 11 Machine Learning Software - Learn before you regret

Tutorial 6 (JAX): Transformers and Multi-Head Attention — UvA DL

Fully Sharded Data Parallel: faster AI training with fewer GPUs

Lecture 2: Development Infrastructure & Tooling - The Full Stack

Lecture 2: Development Infrastructure & Tooling - The Full Stack

Learn JAX in 2023: Part 2 - grad, jit, vmap, and pmap

Deep learning to decompose macromolecules into independent

Compiler Technologies in Deep Learning Co-Design: A Survey

Convolution hierarchical deep-learning neural network (C-HiDeNN