awesome-reading-materials

A curated list of books and papers for reading. For knowledge in general. 'Cuz reading is fun.

Books

1. JavaScript: The Definitive Guide O'Reilly
2. JavaScript: The Good parts O'Reilly
3. Steven Shreve: Stochastic Calculus for Finance Volumes 1 & 2
4. JD Hamilton: Time Series Analysis
5. Simo Sarrka: Bayesian Filtering and Smoothing
6. Dani Gamerman: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference
7. Understanding the Linux Kernel O'Reilly
8. Understanding Linux Networking Internals O'Reilly
9. Linux Device Drivers O'Reilly
10. Martin Baxter: Financial Calculus: An Introduction to Derivative Pricing
11. Rasumssesn, Williams: Gaussian Processes for Machine Learning
12. Michael Kerrisk: The Linux Programming Interface
13. Igor Tulchinsky: Finding Alphas: A Quantitative Approach to Trading Algorithms
14. Simo Sarrka: Applied Stochastic Differential Equations
15. Jason Sanders: CUDA by Example: Introduction to GPGPU Programming
16. John Cheng: Professional CUDA C Programming
17. Thomas Sargent: Quantitative Economics with Python
18. Peter Salzman: The Linux Kernel Module Programming Guide
19. Andrzej Chrzȩszczyk: Matrix computations on the GPU: CUBLAS, CUSOLVER and MAGMA by example
20. Dan Bader: Python Tricks - A Buffet of Awesome Python Features
21. Steve Scargall: Persistent Memory Programming
22. Wulfram Gerstner: Neuronal Dynamics: https://neuronaldynamics.epfl.ch/
23. Roman Vershynin: High Dimensional Probability
24. Steve Merschner: Fundamentals of Computer Graphics
25. Simon Prince: Computer vision: models, learning and inference
26. Feng-Yu Wang: Analysis of Diffusion Processes on Riemannian Manifolds
27. Peter Kloeden: Numerical solution of SDEs
28. Nobuyuki Ikeda: SDEs and Diffusion Processes
29. Shunichi Amari: Methods of Information Geometry
30. Allen Downy: The Little Book of Semaphores
31. Maurice Bach: The Design of the UNIX Operating System
32. David Kirk: Programming Massively parallel processors
33. Natural Language Processing with PyTorch O'Reilly
34. Stephen Boyd: Convex Optimization
35. Duda, Hart, Stork: Pattern Classification
36. Deep Learning in PyTorch, from the core team of PyTorch

Papers

1. Latent ODEs for Irregularly-Sampled Time Series: https://arxiv.org/abs/1907.03907
2. A Framework for Reinforcement Learning and Planning: https://arxiv.org/abs/2006.15009
3. Synthetic Data for Deep Learning: https://arxiv.org/abs/1909.11512
4. Variational Inference with Normalizing Flows: https://arxiv.org/abs/1505.05770
5. Neural Manifold Ordinary Differential Equations: https://arxiv.org/abs/2006.10254
6. Bayesian Learning via Stochastic Gradient Langevin Dynamics, Max Welling et.al
7. Reverse-Engineering Deep ReLU Networks: https://arxiv.org/abs/1910.00744
8. A Neural Scaling Law from the Dimension of the Data Manifold: https://arxiv.org/abs/2004.10802
9. An Introduction to Deep Reinforcement Learning: https://arxiv.org/abs/1811.12560
10. Continuous-discrete smoothing of diffusions: https://arxiv.org/abs/1712.03807
11. Universal Differential Equations for Scientific Machine Learning: https://arxiv.org/abs/2001.04385
12. Visualizing Higher-Layer Features of a Deep Network: Yoshua Bengio et.al
13. Tensor Programs II: Neural Tangent Kernel for Any Architecture: https://arxiv.org/abs/2006.14548
14. Mean Field Residual Networks: On the Edge of Chaos: https://arxiv.org/abs/1712.08969
15. An Artificial Neuron Implemented on an Actual Quantum Processor: https://arxiv.org/abs/1811.02266
16. A Survey of Neuromorphic Computing and NeuralNetworks in Hardware: https://arxiv.org/abs/1705.06963
17. A Conceptual Introduction to Markov ChainMonte Carlo Methods: https://arxiv.org/abs/1909.12313
18. Scalable Gradients for Stochastic Differential Equations: https://arxiv.org/abs/2001.01328
19. An Introduction to MCMC for Machine Learning: https://link.springer.com/content/pdf/10.1023/A:1020281327116.pdf
20. PyQuil: A Practical Quantum Instruction Set Architecture: https://arxiv.org/abs/1608.03355
21. Computational Neuroscience: Mathematical and Statistical Perspectives: https://users.ece.cmu.edu/~byronyu/papers/KassARS2018.pdf
22. Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit: https://arxiv.org/abs/1905.09883
23. Towards an Integration of Deep Learning and Neuroscience: https://www.frontiersin.org/articles/10.3389/fncom.2016.00094/full
24. Theoretical guarantees for sampling and inference in generative models with latent diffusions: https://arxiv.org/abs/1903.01608
25. Hamiltonian Descent Methods: https://arxiv.org/abs/1809.05042
26. Only Bayes should learn a manifold: https://arxiv.org/abs/1806.04994
27. Generative Modeling by Estimating Gradients of the Data Distribution: https://arxiv.org/abs/1907.05600
28. Are All Layers Created Equal?: https://arxiv.org/abs/1902.01996
29. Geometric deep learning: Going beyond Euclidean data: https://arxiv.org/abs/1611.08097
30. A survey on Deep Learning Advances on Different 3D Data Representations: https://arxiv.org/abs/1808.01462
31. Word2vec Parameter Learning Explained: https://arxiv.org/abs/1411.2738
32. moDNN: Memory Optimal Deep Neural Network Training on Graphics Processing Units
33. Machine Learning at Facebook: Understanding Inference at the Edge
34. On Exact Computation with an Infinitely Wide Neural Net: https://arxiv.org/abs/1904.11955
35. Mean-field Behaviour of Neural Tangent Kernel for Deep Neural Networks: https://arxiv.org/abs/1905.13654
36. Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent: https://arxiv.org/abs/1902.06720
37. Statistical Mechanics of Deep Learning: Annual Review of Condensed Matter Physics: Yasaman Bahri et.al
38. Recurrent Neural Processes: Timon Willi, Jurgen Schmidhuber et.al
39. Neural SDE - Information propagation through the lens of diffusion processes: Stefano Peluchetti et.al
40. Deep Neural Networks in Computational Neuroscience: Tim Kietzmann et.al
41. Understanding Deep Neural Networks with ReLu: Raman Arora et.al
42. On the Learning Dynamics of Deep Neural Networks: https://arxiv.org/abs/1809.06848
43. On the Spectral Bias of Neural Networks: https://arxiv.org/abs/1806.08734
44. Riemannian metrics for neural networks I: Feedforward networks: https://arxiv.org/abs/1303.0818
45. (First paper on Neural Networks): A logical calculus of the ideas immanent in nervous activity: Warren McCulloch and Walter Pitts
46. Hybrid computing using a neural network with dynamic external memory: Alex Graves et.al, Nature
47. Stochastic Modified Equations and Adaptive Stochastic Gradient Algorithms: Qianxiao Li et.al
48. Natural Langevin Dynamics for Neural Networks: https://arxiv.org/abs/1712.01076
49. On the Curved Geometry of Accelerated Optimization: Aaron Defazio Facebook AI Research
50. The Variational Gaussian Process: Dustin Tran et.al
51. Stochastic Gradient Langevin Dynamics that exploit Neural Network Structure: Zachary Nado et.al, Google Brain
52. Stochastic Gradient Descent performs Variational Inference, converges to Limit Cycles for Deep Networks, Pratik Chaudhuri et.al
53. Hybrid Monte Carlo (original paper intriducing Hamiltonian Monte Carlo): Simon Duane et.al
54. Fluctuation Dissipation Relations for Stochastic Gradient Descent: Sho Yaida, Facebook AI Research
55. Neural Machine Translation by Jointly Learning to Align and Translate: https://arxiv.org/abs/1409.0473
56. Stochastic Gradient Descent as Approximate Bayesian Inference: https://arxiv.org/abs/1704.04289

Articles

1. Internals of the Python Compiler cpython: https://realpython.com/cpython-source-code-guide/
2. Original article mentioning stack buffer overflows, a technique used in malicious program execution: http://phrack.org/issues/49/14.html#article
3. Linux Internals Descriptions: https://0xax.gitbooks.io/linux-insides/
4. DeepMind's AI for solving the protein folding problem: https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-discovery
5. C Heap internals: https://azeria-labs.com/heap-exploitation-part-1-understanding-the-glibc-heap-implementation/
6. Discovering Symbolic Models from Deep Learning with Inductive Biases (Includes discovery of symbolic mathematical equations from experimental data, and application to dark matter physics): https://astroautomata.com/paper/symbolic-neural-nets/
7. GPU Gems 3: N-body simulations in CUDA: https://developer.nvidia.com/gpugems/gpugems3/part-v-physics-simulation/chapter-31-fast-n-body-simulation-cuda
8. AMD Whitepaper on Nested Paging: http://developer.amd.com/wordpress/media/2012/10/NPT-WP-1%201-final-TM.pdf
9. Neural 3D Mesh Renderer: http://hiroharu-kato.com/projects_en/neural_renderer.html
10. Key trends from NeurIPS 2019: https://huyenchip.com/2019/12/18/key-trends-neurips-2019.html
11. Deriving the Neural Tangent Kernel: https://brynhayder.github.io/jekyll/update/2019/04/02/neural-tangent-kernel.html
12. The Unreasonable Effectiveness of RNNs: http://karpathy.github.io/2015/05/21/rnn-effectiveness/
13. Mathematical Foundations of Neuroscience: https://blog.piekniewski.info/2018/09/03/mathematical-foundations-of-neuroscience/
14. Bayesian Neural Networks Need Not Concentrate: https://jacobbuckman.com/2020-01-22-bayesian-neural-networks-need-not-concentrate/
15. Google AI: DeepDream: https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html
16. Real, protected and long mode in Assembly: https://www.codeproject.com/Articles/45788/The-Real-Protected-Long-mode-assembly-tutorial-for
17. Intel Assembly Manual guide: https://www.codeproject.com/Articles/1273844/The-Intel-Assembly-Manual-3
18. Intel 8085 hardware internals and register electronic structure: https://www.righto.com/2013/03/register-file-8085.html
19. Zoom in to neural circuits: https://distill.pub/2020/circuits/zoom-in/

Repositories

1. Beginner Reinforcement Learning Tutorial Notebook: https://github.com/eemlcommunity/PracticalSessions2020/blob/master/rl/EEML2020_RL_Tutorial.ipynb
2. Neural Differential Equations: https://github.com/rtqichen/torchdiffeq
3. Awesome List to build Computer Systems and softwares from scratch: https://github.com/danistefanovic/build-your-own-x
4. Neural Tangent Kernel: https://github.com/google/neural-tangents
5. N-body simulations, including Dark Matter simulations: https://github.com/franciscovillaescusa/Quijote-simulations
6. A user-space file system for interacting with Google Cloud Storage, written in Go FUSE: https://github.com/GoogleCloudPlatform/gcsfuse
7. Introduction to Flask using Test Driven Development: https://github.com/mjhea0/flaskr-tdd
8. Reinforcement Learning for Algorithmic Trading: https://github.com/tensortrade-org/tensortrade
9. Deep Learning Papers Reading Roadmap: https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap
10. Lucid Framework for Neural Network Interpretability: https://github.com/tensorflow/lucid
11. Sktime: Machine Learning with Time series: https://github.com/alan-turing-institute/sktime
12. Machine Learning Tokyo: https://github.com/Machine-Learning-Tokyo
13. System Design Primer: https://github.com/donnemartin/system-design-primer
14. Apollo-11 code, which took humanity to the moon: https://github.com/chrislgarry/Apollo-11
15. Awesome AI book list: https://github.com/zslucky/awesome-AI-books
16. Jupyter Notebooks on Mathematical Finance: https://github.com/cantaro86/Financial-Models-Numerical-Methods
17. Awesome Transfer Learning Reference: https://github.com/artix41/awesome-transfer-learning

Courses

1. MIT Computer Networks: http://web.mit.edu/6.829/www/currentsemester/lectures_gen.html
2. MIT Operating Systems: https://pdos.csail.mit.edu/6.828/2019/schedule.html
3. MIT Distributed Systems: https://pdos.csail.mit.edu/6.824/schedule.html
4. MIT Finance Theory 1: https://ocw.mit.edu/courses/sloan-school-of-management/15-401-finance-theory-i-fall-2008
5. MIT Statistical Learning Theory: https://www.mit.edu/~9.520/fall19/
6. CMU Statistical Machine Learning: http://www.stat.cmu.edu/~ryantibs/statml/
7. MIT Topics in Mathematics with Applications in Finance: https://ocw.mit.edu/courses/mathematics/18-s096-topics-in-mathematics-with-applications-in-finance-fall-2013/lecture-notes/
8. CalTech GPU Programming: http://courses.cms.caltech.edu/cs179/
9. CMU Deep Reinforcement Learning: https://cmudeeprl.github.io/703website/
10. UC Berkeley Computer Security: https://cs161.org/
11. Aalto University Nonlinear Dynamics and Chaos: https://mycourses.aalto.fi/course/view.php?id=24404&section=1
12. Aalto University Machine Learning Advanced Probabilistic Methods: https://mycourses.aalto.fi/course/view.php?id=24365&section=1
13. Aalto University Kernel Methods in Machine Learning: https://mycourses.aalto.fi/course/view.php?id=24366
14. UWaterloo Distributed Computer Systems: https://cs.uwaterloo.ca/~rtholmes/teaching/2011winter/cs436/index.html
15. UC Berkeley Deep Reinforcement Learning: http://rail.eecs.berkeley.edu/deeprlcourse/
16. Stanford Computing with Physical Objects: Algorithms for Shape and Motion: http://graphics.stanford.edu/courses/cs164-09-spring/
17. UC Berkeley Deep Unsupervised Learning: https://sites.google.com/view/berkeley-cs294-158-sp19/home
18. Stanford Decison Making under uncertainty: https://web.stanford.edu/class/aa228/cgi-bin/wp/
19. Princeton Theoretical Foundations of Deep Learning: https://www.cs.princeton.edu/courses/archive/fall18/cos597G/
20. New York University Inference and Representation: https://inf16nyu.github.io/home/
21. MIT and Metacademy Roadmap: Probabilistic Graphical Models: https://metacademy.org/roadmaps/rgrosse/mit_6_438
22. CMU Computational Complexity Theory: http://www.cs.cmu.edu/~odonnell/complexity/
23. CMU Convex Optimization: http://www.cs.cmu.edu/~aarti/Class/10725_Fall17/
24. CMU Advanced Machine Learning Theory and Methods: http://www.cs.cmu.edu/~pradeepr/716/
25. Stanford Deep Multi Task and Meta-Learning: http://cs330.stanford.edu/
26. MIT Missing Semester of CS Education: https://missing.csail.mit.edu/
27. Princeton Introduction to CS: https://www.cs.princeton.edu/courses/archive/spring20/cos126/syllabus.html
28. UPenn Wharton School of Business Financial Time Series: http://www-stat.wharton.upenn.edu/~steele/Courses/956/

Links

1. CIFAR DLRL Summer Schools: https://dlrlsummerschool.ca/past-years/
2. MLSS Summer Schools: http://mlss.cc/
3. Site for Interesting Algorithmic CS, Probability and Discrete Mathematics problems: https://www.algomuse.net/archive
4. Google India AI Summer School: https://sites.google.com/view/aisummerschool2020
5. Econometrics Books: http://www.econometricsbooks.com/
6. Hugging Face – On a mission to solve NLP: https://huggingface.co/
7. MATLAB Computational Finance: https://in.mathworks.com/solutions/finance-and-risk-management.html
8. Key papers in Deep RL: https://spinningup.openai.com/en/latest/spinningup/keypapers.html
9. AI driven Drug discovery: https://www.deepgenomics.com/
10. Montreal.AI: AI for All Cheatsheet: http://www.montreal.ai/ai4all.pdf
11. AI generated lyrics: https://theselyricsdonotexist.com/
12. Free Programming ebooks: https://goalkicker.com/
13. Deep RL Bootcamp: https://sites.google.com/view/deep-rl-bootcamp/lectures
14. Bayesian Deep Learning NIPS 2019: http://bayesiandeeplearning.org/
15. UCSD Mathematical Neuroscience Lab: http://www.silva.ucsd.edu/home-1
16. UCSD Centre for Engineered Natural Intelligence: http://ceni.ucsd.edu/
17. Institute for Pure and Applied Mathematics: Courses on Theoretical Deep Learning: http://www.ipam.ucla.edu/
18. Physics for Deep Learning: https://sites.google.com/view/icml2019phys4dl/schedule
19. IBM Mathematics of AI Group: https://researcher.watson.ibm.com/researcher/view_group.php?id=9860
20. Deep Learning for Physics: https://dl4physicalsciences.github.io/
21. Geometric Deep Learning: http://geometricdeeplearning.com/
22. Cornell Virtual Workshop on Programming languages and frameworks: https://cvw.cac.cornell.edu/topics
23. Metacacdemy Dynamical Systems for Machine Learning: https://metacademy.org/roadmaps/DanielIm/dsml
24. Metacacdemy Bayesian Machine Learning: https://metacademy.org/roadmaps/rgrosse/bayesian_machine_learning
25. Metacacdemy Differential Geometry for Machine Learning: https://metacademy.org/roadmaps/rgrosse/dgml
26. Massive List of Free Courses from Coursera: https://www.freecodecamp.org/news/coursera-free-online-courses-6d84cdb30da/
27. Systems Programming Blog: https://gpfault.net/
28. Deep Differential equations ICLR: http://iclr2020deepdiffeq.rice.edu/
29. GPU Architecture Resources: https://interplayoflight.wordpress.com/2020/05/09/gpu-architecture-resource
30. List of upcoming seminars and talks in CS and basic sciences: https://researchseminars.org/talks

Softwares and Libraries/Frameworks

1. Deep Graph Library: https://www.dgl.ai/
2. Prometheus - Monitoring system & time series database: https://prometheus.io/
3. Intel® Data Analytics Acceleration Library: https://software.intel.com/content/www/us/en/develop/tools/data-analytics-acceleration-library
4. Intel® Math Kernel Library for Deep Learning Networks: https://software.intel.com/content/www/us/en/develop/articles/intel-mkl-dnn-part-1-library-overview-and-installation.html
5. Google Cloud Storage FUSE, written in Golang: https://cloud.google.com/storage/docs/gcs-fuse
6. Intel Thread Building Blocks for parallel and heterogeneous computing: https://software.intel.com/content/www/us/en/develop/tools/threading-building-blocks.html
7. Turing.jl - Bayesian Inference for Probabilistic Programming: https://turing.ml/dev/
8. Gen.jl - A general purpose Probabilistic Programming Language: https://www.gen.dev/
9. Thermodynamic Analytics Toolkit (TATi), Alan Turing Institute: https://alan-turing-institute.github.io/ThermodynamicAnalyticsToolkit/
10. Stan - A Probabilistic Programming Framework: https://mc-stan.org
11. ArviZ - Exploratory analysis of Bayesian models: https://arviz-devs.github.io/arviz/
12. PyMC3 - Probabilistic Programming in Python: https://docs.pymc.io/
13. Uber Pyro - A Probabilistic Programming Language: https://eng.uber.com/pyro/
14. PyTorch 3D for 3D Deep Learning: https://pytorch3d.org/
15. Kaolin - A framework for 3D Computer Vision: https://kaolin.readthedocs.io/en/latest/
16. DLib C++ Library for Machine Learning: http://dlib.net/
17. WebSocket API: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
18. Box2D - A 2D physics engine for games in C++: https://box2d.org/documentation/index.html
19. Carla simulator for autonomous driving: http://carla.org/
20. CVXPY - Differentiable Convex Optimization Layers: https://locuslab.github.io/2019-10-28-cvxpylayers/
21. NeuroML - Models for Computational Neuroscience: https://neuroml.org/
22. NeuroTools - Computational neuroscience framework: https://pythonhosted.org/NeuroTools/
23. Stumpy - Time series motif and data framework: https://stumpy.readthedocs.io/en/latest/index.html
24. Affinity - Deep Learning for Molecular Geometry: https://affinity.mit.edu/
25. Ray- Framework for running distributed applications: https://docs.ray.io/en/latest/index.html
26. Magnum engine - Graphics engine in C++: https://magnum.graphics/
27. TA-Lib - Trading and Quantitative Finance Library: https://mrjbq7.github.io/ta-lib/