Algebra and Computation
Lecture notes and Cheat Sheet for Course: Algebra and Computation, which is a course offered to Freshman students at Yao Class.
Posts
Titans: Learning to Memorize at Test Time
This article introduces Titans, a novel architecture that as a meta in-context learner, learns to memorize at test time. Through designing a long-term memory module, and proposing three variants of Titans (MAC, MAG, MAL), the model achieves superior performance compared to Transformers and other baselines, especially in long-context tasks.
Life Hacks Series: 1. How to manage your Python Environment
This is the first article in the Life Hacks Series. It covers how to manage your Python environment. Basically, it covers how to install packages, how to create a new environment, how to clone an environment, and how to pack an environment. A special mention is Conda-Pack, which really made my life a lot easier.
Machine Learning Series: 5.Hyperparameter Selection
This is the fifth article in the Machine Learning Series. It covers classic approaches to Hyperparameter Selection, including Bayesian Optimization, Gradient Optimization, Random Search, Multi-Arm Bandits and Neural Architecture Search.
Machine Learning Series: 4.Robust Machine Learning
This is the fourth article in the Machine Learning Series. It covers classic approaches to Robust Machine Learning, including Adversial Attacks, Adversial Training, Robust Features, Obfuscated Gradients and Provable Robust Certificates.
Machine Learning Series: 3.Unsupervised Learning(II)
This is the third article in the Machine Learning Series. It covers the second part of unsupervised learning, including topics like Clustering, Spectral Graph Clustering, SimCLR, SNE and t-SNE.
Machine Learning Series: 2.Unsupervised Learning(I)
This is the second article in the Machine Learning Series. It covers the first part of unsupervised learning, including topics like Dimension Reduction, PCA, k-NN, LSH and Metric Learning.
Natural Language Processing: Part B. Modern Approaches
This is the second part of the Natural Language Processing Series. It covers modern approaches in natural language processing, including RNNs, VAE-LMs, Transformer, BERT, GPT, GAN-LMs, In-Context Learning, CoT, RLHF, DPO, etc.
Machine Learning Series: 1.Optimization, Generalization and Supervised Learning
This is the first article in the Machine Learning Series. It covers the basics of optimization(GD,SGD,SVRG,Mirror Descent,Linear Coupling), generalization(No Free Lunch, PAC Learning, VC Dimension), and supervised learning(Linear Regression, Logistic Regression, Compressed Sensing).
Complex Analysis and its application in Electrostatics
This blog post is a summary of my notes on complex analysis and its application in electrostatics.