- Hi, this is Nemo speaking. I’m a second-year undergraduate student at IIIS(Yao Class), Tsinghua University, persuing a B.S. in Computer Science.
- My goal is to share a piece of my mind here. Topics may include but not limited to: Computer Science (especially AI-related subjects, e.g. Machine Learning, Reinforcement Learning, Generative Models,etc.), Mathematics, Physics, and Simply Thoughts on Life. Enjoy!
Titans: Learning to Memorize at Test Time
This article introduces Titans, a novel architecture that as a meta in-context learner, learns to memorize at test time. Through designing a long-term memory module, and proposing three variants of Titans (MAC, MAG, MAL), the model achieves superior performance compared to Transformers and other baselines, especially in long-context tasks.
Life Hacks Series: 1. How to manage your Python Environment
This is the first article in the Life Hacks Series. It covers how to manage your Python environment. Basically, it covers how to install packages, how to create a new environment, how to clone an environment, and how to pack an environment. A special mention is Conda-Pack, which really made my life a lot easier.
Machine Learning Series: 5.Hyperparameter Selection
This is the fifth article in the Machine Learning Series. It covers classic approaches to Hyperparameter Selection, including Bayesian Optimization, Gradient Optimization, Random Search, Multi-Arm Bandits and Neural Architecture Search.
Machine Learning Series: 4.Robust Machine Learning
This is the fourth article in the Machine Learning Series. It covers classic approaches to Robust Machine Learning, including Adversial Attacks, Adversial Training, Robust Features, Obfuscated Gradients and Provable Robust Certificates.
Machine Learning Series: 3.Unsupervised Learning(II)
This is the third article in the Machine Learning Series. It covers the second part of unsupervised learning, including topics like Clustering, Spectral Graph Clustering, SimCLR, SNE and t-SNE.
Machine Learning Series: 2.Unsupervised Learning(I)
This is the second article in the Machine Learning Series. It covers the first part of unsupervised learning, including topics like Dimension Reduction, PCA, k-NN, LSH and Metric Learning.
Natural Language Processing: Part B. Modern Approaches
This is the second part of the Natural Language Processing Series. It covers modern approaches in natural language processing, including RNNs, VAE-LMs, Transformer, BERT, GPT, GAN-LMs, In-Context Learning, CoT, RLHF, DPO, etc.
Machine Learning Series: 1.Optimization, Generalization and Supervised Learning
This is the first article in the Machine Learning Series. It covers the basics of optimization(GD,SGD,SVRG,Mirror Descent,Linear Coupling), generalization(No Free Lunch, PAC Learning, VC Dimension), and supervised learning(Linear Regression, Logistic Regression, Compressed Sensing).
Complex Analysis and its application in Electrostatics
This blog post is a summary of my notes on complex analysis and its application in electrostatics.
Natural Language Processing: Part A. Classical Methods
This is the first part of the Natural Language Processing Series. It covers classical methods in natural language processing, including CFG, LSA, HMM, N-gram, Word2Vec, etc.