EXPERIMENT — PROGRESS

ML From Scratch

A machine learning library built from first principles, implementing seven core algorithms without relying on ready-made model implementations. The project focuses on understanding the mathematics, optimization, and mechanics behind the abstractions hidden inside `.fit()` and `.predict()`.

Overview

ML From Scratch is an ongoing attempt to understand machine learning by rebuilding its core algorithms from the ground up.

Rather than treating models as black boxes, each implementation is derived from the underlying mathematics and translated into working Python code. The goal is to understand exactly what happens between the data entering a model and a prediction coming out.

The project currently contains implementations of seven machine learning algorithms:

• Linear Regression
• Logistic Regression
• Decision Trees
• K-Nearest Neighbors
• Naive Bayes
• Bagging
• Random Forest

Each algorithm is implemented without using a ready-made machine learning model from libraries such as scikit-learn.

What I am learning

The project is less about collecting implementations and more about understanding the mechanisms behind them.

That means working through:

• Loss functions and objective functions
• Gradients and optimization
• Splitting criteria and information gain
• Probability and statistical assumptions
• Distance-based prediction
• Ensemble methods
• Bootstrap sampling
• Recursive tree construction
• Prediction and decision boundaries

The aim is to make the abstraction disappear and leave the underlying mechanism visible.

Why build it?

Modern machine learning libraries make experimentation extremely convenient, but that convenience can hide a lot of important details.

Building the algorithms myself forces me to answer questions that .fit() normally hides:

What is actually being optimized?

How does the model decide what to learn?

Where does each prediction come from?

What mathematical assumption is the algorithm making?

This project is my way of answering those questions through implementation rather than just theory.

Direction

The library is still evolving.

The long-term goal is to turn these individual implementations into a small, clean machine learning library with a consistent API, reusable utilities, evaluation tools, preprocessing components, and eventually benchmarks against established libraries.

The next stage is to move beyond individual algorithms and start building the infrastructure that makes them work together as a coherent library.