Mini-GPT From Scratch (PyTorch + Django) — AI Final Year Project With Live Demo

What This Project Actually Does

This isn't another project that imports a giant library and calls it a day. It builds a small GPT language model from the ground up in PyTorch — the actual attention math, the causal mask, the token and positional embeddings, all of it written out in model/gpt.py where you can read it. No transformers library. No pretrained weights pulled from somewhere. The model learns to predict the next character of text one step at a time, and you get to watch it happen.

And it's tiny on purpose. It trains on a plain CPU laptop with 8GB RAM in under half an hour, so you don't need a fancy GPU or a Colab subscription to run your own final year project end of the semester.

The Django web page is where this gets interesting for a viva. You type a prompt, hit Generate, and the model writes text character by character in front of you — while the same page shows you the tokenization, the training loss curve, a before-vs-after-training comparison, the top next-character probabilities, and an attention heatmap of what the model was focusing on. Basically everything an examiner might poke at is already on screen.

Key Features

• A full GPT written from scratch — Q/K/V projections, multi-head causal self-attention, a per-token feed-forward network, residual connections, pre-norm LayerNorm — all in one readable file you can defend line by line.
• Type a prompt in the browser and watch text generate live through the /generate/ endpoint, with temperature and top-k sampling controls you can actually change and see the effect of.
• An attention heatmap of the last transformer block, drawn as a colored grid, so "self-attention" stops being an abstract phrase and becomes something sitting on your screen.
• A before-vs-after-training panel — pure gibberish from random weights on one side, readable-ish text on the other. This one comparison is the clearest proof your model actually learned something.
• Live training loss curve via Chart.js, pulled straight from logs/loss_history.json, plus a top-10 next-character probability bar chart.
• Every generation click gets logged to SQLite through a GenerationLog model, viewable in Django admin — a nice answer for when someone asks "where's the database actually used?"

Real-World Applications

Look, nobody's shipping this tiny model into production — and that's not the point. But the ideas inside it are exactly what runs ChatGPT, Gemini, and every other LLM you've heard of, just shrunk to a size a laptop can handle. Understanding this project means you can actually explain how modern language models work instead of hand-waving your way through the viva.

It also works well as a teaching demo for a "how do transformers work" session, or as a starting point if you later want to train it on a different corpus — a specific author's writing, code, song-style text, whatever. Swap data/input.txt, retrain, done.

Who Should Buy This

If you're a BCA, MCA, BTech CSE, or BSc IT student who picked "AI" for your final year and then realized most so-called AI projects online are just a scikit-learn model with a Streamlit wrapper — this is the one that'll actually stand out. It's genuinely from scratch.

Fair warning on the one hard part: the attention mechanism takes real effort to understand well enough to defend under questioning. But here's the thing that makes it easier than it looks — the whole project is built so the visualizations explain the concept for you. You see the heatmap light up over the earlier characters, you get why it's called self-attention. If you'd rather not go deep on the math, the Python final year projects hub has simpler options. But if you want a project that doesn't blend into a room full of CRUD apps, this is it.

Why CodeAj

You get the full source code, a college-format project report, and setup support until it's running on your machine. If your PyTorch install is fighting you or the checkpoint won't load, we get on it with you — that's part of the deal, not an upsell. Compare it against other machine learning projects with source code or browse the wider AI/ML project category before you commit if you want.