Mini-GPT From Scratch (PyTorch + Django) — AI Final Year Project With Live Demo
A working GPT built from scratch in PyTorch — no transformers library, no pretrained weights — with a live Django demo that shows attention, tokenization, and text generation happening in real time. Trains on a normal laptop.
Technology Used
Python | PyTorch | Django | SQLite | Chart.js | HTML/CSS/JavaScript
What This Project Actually Does
This isn't another project that imports a giant library and calls it a day. It builds a small GPT language model from the ground up in PyTorch — the actual attention math, the causal mask, the token and positional embeddings, all of it written out in model/gpt.py where you can read it. No transformers library. No pretrained weights pulled from somewhere. The model learns to predict the next character of text one step at a time, and you get to watch it happen.
And it's tiny on purpose. It trains on a plain CPU laptop with 8GB RAM in under half an hour, so you don't need a fancy GPU or a Colab subscription to run your own final year project end of the semester.
The Django web page is where this gets interesting for a viva. You type a prompt, hit Generate, and the model writes text character by character in front of you — while the same page shows you the tokenization, the training loss curve, a before-vs-after-training comparison, the top next-character probabilities, and an attention heatmap of what the model was focusing on. Basically everything an examiner might poke at is already on screen.
Key Features
- A full GPT written from scratch — Q/K/V projections, multi-head causal self-attention, a per-token feed-forward network, residual connections, pre-norm LayerNorm — all in one readable file you can defend line by line.
- Type a prompt in the browser and watch text generate live through the
/generate/endpoint, with temperature and top-k sampling controls you can actually change and see the effect of. - An attention heatmap of the last transformer block, drawn as a colored grid, so "self-attention" stops being an abstract phrase and becomes something sitting on your screen.
- A before-vs-after-training panel — pure gibberish from random weights on one side, readable-ish text on the other. This one comparison is the clearest proof your model actually learned something.
- Live training loss curve via Chart.js, pulled straight from
logs/loss_history.json, plus a top-10 next-character probability bar chart. - Every generation click gets logged to SQLite through a
GenerationLogmodel, viewable in Django admin — a nice answer for when someone asks "where's the database actually used?"
Real-World Applications
Look, nobody's shipping this tiny model into production — and that's not the point. But the ideas inside it are exactly what runs ChatGPT, Gemini, and every other LLM you've heard of, just shrunk to a size a laptop can handle. Understanding this project means you can actually explain how modern language models work instead of hand-waving your way through the viva.
It also works well as a teaching demo for a "how do transformers work" session, or as a starting point if you later want to train it on a different corpus — a specific author's writing, code, song-style text, whatever. Swap data/input.txt, retrain, done.
Who Should Buy This
If you're a BCA, MCA, BTech CSE, or BSc IT student who picked "AI" for your final year and then realized most so-called AI projects online are just a scikit-learn model with a Streamlit wrapper — this is the one that'll actually stand out. It's genuinely from scratch.
Fair warning on the one hard part: the attention mechanism takes real effort to understand well enough to defend under questioning. But here's the thing that makes it easier than it looks — the whole project is built so the visualizations explain the concept for you. You see the heatmap light up over the earlier characters, you get why it's called self-attention. If you'd rather not go deep on the math, the Python final year projects hub has simpler options. But if you want a project that doesn't blend into a room full of CRUD apps, this is it.
Why CodeAj
You get the full source code, a college-format project report, and setup support until it's running on your machine. If your PyTorch install is fighting you or the checkpoint won't load, we get on it with you — that's part of the deal, not an upsell. Compare it against other machine learning projects with source code or browse the wider AI/ML project category before you commit if you want.
Frequently Asked Questions
Extra Add-Ons Available – Elevate Your Project
Add any of these professional upgrades to save time and impress your evaluators.
Live 1-on-1 Mentorship
Personal session with an expert developer
Project Setup
We'll install and configure the project on your PC via remote session (Google Meet, Zoom, or AnyDesk).
Source Code Explanation
1-hour live session to explain logic, flow, database design, and key features.
Want to know exactly how the setup works? Review our detailed step-by-step process before scheduling your session.
₹999
Custom Documents (College-Tailored)
- Custom Project Report: ₹1,500
- Custom Research Paper: ₹1,000
- Custom PPT: ₹800
Fully customized to match your college format, guidelines, and submission standards.
Project Modification
Need feature changes, UI updates, or new features added?
Charges vary based on complexity.
We'll review your request and provide a clear quote before starting work.
Project Files
ReviewsWhat Our Students Say
He explained the project clearly and thoroughly. He was very supportive throughout the process and was always available to help when needed. His guidance and assistance made it easier to understand the project and complete tasks effectively.
I had a very good experience with CodeAj Marketplace. The team is professional and supportive, especially Ajay Pandit sir who guides clearly throughout the project. The work was completed neatly and according to requirements. A great place for learning and project development. Highly recommended for students and developers.
Excellent explanation! Concepts are explained in a very simple and understandable way. It really helped me a lot in improving my knowledge. Great work!
Codeaj was really helpful with my project, especially during the customization part. He didn't just do the work but also took the time to explain everything, from the project structure and database to the code itself. It was a huge help, especially for students trying to understand how everything fits together. I really appreciated his support and the way he made things easier to follow.
Working with CodeAj Marketplace has been such a positive experience. The projects delivered are original, carefully crafted from scratch, and tailored to meet expectations with impressive attention to detail. What really makes him stand out is his willingness to go the extra mile — not just in creating outstanding work, but also in helping others understand how everything functions. He explains things clearly, addresses questions patiently, and ensures the whole process feels like a learning journey instead of just a handover. On top of the technical expertise, his calm and approachable personality makes communication effortless. No unnecessary stress, just smooth collaboration and genuine support. I truly value the dedication he brings to his work and would highly recommend him to anyone in search of high-quality, authentic projects.
I purchased a project from him recently and I have to say it's one of the best decisions I've made. The project was absolutely beautiful, built completely from scratch with no shortcuts, and perfectly matched what I wanted. What makes him stand out is not just his ability to create anything we ask for, no matter how complex, but also the way he helps us understand every part of it. He patiently explains the process, clears doubts, and makes sure we actually learn from the project instead of just using it. On top of his skills, his personality is amazing — he's calm, chill, and very friendly to talk to. There's no stress when working with him, only smooth communication and genuine support. I truly appreciate the effort he puts in and would highly recommend him to anyone looking for original, high-quality projects.
I received my project from CodeAJ Marketplace on time, and the work was very neat and well-presented. The team maintained good communication and delivered everything as promised. I appreciate their timely completion and professional approach.
I sincerely thank Mr. Ajay Pandit for his constant support and valuable guidance throughout the project. His timely advice, encouragement, and expertise greatly contributed to the successful completion of the work. I'm truly grateful for his mentorship.
I had an outstanding experience with CodeAj while developing my mobile app. Their team was incredibly cooperative, ensuring every customization I requested was implemented flawlessly. The service was top-notch, and the final product exceeded my expectations. Highly recommend CodeAj for anyone seeking professional and reliable app development services! 11/10
I would like to sincerely thank Ajay Pandit for providing me with the project. His guidance and explanation of the system's architecture, modules, and AI integration were extremely helpful in understanding the project thoroughly. Ajay's support played a key role in helping me implement the hotel booking and payment module effectively. His willingness to clarify concepts and walk me through complex logic made the learning process smooth and insightful. I truly appreciate his mentorship and the time he dedicated to ensuring I was well-prepared for both the development and presentation of this project.
I had a great experience with the project I purchased. It was well-structured and met all my expectations. The team was responsive and supportive throughout, addressing all my queries promptly. Highly recommended for anyone looking for quality projects and good service.
I'm really thankful to Codeaj for helping me with my project explanation and guiding me throughout. Your support made everything much easier and clearer.
The project he made was wonderful, and the way he patiently listened, understood all my ideas and presented them so well in the project, was amazing. I would highly recommend him to everyone.
I would like to sincerely thank Ajay Pandit for providing me with the project. His guidance and explanation of the system's architecture, modules, and AI integration were extremely helpful in understanding the project thoroughly.
Too good experience, I love their projects and they are available for you to help, they are incredible good. Must avail their services. I am happy customer.
The team was very supportive and their explanation was very good, they are clarifying each and every doubts with lots of patience. Highly recommended for quality projects.
I really like their work and they made good project in less amount.
Thank you, CodeAj Bro, for supporting the Doctor Appointment System project. Your guidance during the project setup, installation, and report creation was very helpful and greatly appreciated.
One of the best Software Developer provide the best setup environment for project, good explain skills, and good technical skills, step by step explain project through google meet and budgetly price project Available.
Thank you so much for your excellent work on our project! Your kindness and clear explanations made the entire process smooth and enjoyable. Highly recommend!
Really very supportive, helpful and kind, loving nature. Available for any time for any query, highly recommend to get projects from him.
Awesome projects. They support on setting up the project after purchasing. Completely legal one. Affordable projects too.
Code AJ guide team are very formal, friendly to setup project step by step and explain each question 🙂
Thank you codeAj, Best site to get your project done quickly, with fast responses and instant clarification of any project-related doubts.
I really liked the work, and they solved my issues very well too.
They provide very good service and code also running condition and they fulfill all the queries it's very helpful.
Really helpful, makes things easy to understand and very kind nature, you can look forward to get help from here.
Thank you so much for building the Project for my requirement.
Looking forward to many many collaborations and always tend to buy the projects.
I am so satisfied with codeAJ because of his help.
Thank you so much sir for your support and help.
Ajay bro has excellent knowledge and his response is nice.
This is a great app I love it with good functionality.
Thank you so much for your help and support.
Highly recommended for the projects.
Loved the project work.
It is super for students.
I recently purchased this project online and I am very satisfied with it. The project was well-designed, properly organized, and included all the necessary files and documentation. The source code was clean, easy to understand, and worked as expected. A special appreciation for the project explanation and setup session. The session was detailed and easy to understand, covering both the project implementation and the complete setup process. All my questions were answered clearly, which helped me understand the project much better. Thanks for giving your time.