AI-Generated Text Detection Using Machine Learning with SHAP Explainability Python Final Year Project with Source Code

Abstract

Every college in India is currently having the same argument in the staff room. Students are submitting assignments that read a little too clean, and nobody can prove anything. That's the exact gap this project fills.

This is a CPU-only, fully explainable machine learning system that classifies a piece of text as AI-generated or human-written. Instead of throwing a giant transformer at the problem and calling it a day, it pulls 32 handcrafted linguistic features out of the text using spaCy and textdescriptives — readability scores, POS-tag ratios, dependency distances, lexical diversity, punctuation rates — and feeds those into three different models. A 1-D CNN built in TensorFlow/Keras, a Random Forest from scikit-learn, and a Calibrated Random Forest that wraps the RF in CalibratedClassifierCV with isotonic regression.

The interesting result, and the one your examiner will actually ask about, is that calibration drops log loss from roughly 0.28 to 0.26 while accuracy stays flat around 0.88. Meaning: the model's confidence numbers start telling the truth. A 90% prediction actually behaves like a 90% prediction. That's a real finding you can defend, not a vanity metric.

And every single prediction comes with a SHAP breakdown showing the top 5 features that pushed the decision one way or the other. No black box.

What This Project Actually Does

You open the Flask app in your browser at http://127.0.0.1:5001, paste in a chunk of text, and hit analyse. Two things come back.

First, a document-level verdict — AI-generated or human-written — with a confidence bar. Second, a sentence-by-sentence breakdown where each sentence gets colour-coded. Red means that sentence looks machine-written. Green means it reads human. This matters because real submissions are usually mixed. A student writes three paragraphs themselves and then panics and pastes in two from ChatGPT, and a document-level verdict alone would miss that completely.

Underneath, src/features.py runs the spaCy pipeline over your text, computes all 32 features, scales them with the saved scaler.joblib, and passes the vector to whichever model is loaded. The models load once at startup, so predictions come back fast instead of making you wait while TensorFlow warms up on every request.

The 32 Features It Looks At

These aren't vague "text statistics." They're grouped and documented, which makes the viva a lot easier:

• Readability (5) — Flesch Reading Ease, Flesch-Kincaid Grade, Gunning Fog, Coleman-Liau, ARI
• Descriptive statistics (5) — mean, median and standard deviation of sentence length, plus mean and std of token length
• Raw counts (2): total tokens and total sentences
• POS-tag ratios (9) — NOUN, VERB, ADJ, ADV, PRON, ADP, DET, CCONJ, PUNCT
• Syntactic complexity (3) — mean and std dependency distance, mean parse tree depth
• Lexical diversity: type-token ratio, unique word ratio, hapax legomenon ratio
• Punctuation rates (5) — commas, periods, question marks, exclamations, semicolons

Here's the thing about this feature set. LLMs write with unusually consistent sentence lengths and a slightly flatter punctuation profile than humans do. The standard deviation of sentence length is doing a lot of quiet work in this model, and SHAP will show you exactly that.

Key Features

• Paste a 500-word essay and get a verdict back in under two seconds on a normal laptop, no GPU anywhere in the pipeline
• Sentence-level colour coding that flags the specific lines that look machine-written, not just an overall score
• Top 5 SHAP features shown with every result, so the app explains why it decided what it decided
• Three models trained and compared side by side — CNN, Random Forest, Calibrated RF — with accuracy, precision, recall, F1, ROC-AUC and log loss for each
• A calibration reliability diagram saved to outputs/calibration_diagram.png that visually proves the confidence scores are honest
• Dark-themed four-page web app: home, analyse, about, contact — built with Flask and Jinja2 templates, no React, no npm, no build step to break
• Mobile responsive with a working hamburger nav, so screenshots for your report look decent on any screen
• Full visualisation set generated during training: confusion matrices, ROC curves, CNN training history, SHAP bar, beeswarm and waterfall plots — that's your entire results chapter, already plotted
• Jupyter notebook (train_notebook.ipynb) with inline plots if your guide wants to see the training run cell by cell

Real-World Applications

University submission portals are the obvious one. A department can run incoming assignments through this and flag suspicious ones for a human to review — and because it's explainable, the professor has something concrete to point at during the conversation with the student.

Beyond campus: newsrooms verifying freelance copy before publishing, hiring teams screening cover letters and SOPs, content agencies auditing what their writers actually deliver, and moderation teams on review sites trying to catch bulk-generated fake reviews. Any place where "did a person write this" is a question worth an answer.

What's Genuinely Hard (And What's Easier Than It Looks)

The hard part is feature extraction speed. Running spaCy's full pipeline across 1,332 texts takes several minutes on CPU, and if you try to train on a much bigger dataset without batching, you'll be staring at a terminal for a long time. Plan your training run before your deadline day, not during it.

The easy part surprises most people: there's no GPU requirement, no CUDA install, no Node build. You create a virtual environment, install from requirements.txt, download one 12 MB spaCy model, and run python app.py. The pre-trained models are already sitting in models/, so if you just want to demo the app first and understand the training later, you can. Fifteen minutes from download to working browser tab.

Who Should Buy This

If you're a student who needs a working project for your college submission, final-year project, semester project, internship, or academic demonstration, this is for you. Whether you're studying BCA, B.Tech, MCA, M.Tech, Computer Science, IT, AI, Data Science, or another related field, you can choose a project that fits your requirements. If you need the source code, project report, documentation, or help setting up and running the project, CodeAj gives you the resources and support to get started faster.

It fits especially well if your syllabus wants NLP plus machine learning plus a deployed interface in one submission. You get all three without having to stitch three tutorials together.

Why CodeAj

You get the complete codebase — app.py, all four Jinja templates, features.py, train.py, predict.py, the notebook, the saved .joblib and .h5 model files — plus a project report written in the format most Indian universities expect, with abstract, literature survey, methodology, results and conclusion already structured. If something breaks during setup, our team walks you through it instead of leaving you on a forum thread from 2019. Browse more AI final year projects if you want to compare options, check the wider machine learning projects with source code collection, or look at our plagiarism checker project if academic integrity is the theme your department wants.