The first time I looked at our confusion matrix, I thought we had a great model.
96% accuracy. Clean numbers. My manager nodded. I felt good.
Then a senior data scientist on the team asked me one question: “What’s the base rate of fraud in the dataset?”
About 0.4%.
She didn’t say anything else. She didn’t need to. A model that predicted “not fraud” for every single transaction would have been 99.6% accurate — and completely useless.
That was week three at Accenture. I had a lot more to learn.
What the Project Actually Was
I joined Accenture as a Machine Learning Engineer in mid-2021, working on a fraud detection and risk scoring platform for a large BFSI client. Banking, financial services, insurance — an industry where the cost of getting something wrong isn’t abstract. It’s real money, real customers, real consequences.
Our task: analyze over a million transaction records and build a system that could identify fraudulent activity, flag anomalies, and generate risk scores for review.
On paper, it sounded like a classic ML project. Train a classifier, evaluate on a test set, deploy.
In reality, it was one of the most humbling experiences of my career.
The Data Was the Actual Job
Before I wrote a single line of model code, I spent weeks just understanding the data.
Fraud data has a problem that most textbook ML datasets don’t: it’s almost entirely negative. Fraud is rare by design — bad actors try hard not to look like bad actors. In our dataset, fraudulent transactions were a tiny fraction of the total. If your model learns to just say “not fraud” every time, it will score well on accuracy and fail completely at the one thing it’s supposed to do.
So we got to work on the feature engineering side — and this is where most of the real value was created.
We built over 40 features: transaction frequency patterns, amount deviation from a customer’s historical baseline, merchant category mismatches, device fingerprints, geographic anomalies, account age, payment channel behavior, and historical fraud indicators. Each of these was a hypothesis about what a fraudulent transaction looks like, encoded into something a model could learn from.
Then came the imbalance problem. We used SMOTE to generate synthetic minority-class samples, combined with undersampling and class-weight tuning. That combination improved our fraud recall by nearly 20% — meaning we caught significantly more actual fraud — while keeping false positives at a level the client’s review team could actually handle.
That balance matters more than people realize. Flag too much and your human reviewers drown. Flag too little and the fraud slips through. The model isn’t making that decision alone — it’s making it in partnership with the people downstream.
The Model Wasn’t the Hard Part
We landed on XGBoost for the core classifier. It handled the tabular data well, gave us interpretable feature importances, and responded well to hyperparameter tuning. After cross-validation and threshold optimization, we were sitting at roughly 0.86 ROC-AUC — an 18–22% improvement over the baseline the client had been using.
We also layered in Isolation Forest for anomaly detection. The XGBoost model was trained on historical fraud patterns — it was good at catching things that looked like fraud it had seen before. Isolation Forest was good at catching things that just looked weird, regardless of whether they matched a known pattern. Together they covered more surface area.
But here’s what I actually learned from all of this: the model performance metrics were almost never the most important conversation.
The most important conversations were about thresholds.
At what score do you flag a transaction for human review? At what score do you block it automatically? What’s the cost of a false positive to a legitimate customer who gets their card declined? What’s the cost of a false negative to the client when a fraudulent transaction goes through?
These aren’t ML questions. They’re business questions. And the answers changed depending on who you asked — the risk team, the product team, the compliance team, the executive sponsor. Part of my job was translating between what the model could do and what those conversations actually required.
The Lesson That Has Stayed With Me
I came into Accenture thinking the job was to build a good model.
I left understanding that the job was to build a good decision system — one where the model was one component, the features were another, the threshold logic was another, the human review process was another, and the monitoring pipeline that caught when the distribution shifted was another.
A model that performs well in evaluation and degrades quietly in production without anyone noticing isn’t a success. It’s a slow failure.
We implemented monitoring and batch scoring automation partly to speed up the review workflow, but also to give the team a way to see when something was changing — when fraud patterns were evolving, when a new attack vector was emerging, when the model’s assumptions were starting to drift from reality.
That’s the part of ML that doesn’t make it into blog posts about model architectures. It’s less exciting than a good ROC curve. But it’s the difference between a project and a product.
What I’d Tell a Junior ML Engineer Starting Out
Fall in love with the features, not the model. Anyone can throw XGBoost at a dataset. The people who create real value understand the domain well enough to know which signals matter and why.
Learn to think in thresholds, not accuracy. Almost every real ML decision involves a tradeoff between precision and recall, between false positives and false negatives. Understand the asymmetric costs in your domain before you touch a single hyperparameter.
The model is not the system. Data pipelines, feature stores, monitoring, human-in-the-loop workflows — these are not afterthoughts. They are the product.
Ask the dumb question. The confusion matrix lesson in week three came because someone asked a question I should have asked myself. In every project since, I’ve tried to be the person who asks it first.
Fraud detection taught me that machine learning in the real world is less about elegance and more about understanding what happens when you’re wrong — and building something resilient enough to handle it.
I’m still building on that foundation.