Every strong machine learning project starts before the first line of code. It starts with a question. That’s exactly what problem framing and data collection are about, and it’s exactly what this article covers.
This post is the companion guide for EP105 of the Intelevo YouTube series. If you’d rather watch the full walkthrough with live code, check out the video first. Then come back here to review the concepts at your own pace.
In EP104, we let AutoML search for the best model automatically. That saved hours of manual tuning. But AutoML only works once someone hands it a clean target and a ready dataset. Nobody has automated that decision yet. It still starts with you. So today, we slow down and cover the step that comes before any model: problem framing and data collection.
Why Problem Framing and Data Collection Come First
Picture two teams starting the same project. Team A jumps straight into the data. They pick a column that looks like a good target, load a dataset, and start training models. Team B does something different first. They sit down and agree on the exact question they’re answering. Only then do they touch any data.
Team B almost always wins. Here’s why. A model can only be as good as the question behind it. If the target is wrong, or the data can’t actually answer the question, no algorithm can fix that later. That’s the core idea behind problem framing and data collection: get the question and the data right, and everything downstream gets easier.
The Road Trip Analogy
Think about planning a road trip. You wouldn’t start the car and figure out the destination along the way. Instead, you’d decide where you’re going first. Next, you’d check the route. Finally, you’d pack for the trip. All of this happens before the engine even starts.
A data science project works the same way. Deciding the destination is your business goal. It’s the real-world decision this project should change. Mapping the route means translating that goal into a specific, measurable machine learning question. And packing what you need means gathering the data that can honestly answer that question.
This analogy makes the whole process click. Once you see a project as a trip you’re planning, problem framing and data collection stop feeling abstract. They become practical steps you take before you ever open a notebook.
What Problem Framing Actually Means
Problem framing is the practice of turning an open-ended business goal into a precise, testable question. A well-framed problem has three ingredients: a named target, a named metric, and named limits.
Skip this step, and even a perfect AutoML search ends up aimed at the wrong thing. That’s a wasted afternoon at best, and a wasted project at worst.
Here’s the chain that turns a goal into a task:
Business Question → ML Question → Target Variable → Success Metric
Let’s walk through a real example. Start with the business question: “reduce customer churn.” On its own, that’s too vague for a model to use. So, translate it into an ML question: “predict which customers will cancel in the next 30 days.” Now you have a target variable, will_cancel, and you can choose a metric.
In this case, recall on the cancel class makes more sense than plain accuracy. Why? Because missing a customer who’s about to leave costs far more than a false alarm. That single choice, made before any modeling starts, shapes everything that follows.
The Two Halves of Starting a Project Right
A capstone kickoff isn’t one task. It’s actually two tasks, and the order matters.
First, problem framing. Decide which decision you’re supporting, what the model should predict, and how you’ll know if it worked.
Second, data collection. Once the question is fixed, gather the records that can honestly answer it, and check what shape they’re in.
Notice that framing comes first. That’s not an arbitrary choice. Framing decides what data even counts as useful. If you collect data before you know your target, you often end up with a folder full of files and no clear way to use them. So, always frame first, and collect second.
Five Steps From Business Goal to ML Task
Here’s a concrete process you can follow for your own capstone project.
- State the goal. Write the business objective in one plain sentence. No jargon allowed.
- Pick the target. Name the exact column a model would need to predict.
- Choose the metric. Decide what “good” looks like before you see any results. This keeps you honest later.
- Set the scope. Note who, what time range, and what’s explicitly out of bounds.
- Write it down. One paragraph, shared with your team, before any coding begins.
This process isn’t a one-and-done checklist, though. It’s actually a loop. You state a goal, then check whether data exists to support it. If it doesn’t, you adjust the goal until both agree.
How long does this take? It depends. A quick capstone with a familiar dataset might take a few minutes. A well-scoped problem with a mentor typically takes about an hour. And in real companies, this step often takes a full day or more, simply because it involves waiting on stakeholders to align. Either way, that time is well spent.
Why Skipping Framing Costs You Later
Let’s compare two paths side by side, because the difference is stark.
Without framing: Whoever is coding picks a target because it “looks right.” Success gets judged after the fact, often by gut feeling. Scope silently expands as new edge cases appear. Rework gets discovered weeks in, after modeling has already started.
With framing: The target is written down and agreed on by the team. A metric is chosen before any model gets trained. Who, what, and when are fixed in advance. As a result, disagreements get caught in a conversation, not in a re-run of your entire pipeline.
In short, a few minutes of framing prevents hours, or even weeks, of rework. That trade-off alone makes problem framing and data collection worth the upfront investment.
Where Real Project Data Actually Comes From
Once your question is framed, it’s time to collect data. But where does real project data actually come from? Here are six common sources.
- Internal databases. Records your organization already keeps, such as sales, users, or transactions.
- Public datasets. Sources like Kaggle, UCI, and government portals. These are ready-made and well-documented.
- APIs and logs. Live feeds from apps, sensors, or third-party services.
- Forms and surveys. Data you design and collect yourself, built for exactly this question.
- Web scraping. Public web pages, gathered carefully and within their terms of use.
- Vendor data. Purchased or licensed datasets that fill a specific gap in what you already have.
For most capstone projects, an internal-style dataset works best. It mirrors what you’ll encounter in a real job, and it forces you to practice cleaning messy, real-world data.
Three Checks Before You Trust Any Dataset
Before any dataset earns your trust, run these three checks.
Missing values. How much data is missing? And just as important, is it missing randomly, or for a telling reason? Missing data that follows a pattern often carries information of its own.
Class balance. Is the outcome you care about rare? If so, that changes how you’ll measure success. A rare target needs a different evaluation strategy than a balanced one.
Leakage risk. Does any column quietly reveal the answer early? Leakage happens when a feature contains information you won’t actually have at prediction time. It’s one of the sneakiest bugs in machine learning, because your model looks great in testing, and then fails in production.
Here’s a useful rule of thumb: if your class split is more lopsided than roughly 90/10, plain accuracy will lie to you. A model that always predicts the majority class can score 90% accuracy and still be completely useless. Keep that in mind before you celebrate a high accuracy number.
Hands-On: Loading the Data and Checking Its Quality
Let’s get practical. Here’s how to load a raw dataset in Python and immediately size up its quality.
import pandas as pd
df = pd.read_csv("customers_raw.csv")
print(df.shape)
print(df.info())
# how much is missing, column by column
print(df.isnull().mean().sort_values(ascending=False))
# is the target we care about rare?
print(df['churned'].value_counts(normalize=True))
Let’s break this down. First, we import pandas and read the CSV into a DataFrame called df. Next, df.shape tells us how many rows and columns we’re working with, at a glance. Then, df.info() shows the data type of every column, plus how many non-null values each one has.
After that, we compute the fraction of missing values per column with isnull().mean(), sorted from worst to best. This single line handles our missing-values check. Finally, value_counts on the churned column, with normalize=True, shows us the class balance as proportions. That’s our second check, done in one line of code.
Hands-On: Separating the Target From the Features
Once you’ve named your target, splitting it out from the rest of the data takes just one line of code. This step also locks in your framing decision, directly inside your codebase.
target = 'churned'
# everything except the target is a feature
X = df.drop(columns=[target])
y = df[target]
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42
)
print(f"Train: {X_train.shape} Test: {X_test.shape}")
Here’s what’s happening. We store the target column name in a variable, so it’s named once and reused everywhere. Then, X becomes every column except the target, and y becomes the target itself.
Next, we bring in train_test_split from scikit-learn to hold back 20% of the data for testing. Notice the stratify=y argument. This keeps the same class balance in both the training and test sets, which matters a lot when your target is rare. Finally, random_state=42 makes the split reproducible, so you get the same train and test rows every time you rerun the code.
Three Mistakes First-Timers Make
Before wrapping up, let’s cover three traps to avoid in your own capstone project.
Skipping the business conversation. Jumping straight into data before anyone agrees on what “success” actually means sets your project up for confusion later.
Trusting the leaderboard blindly. Picking a target because it’s easy for a model to predict, rather than because it actually matters, leads to technically impressive but practically useless results.
Treating data as fully hands-off. Assuming a public or internal dataset is automatically clean, complete, and unbiased, just because someone else collected it, is a risky assumption. Always verify before you trust.
The good news? Every one of these traps is avoidable. Simply slowing down at the framing stage solves all three.
Quick Recap
Let’s tie everything together. Problem framing turns a vague goal into a named target, metric, and scope, before any code runs. A business question becomes an ML task through a simple chain: goal, question, target, metric.
Real project data comes from databases, public sources, APIs, forms, scraping, and vendors. Missing values, class balance, and leakage risk are the three checks worth running first. And most importantly, framing replaces guesswork about the target with a decision your whole team has agreed on.
What’s Next: Modeling and Evaluation
You’ve now framed the question, sized up the data, and split it correctly. That’s the hard part done. In EP106, Capstone Part 2: Modeling and Evaluation, we’ll put all of this groundwork to work. We’ll train models on this same capstone dataset, and judge them using the metric we chose today, not whichever one happens to be easiest to compute.
If this article helped you understand problem framing and data collection, the full video walkthrough is even more useful, since it includes live coding and visual explanations for every step. Watch EP105 on the Intelevo YouTube channel, and don’t forget to subscribe so you catch EP106 as soon as it’s live.
Got questions about framing your own capstone problem? Drop them in the comments on YouTube. I’d love to help you think it through.