Regression in Python: Fit Models with NumPy

To estimate a numeric target from input values, regression gives you a way to model how those values relate to the target. NumPy’s least-squares routine finds coefficients from arrays, with each coefficient describing how an input value changes the prediction.

I’ll show how to fit a regression model with NumPy and evaluate its predictions, including why R-squared should be read alongside an error measure.

TL;DR: Fit regression with NumPy

Use np.linalg.lstsq() to fit coefficients without explicitly inverting a matrix, then evaluate predictions on data the fit did not use.

• Keep one row per observation and one column per feature.
• Add a column of ones when the model needs an intercept.
• Read R-squared with an error metric such as RMSE, not alone.

What is linear regression in Python?

Linear regression estimates a numeric target from one or more input features. Each fitted coefficient describes the model’s change in prediction when its feature changes by one unit, holding the other features fixed.

NumPy’s least-squares routine solves for the coefficients and reports matrix diagnostics.

y_hat = intercept + slope * x

A design matrix stores observations in rows and model terms in columns. The target vector holds the observed values the model tries to approximate.

Use NumPy directly when the matrix operations are part of the task or you need a compact calculation. For a reusable estimator workflow with fit and predict methods, scikit-learn’s LinearRegression provides that interface.

How to fit and evaluate a NumPy regression model

The fitting example uses NumPy arrays directly. I ran the named Python file and captured its output alongside the code.

Step 1: Prepare the feature and target arrays

The sample arrays contain four paired observations. Split actual data into training and evaluation portions before measuring predictive performance.

import numpy as np

x = np.array([1.0, 2.0, 3.0, 4.0])
y = np.array([3.1, 5.0, 7.2, 8.9])

Step 2: Build the design matrix

The ones column gives the fit a free intercept. The second column carries the measured feature. With multiple features, add one column for each feature while preserving one row per observation.

A = np.column_stack((np.ones_like(x), x))

Step 3: Solve for the coefficients

Pass the design matrix and target to np.linalg.lstsq(). The first returned value is the coefficient vector. The rcond=None argument uses NumPy’s default cutoff for small singular values.

intercept, slope = np.linalg.lstsq(A, y, rcond=None)[0]

Step 4: Predict and calculate error

Calculate predictions with the fitted parameters. RMSE is the square root of the mean squared residual, so it uses the same units as the target. R-squared compares the residual sum of squares with the total variation around the target mean.

predicted = intercept + slope * x
residuals = y - predicted
ss_res = np.sum(residuals**2)
ss_tot = np.sum((y - y.mean())**2)
r_squared = 1 - ss_res / ss_tot
rmse = np.sqrt(np.mean(residuals**2))

The program’s output is:

intercept: 1.150
slope: 1.960
R-squared: 0.998

The output gives a slope near two and an intercept near one for this synthetic sample. R-squared is close to one on those same four points, but the sample is too small to support a claim about new observations. The script prints its displayed results and also computes RMSE for further inspection.

What should you check before trusting a fit?

Check the input values, dimensions, feature relationships and evaluation method before interpreting coefficients. A successful solve does not establish that the model is suitable for prediction.

Some common cases change how a fit should be read. Use the table to decide what to inspect before trusting a coefficient or score.

There is no universal threshold that makes these conditions harmless. Review them in the context of the measurement process and the decision the prediction supports.

Check shapes and non-finite values

The design matrix has shape (observations, features) and the target has one value per observation. A transposed matrix may treat features as samples. Check that the number of rows matches the target length and that both arrays contain finite numbers before calling the solver.

if A.shape[0] != y.shape[0]:
    raise ValueError("A and y must have the same number of observations")
if not np.isfinite(A).all() or not np.isfinite(y).all():
    raise ValueError("A and y must contain only finite values")

Missing values require an explicit data-handling decision, such as filtering aligned rows or imputing from training data. Do not delete missing feature values without removing the corresponding target values.

Inspect dependence and the intercept

NumPy returns rank and singular values with the solution. Inspect both if the fitted coefficients look unstable. A constant feature duplicates the intercept column.

Centering changes the feature reference point and therefore the intercept’s meaning. Without an intercept column, the model predicts zero when all features are zero, so use that form only when the model and data justify the constraint.

Read the metrics in context

R-squared can be negative on held-out data when predictions are worse than using the target mean. A high training score can coexist with weak performance on new observations. Compare against a simple baseline and calculate metrics on data that was not used to fit the model.

RMSE emphasizes larger errors because residuals are squared before averaging. Inspect residuals as well: a visible curve or changing spread can indicate that a straight-line model misses structure or that error size changes across the range. Neither metric diagnoses every assumption by itself.

Avoid unsupported extrapolation

A fitted line extends mathematically beyond the observed feature values, but that does not mean the relationship remains valid there. Check the range represented in the training observations and treat predictions outside it as extrapolations that need domain justification.

These decisions matter because regression output depends on how the matrix represents the data. A coefficient is conditional on the other columns in the model, and changing units changes the coefficient’s numeric scale. If a feature is measured in thousands instead of single units, predictions can stay the same while that coefficient becomes one thousand times smaller.

For multiple regression, construct a two-dimensional array even when there is only one feature. A one-dimensional x array is convenient for the formula in the small example, but the matrix solve expects one row per observation and one column per modeled term. A shape check before fitting catches many accidental transposes.

When observations arrive in batches, combine or summarize them according to the model you intend to fit. Repeating the same full least-squares solve inside a Python loop is not the same as vectorizing one fit, and it does not become a faster algorithm merely because the arrays use NumPy. Measure a representative workload before making a performance choice.

For a single fitted model, NumPy’s linear algebra routines use optimized low-level libraries where available, but the actual runtime depends on matrix dimensions, data movement and the linked implementation. The NumPy documentation describes its linear algebra backends as BLAS and LAPACK. That is a technical implementation detail, not a guarantee that every workload benefits from a larger matrix or from running many repeated fits.

Use a least-squares solve for ordinary linear regression. Choose a higher-level estimator when the workflow needs fit and predict methods, pipelines or cross-validation. Scikit-learn provides an intercept option, so choose the interface based on the surrounding program rather than an assumed speed advantage.

Conclusion: Use the right regression workflow

NumPy’s least-squares solver is a direct way to fit linear coefficients from arrays. Build the matrix deliberately, validate inputs, and evaluate predictions on held-out data instead of relying on a training score alone.

For the estimator interface, see the scikit-learn LinearRegression documentation. AskPython’s guides to simple linear regression and linear regression from scratch cover adjacent approaches.

FAQ

These answers cover common choices when fitting a regression model with Python.