Closed-form solution

The one-dimensional Ridge regression problem admits a simple closed-form solution.
Given a dataset $(x_i,y_i)$ for $i=1,\dots ,n$, and a regularization parameter $\lambda >0$, the Ridge estimator is:

$${\hat{\beta }}_{\lambda }=\frac{{\sum }_{i=1}^n{x}_i{y}_i}{{\sum }_{i=1}^n{x}_i^2+n\lambda }$$

This form assumes that the data has no intercept term, i.e., the model is $y=\beta x$, or equivalently, that the data is centered around zero. In practice, it is common to subtract the means of both $x$ and $y$ before computing the estimator. This removes the intercept and gives:

$${\hat{\beta }}_{\lambda }=\frac{{\sum }_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{{\sum }_{i=1}^n(x_i-\bar{x}{)}^2+n\lambda }$$

We now implement this solution in Rust, using only the standard library.

#![allow(unused)]
fn main() {
pub fn ridge_estimator(x: &[f64], y: &[f64], lambda2: f64) -> f64 {
    let n: usize = x.len();
    assert_eq!(n, y.len(), "x and y must have the same length");

    let x_mean: f64 = x.iter().sum::<f64>() / n as f64;
    let y_mean: f64 = y.iter().sum::<f64>() / n as f64;

    let num: f64 = x
        .iter()
        .zip(y)
        .map(|(xi, yi)| (xi - x_mean) * (yi - y_mean))
        .sum::<f64>();

    let denom: f64 = x.iter().map(|xi| (xi - x_mean).powi(2)).sum::<f64>() + lambda2 * (n as f64);

    num / denom
}
}

You could also express it as a single iterator chain, similar to how we implemented the loss function earlier.