Skip to content

Fairness metrics

Fairness Metrics

FairnessMetric

Base class for fairness metrics.

Parameters:

  • name (str) –

    Metric name.

  • rho (float, default: 0.0 ) –

    Smoothing/temperature parameter (used by some metrics).

fun(w, dset, train_test) -> ArrayLike

Compute the metric value.

Parameters:

  • w (ArrayLike) –

    Parameter vector.

  • dset –

    Dataset.

  • train_test ((train, test), default: "train" ) –

    Split to use.

Returns:

  • ArrayLike –

    Scalar metric value.

Raises:

  • NotImplementedError –

    If not implemented.

grad(w, dset, train_test='train') -> jnp.ndarray

Gradient of :meth:fun w.r.t. w.

Parameters:

  • w (ArrayLike) –

    Parameter vector.

  • dset –

    Dataset.

  • train_test ((train, test), default: "train" ) –

    Split to use.

Returns:

  • ndarray –

    Gradient array with the same shape as w.

set_model(model=None)

Attach a model to the metric.

Parameters:

  • model (object, default: None ) –

    Model instance (required by metrics that call model methods).

Returns:

DemographicParity

Bases: FairnessMetric

Demographic parity score across groups.

Demographic parity is also known as statistical parity, group fairness, independence, or disparate impact.

DisparateMistreatment

Bases: FairnessMetric

Disparate mistreatment score based on squared covariance between group index and prediction.

EqualOpportunity

Bases: FairnessMetric

Equal opportunity score based on differences in true positive rate (TPR) across groups.

EqualizedOdds

Bases: FairnessMetric

Equalized odds score based on differences in TPR and FPR across groups.

GroupVariance

Bases: FairnessMetric

Variance of per-group losses.

Requires a model that implements _group_loss(dset, train_test, w_full).

HSIC

Bases: FairnessMetric

HSIC-like dependence score between group index and predictions.

IndividualFairness

Bases: FairnessMetric

Individual fairness score based on cross-group pairwise prediction differences.