Cramér's V and Chi-Square Test

Overview

The Chi-Square test in Q-API provides a statistically sound screening procedure to evaluate the relationship between features (predictors or indicators) and a target variable. This is particularly useful when dealing with a large set of potential predictors, as it helps identify which features have the strongest statistical relationship with the target before building predictive models.

Unlike traditional correlation analysis that only captures linear relationships, the Chi-Square test can detect nonlinear associations through contingency tables—simultaneous partitioning of features and targets into categories—measuring the degree to which membership in feature categories relates to membership in target categories.

Cramér's V is a normalized measure utilizing Chi-Square Test. It ranges from 0 to 1, providing interpretable effect sizes for feature importance:

V = √(χ² / (n × min(k-1, r-1)))

Key Concepts

Contingency Tables

A contingency table shows the distribution of cases across combinations of feature and target categories. By analyzing this distribution, we can determine whether the categorization of features is related to the categorization of targets beyond what would be expected by random chance.

Statistical Measures

The Chi-Square test provides several related statistical measures:

1. Chi-Square Value: Measures the overall deviation between observed and expected frequencies
2. Contingency Coefficient: A nominal correlation coefficient (maximum value is less than one)
3. Cramér's V: An improved nominal correlation coefficient ranging from 0 (no relationship) to 1 (perfect relationship)
4. p-value: Indicates the statistical significance of the relationship

Cramér's V is generally considered the most useful measure as it:

• Has a standardized range (0 to 1)
• Is easily interpretable
• Handles tables of different dimensions
• Performs better when features contain numerous ties

Configuration Options

The Chi-Square test in Q-API can be configured using several parameters:

Parameter	Description
featureGrouping	Determines how features are grouped: "bins" (equal frequency bins) or "tails" (most extreme values)
featureBinsOrTails	For "bins", specifies the number of bins (e.g., 3). For "tails", specifies the fraction ranging from 0.0 to 0.5 (e.g., 0.1 for top and bottom 10%)
targetGrouping	Determines how the target is grouped: "bins" (equal bins) or "sign" (positive/negative return)
targetBins	When targetGrouping="bins", specifies the number of bins for the target

Grouping Approaches

Binning Method

Using equal-frequency bins divides the data points into a specified number of groups with approximately equal counts:

featureGrouping=bins, featureBinsOrTails=3, targetGrouping=bins, targetBins=3

This configuration splits each feature and the target into three equal-sized bins.

Tails Method

The tails approach focuses only on the most extreme values, which often contain the most information:

featureGrouping=tails, featureBinsOrTails=0.1, targetGrouping=bins, targetBins=3

This configuration selects the most extreme 10% of values from each tail of the feature distribution.

Sign based Target Grouping

The target can also be grouped based on the sign (profit or loss) of the returns:

featureGrouping=tails, featureBinsOrTails=0.1, targetGrouping=sign

Statistical Considerations

Bin Size and Reliability

Larger numbers of bins increase the test's sensitivity to subtle relationships but can reduce reliability when bin counts are small. As a general guideline:

• Most cells should have an expected frequency of at least five
• No cell should have an expected frequency less than one

Multiple Testing

The p-values provided don't account for multiple comparisons when testing many features simultaneously. The primary value of this test is the relative ordering of features by Cramér's V rather than the absolute significance levels.

API Endpoint

POST /statistical-tests/v1/cramers-v

Request Format

The test requires a CSV containing the data and query parameters specifying how to group features and targets:

POST /statistical-tests/v1/cramers-v?featureGrouping=bins&featureBinsOrTails=3&targetGrouping=bins&targetBins=3

With a request body containing CSV data:

DateTime,Symbol,StrategyNAV,Feature1,Feature2,Feature3
2024-01-02,STRAT1,9999.9,0.9993,0.1113,0.210000
2024-01-03,STRAT1,9991.5,0.9965,0.9991,0.210000
2024-01-04,STRAT1,9708.0,0.9945,0.9926,0.19991
2024-01-05,STRAT1,9938.9,0.9945,0.10021,0.210031
2024-01-08,STRAT1,9963.9,0.9945,0.10276,0.210350
2024-01-09,STRAT1,9736.4,0.9945,0.10371,0.210555
2024-01-10,STRAT1,9963.9,0.9945,0.10473,0.210751
...

Response Format

The response is a JSON object containing Cramér's V statistics for each feature, sorted by strength of relationship:

{
  "Features": {
    "Feature2": {
      "CramersV": 1,
      "ChiSquare": 2,
      "Contingency": 0.707,
      "pValue": 0.3679,
      "LowerThreshold": 0.1002,
      "UpperThreshold": 0.9991
    },
    "Feature1": {
      "CramersV": 0.612,
      "ChiSquare": 1.88,
      "Contingency": 0.522,
      "pValue": 0.3916,
      "LowerThreshold": 0.9945,
      "UpperThreshold": 0.9993
    },
    "Feature3": {
      "CramersV": 0,
      "ChiSquare": 0,
      "Contingency": 0,
      "pValue": 1,
      "LowerThreshold": 0.1999,
      "UpperThreshold": 0.2106
    }
  }
}

Practical Application

The Chi-Square test is particularly useful as a pre-processing step in feature selection for predictive modeling. By identifying features with the strongest relationship to the target variable, you can:

1. Reduce a large set of potential features to a more manageable number
2. Discover nonlinear relationships that might be missed by linear correlation
3. Identify feature combinations that have significant predictive power:
   • 0.0 to 0.2: Weak relationship
   • 0.2 to 0.4: Moderate relationship
   • 0.4 to 1.0: Strong relationship

When building models with SETS or other predictive approaches, running the Chi-Square test first can significantly improve model quality by ensuring you focus on the most relevant predictors.

Usage Example

Screening Features for a Trading Model

Consider a scenario where you have 100 potential technical indicators as features, and you want to determine which are most relevant to predicting next-day returns:

1. Run the Chi-Square test with the tails approach:

   POST /statistical-tests/v1/cramers-v?featureGrouping=tails&featureBinsOrTails=0.15&targetGrouping=bins&targetBins=2

2. Select the top features based on Cramer's V values (e.g. beyond 0.3 or 0.5)

3. Use these features as inputs to the SETS model, either directly or through the ModelToFeaturesMapping parameter

This workflow ensures your models focus on features with demonstrated statistical relationships to the target, improving both computational efficiency and model quality.