Testing Standards and Coverage Requirements

This rule enforces comprehensive testing practices including unit test isolation, coverage thresholds, test tagging, and quality gates to ensure reliable, maintainable software.

Unit Test Isolation and Coverage

Description

Each business class or function MUST include deterministic unit tests covering at least 80% of its logic branches. The goal is to validate correctness at the unit level, independently from external systems.

Purpose

To catch regressions early, improve maintainability, and ensure every class's logic behaves as expected under varied inputs. This contributes to secure, reliable software that aligns with SDLC testing practices and auditability.

Scope

• Java modules with business logic (JUnit/TestNG)
• .NET services and ViewModels (xUnit/NUnit with FakeItEasy or Moq)
• Python modules (PyTest)
• TypeScript/JavaScript (Jest/Vitest)
• Applies to developers and QA engineers

SDLC Integration

• Planning: Test strategy includes unit coverage goals
• Analysis: Maps logic branches to test coverage metrics
• Design: Encourages modular, testable code
• Development: Enforces deterministic, isolated testing
• Testing: Evaluates line/branch coverage for compliance
• Deployment: Prevents merge if coverage fails threshold
• Maintenance: Flags regressions due to coverage loss

Coverage Thresholds and Requirements

New Code Standards

1. Coverage Target: New or modified code MUST maintain or increase module test coverage to ≥ 80%
2. Coverage Floor: Hard enforcement floor of 70% - code changes reducing coverage below this threshold MUST be blocked
3. Regression Prevention: Coverage MUST NOT decrease on any change
4. Test Tagging: Tests MUST be tagged (unit, integration, performance, security) for selective CI runs

Existing Code Standards

5. Bug Fix Tests: SHOULD add regression tests on every bug-fix commit
6. Flaky Test Management: MUST quarantine flaky tests (> 3% failure rate) within 24 hours and open a ticket
7. Coverage Growth: Overall coverage SHOULD trend upward quarter-on-quarter

Test Data Standards

• MUST use synthetic/generated data for all tests
• MUST NOT include real PII or PHI in test fixtures, logs, or error messages
• SHOULD use data builders or factories for test data generation

Unit Test Isolation Requirements

Core Principles

• Business classes MUST include deterministic unit tests
• Logic path coverage MUST be ≥ 80% for each class or function
• External dependencies MUST be mocked or stubbed
• Tests MUST be stable, fast, and repeatable
• Tests MUST be independent (no shared state between tests)

Mocking Requirements

• External APIs MUST be mocked (100% of usage)
• Database calls MUST be mocked in unit tests
• File system operations MUST be abstracted and mocked
• Time-dependent code MUST use injectable clock/time providers

Actionable Metrics

Metric	Target Value	Measurement Method	Enforcement Level
Module test coverage	≥ 80%	Jacoco (Java), coverlet (.NET)	MUST
Coverage floor	≥ 70%	CI test gate	MUST
Coverage regression	0% drop on change	CI test gate	MUST
External API mock coverage	100% of usage	Code review, mocking libs	MUST
Flaky test rate	≤ 3%	CI test results tracking	MUST
Test tagging compliance	100%	Test metadata review	MUST

Implementation

Configuration Requirements

Java (Maven/Gradle):

<!-- Maven Surefire -->
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <configuration>
        <rules>
            <rule>
                <element>PACKAGE</element>
                <limits>
                    <limit>
                        <counter>LINE</counter>
                        <value>COVEREDRATIO</value>
                        <minimum>0.80</minimum>
                    </limit>
                </limits>
            </rule>
        </rules>
    </configuration>
</plugin>

.NET (coverlet):

dotnet test --collect:"XPlat Code Coverage" /p:CoverageThreshold=80

Python (pytest-cov):

pytest --cov=myapp --cov-fail-under=80

Test Naming Conventions

Format: MethodName_Scenario_ExpectedResult

Examples:

• GetUserAsync_ValidId_ReturnsUser
• SaveAsync_NullDto_ThrowsArgumentNullException
• Calculate_NegativeAmount_ThrowsArgumentException

Example: Unit Test with Mocking (C#/.NET)

public class TaxCalculatorTests
{
    private readonly IRatesApi _ratesApi = A.Fake<IRatesApi>();
    private readonly TaxCalculator _calculator;

    public TaxCalculatorTests()
    {
        _calculator = new TaxCalculator(_ratesApi);
    }

    [Fact]
    public void Calculate_ValidAmount_ReturnsTax()
    {
        // Arrange
        A.CallTo(() => _ratesApi.GetRate("CA")).Returns(0.12m);

        // Act
        var result = _calculator.Calculate(100.0m, "CA");

        // Assert
        Assert.Equal(12.0m, result);
    }

    [Theory]
    [InlineData(0)]
    [InlineData(-100)]
    public void Calculate_InvalidAmount_ThrowsArgumentException(decimal amount)
    {
        // Act & Assert
        Assert.Throws<ArgumentException>(() => _calculator.Calculate(amount, "CA"));
    }
}

Example: Unit Test with Mocking (Java)

@Test
public void shouldCalculateTax() {
    // Arrange
    IRatesApi ratesApiMock = mock(IRatesApi.class);
    when(ratesApiMock.getRate("CA")).thenReturn(0.12);
    TaxCalculator calc = new TaxCalculator(ratesApiMock);
    
    // Act
    double result = calc.calculate(100.0, "CA");
    
    // Assert
    assertEquals(12.0, result, 0.01);
}

Example: Test Tagging

C#/.NET (xUnit):

[Trait("Category", "unit")]
[Fact]
public void GetUser_ValidId_ReturnsUser() { }

[Trait("Category", "integration")]
[Fact]
public void SaveUser_ValidUser_PersistsToDatabase() { }

Java (JUnit 5):

@Tag("unit")
@Test
public void shouldReturnUser() { }

@Tag("integration")
@Test
public void shouldPersistToDatabase() { }

Example: Synthetic Test Data

public class TestDataBuilder
{
    public static User CreateTestUser(string id = null)
    {
        return new User
        {
            Id = id ?? Guid.NewGuid().ToString(),
            Name = $"Test User {Random.Shared.Next(1000)}",
            Email = $"test{Random.Shared.Next(1000)}@example.com",
            CreatedAt = DateTime.UtcNow
        };
    }
}

[Fact]
public void ProcessUser_ValidUser_Success()
{
    // Arrange
    var user = TestDataBuilder.CreateTestUser();
    
    // Act & Assert
    // ...
}

Test Organization

Test Structure (AAA Pattern)

All tests MUST follow the Arrange-Act-Assert pattern:

[Fact]
public void MethodName_Scenario_ExpectedResult()
{
    // Arrange - Set up test data and dependencies
    var dependency = A.Fake<IDependency>();
    var sut = new SystemUnderTest(dependency);
    
    // Act - Execute the method being tested
    var result = sut.MethodUnderTest(input);
    
    // Assert - Verify the expected outcome
    Assert.Equal(expectedValue, result);
}

Test File Organization

• Place test files in parallel directory structure to source files
• Use clear test file naming: {ClassUnderTest}Tests.cs or {ClassUnderTest}Test.java
• Group related tests in nested classes when appropriate

Flaky Test Management

Identification

A test is considered flaky if it:

• Fails > 3% of runs without code changes
• Produces non-deterministic results
• Depends on timing, external resources, or shared state

Response Protocol

1. Quarantine: Mark test with [Trait("Category", "quarantined")] or @Tag("quarantined")
2. Track: Open ticket with failure rate and investigation details
3. Fix or Remove: Within 24 hours, either fix the root cause or remove the test
4. Document: If test must remain quarantined, document why and remediation plan

CI Integration

Pre-Merge Gates

All PRs MUST pass these gates:

• Unit tests pass with ≥ 80% coverage
• No coverage regression from base branch
• No flaky tests introduced
• All tests properly tagged

Coverage Reporting

• Generate coverage reports in CI
• Block merge if coverage falls below threshold
• Display coverage trends over time
• Alert team on significant coverage drops