Coverage-Guided Fairness Testing

Daniel Perez Morales, Takashi Kitamura, Shingo Takada

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Software testing is a crucial task. Unlike conventional software, AI software that uses decision-making algorithms or classifiers needs to be tested for discrimination or bias. Such bias can cause discrimination towards certain individuals based on their protected attributes, such as race, gender or nationality. It is a major concern to have discrimination as an unintended behavior. Previous work tested for discrimination randomly, which has resulted in variations in the results for each test execution. These varying results indicate that, for each test execution, there is discrimination that is not found. Even though it is nearly impossible to find all discrimination unless we check all possible combinations in the system, it is important to detect as much discrimination as possible. We thus propose Coverage-Guided Fairness Testing (CGFT). CGFT leverages combinatorial testing to generate an evenly-distributed test suite. We evaluated CGFT with two different datasets, creating three models with each. The results show an improvement in the number of unfairness found using CGFT compared to previous work.

Original languageEnglish
Title of host publicationComputer and Information Science, 2021
EditorsRoger Lee
PublisherSpringer Science and Business Media Deutschland GmbH
Pages183-199
Number of pages17
ISBN (Print)9783030794736
DOIs
Publication statusPublished - 2021
Event20th IEEE/ACIS International Summer Semi-Virtual Conference on Computer and Information Science, ICIS 2021 - Shanghai, China
Duration: 2021 Jun 232021 Jun 25

Publication series

NameStudies in Computational Intelligence
Volume985
ISSN (Print)1860-949X
ISSN (Electronic)1860-9503

Conference

Conference20th IEEE/ACIS International Summer Semi-Virtual Conference on Computer and Information Science, ICIS 2021
Country/TerritoryChina
CityShanghai
Period21/6/2321/6/25

Keywords

  • Combinatorial testing
  • Fairness
  • Machine learning
  • Testing

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Coverage-Guided Fairness Testing'. Together they form a unique fingerprint.

Cite this