To unify or not to unify: A case study on unified builds (in WebKit)

Takafumi Kubota, Yusuke Suzuki, Kenji Kono

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Unified builds are a simple but effective technique to reduce the build time of large software projects. Unified builds generate large compiler tasks by bundling multiple source files into one, resulting in a significant reduction in build time through removal of redundant work incurred by shared headers. However, unified builds have a negative effect on incremental builds because each compiler task gets larger. An ad-hoc unification strategy causes an excessive slowdown in incremental builds. A rough report from WebKit says the worst slowdown is 20% (6s → 7s), but our investigation shows it is as high as 479% (19s → 110s). In this paper, we investigate the characteristics of unified builds to find a sweet spot, which generates compiler tasks that reduce the full build time without increasing the incremental build time. An in-depth analysis of WebKit reveals 1) source files with higher header similarity should be unified, 2) source files that have significant differences in compile times should not be unified, and 3) source files that are not frontend-intensive should not be unified. Our case study shows the total build time is reduced by 2.66%, and the worst slowdown falls from 479% to 129%. These findings will be of help in deriving a more intelligent strategy of unification and give a basis for discussions on future build systems, compilers, and module systems that cooperatively generate efficient compiler tasks.

Original languageEnglish
Title of host publicationCC 2019 - Proceedings of the 28th International Conference on Compiler Construction
EditorsJ. Nelson Amaral, Milind Kulkarni
PublisherAssociation for Computing Machinery
Pages42-52
Number of pages11
ISBN (Electronic)9781450362771
DOIs
Publication statusPublished - 2019 Feb 16
Event28th International Conference on Compiler Construction, CC 2019 - Washington, United States
Duration: 2019 Feb 162019 Feb 17

Publication series

NameACM International Conference Proceeding Series

Conference

Conference28th International Conference on Compiler Construction, CC 2019
CountryUnited States
CityWashington
Period19/2/1619/2/17

Keywords

  • Build system
  • Compile time analysis
  • Incremental builds
  • Source code characterization
  • Unified builds

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Kubota, T., Suzuki, Y., & Kono, K. (2019). To unify or not to unify: A case study on unified builds (in WebKit). In J. N. Amaral, & M. Kulkarni (Eds.), CC 2019 - Proceedings of the 28th International Conference on Compiler Construction (pp. 42-52). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3302516.3307347

To unify or not to unify : A case study on unified builds (in WebKit). / Kubota, Takafumi; Suzuki, Yusuke; Kono, Kenji.

CC 2019 - Proceedings of the 28th International Conference on Compiler Construction. ed. / J. Nelson Amaral; Milind Kulkarni. Association for Computing Machinery, 2019. p. 42-52 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kubota, T, Suzuki, Y & Kono, K 2019, To unify or not to unify: A case study on unified builds (in WebKit). in JN Amaral & M Kulkarni (eds), CC 2019 - Proceedings of the 28th International Conference on Compiler Construction. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 42-52, 28th International Conference on Compiler Construction, CC 2019, Washington, United States, 19/2/16. https://doi.org/10.1145/3302516.3307347
Kubota T, Suzuki Y, Kono K. To unify or not to unify: A case study on unified builds (in WebKit). In Amaral JN, Kulkarni M, editors, CC 2019 - Proceedings of the 28th International Conference on Compiler Construction. Association for Computing Machinery. 2019. p. 42-52. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3302516.3307347
Kubota, Takafumi ; Suzuki, Yusuke ; Kono, Kenji. / To unify or not to unify : A case study on unified builds (in WebKit). CC 2019 - Proceedings of the 28th International Conference on Compiler Construction. editor / J. Nelson Amaral ; Milind Kulkarni. Association for Computing Machinery, 2019. pp. 42-52 (ACM International Conference Proceeding Series).
@inproceedings{6d7402a3ca5f4433a61037d3af02bf4d,
title = "To unify or not to unify: A case study on unified builds (in WebKit)",
abstract = "Unified builds are a simple but effective technique to reduce the build time of large software projects. Unified builds generate large compiler tasks by bundling multiple source files into one, resulting in a significant reduction in build time through removal of redundant work incurred by shared headers. However, unified builds have a negative effect on incremental builds because each compiler task gets larger. An ad-hoc unification strategy causes an excessive slowdown in incremental builds. A rough report from WebKit says the worst slowdown is 20{\%} (6s → 7s), but our investigation shows it is as high as 479{\%} (19s → 110s). In this paper, we investigate the characteristics of unified builds to find a sweet spot, which generates compiler tasks that reduce the full build time without increasing the incremental build time. An in-depth analysis of WebKit reveals 1) source files with higher header similarity should be unified, 2) source files that have significant differences in compile times should not be unified, and 3) source files that are not frontend-intensive should not be unified. Our case study shows the total build time is reduced by 2.66{\%}, and the worst slowdown falls from 479{\%} to 129{\%}. These findings will be of help in deriving a more intelligent strategy of unification and give a basis for discussions on future build systems, compilers, and module systems that cooperatively generate efficient compiler tasks.",
keywords = "Build system, Compile time analysis, Incremental builds, Source code characterization, Unified builds",
author = "Takafumi Kubota and Yusuke Suzuki and Kenji Kono",
year = "2019",
month = "2",
day = "16",
doi = "10.1145/3302516.3307347",
language = "English",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
pages = "42--52",
editor = "Amaral, {J. Nelson} and Milind Kulkarni",
booktitle = "CC 2019 - Proceedings of the 28th International Conference on Compiler Construction",

}

TY - GEN

T1 - To unify or not to unify

T2 - A case study on unified builds (in WebKit)

AU - Kubota, Takafumi

AU - Suzuki, Yusuke

AU - Kono, Kenji

PY - 2019/2/16

Y1 - 2019/2/16

N2 - Unified builds are a simple but effective technique to reduce the build time of large software projects. Unified builds generate large compiler tasks by bundling multiple source files into one, resulting in a significant reduction in build time through removal of redundant work incurred by shared headers. However, unified builds have a negative effect on incremental builds because each compiler task gets larger. An ad-hoc unification strategy causes an excessive slowdown in incremental builds. A rough report from WebKit says the worst slowdown is 20% (6s → 7s), but our investigation shows it is as high as 479% (19s → 110s). In this paper, we investigate the characteristics of unified builds to find a sweet spot, which generates compiler tasks that reduce the full build time without increasing the incremental build time. An in-depth analysis of WebKit reveals 1) source files with higher header similarity should be unified, 2) source files that have significant differences in compile times should not be unified, and 3) source files that are not frontend-intensive should not be unified. Our case study shows the total build time is reduced by 2.66%, and the worst slowdown falls from 479% to 129%. These findings will be of help in deriving a more intelligent strategy of unification and give a basis for discussions on future build systems, compilers, and module systems that cooperatively generate efficient compiler tasks.

AB - Unified builds are a simple but effective technique to reduce the build time of large software projects. Unified builds generate large compiler tasks by bundling multiple source files into one, resulting in a significant reduction in build time through removal of redundant work incurred by shared headers. However, unified builds have a negative effect on incremental builds because each compiler task gets larger. An ad-hoc unification strategy causes an excessive slowdown in incremental builds. A rough report from WebKit says the worst slowdown is 20% (6s → 7s), but our investigation shows it is as high as 479% (19s → 110s). In this paper, we investigate the characteristics of unified builds to find a sweet spot, which generates compiler tasks that reduce the full build time without increasing the incremental build time. An in-depth analysis of WebKit reveals 1) source files with higher header similarity should be unified, 2) source files that have significant differences in compile times should not be unified, and 3) source files that are not frontend-intensive should not be unified. Our case study shows the total build time is reduced by 2.66%, and the worst slowdown falls from 479% to 129%. These findings will be of help in deriving a more intelligent strategy of unification and give a basis for discussions on future build systems, compilers, and module systems that cooperatively generate efficient compiler tasks.

KW - Build system

KW - Compile time analysis

KW - Incremental builds

KW - Source code characterization

KW - Unified builds

UR - http://www.scopus.com/inward/record.url?scp=85062260387&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062260387&partnerID=8YFLogxK

U2 - 10.1145/3302516.3307347

DO - 10.1145/3302516.3307347

M3 - Conference contribution

AN - SCOPUS:85062260387

T3 - ACM International Conference Proceeding Series

SP - 42

EP - 52

BT - CC 2019 - Proceedings of the 28th International Conference on Compiler Construction

A2 - Amaral, J. Nelson

A2 - Kulkarni, Milind

PB - Association for Computing Machinery

ER -