Early detection of configuration errors to reduce failure damage

The Paper

https://www.usenix.org/system/files/conference/osdi16/osdi16-xu.pdf

Summary by Adrian Colyer on the Morning Paper here: https://blog.acolyer.org/2016/11/29/early-detection-of-configuration-errors-to-reduce-failure-damage/

Format

We start at 6:10, don't be late!

The discussion lasts for about 1 to 1.5 hours, depending upon the paper.

• Read the paper (done before you arrive)

• Introductions (name, and background)

• First impressions (1-2 minutes this is what I thought)

• Structured review (we move through the paper in order, everyone gets a chance to ask questions, offer comments, and raise concerns)

• Free form discussion

• Nominate and vote on the next paper

• Adjourn for food and/or drink at Steamworks (https://www.google.ca/maps?ion=1&espv=2&q=steamworks&bav=on.2,or.&biw=1674&bih=956&dpr=2&um=1&ie=UTF-8&sa=X&ved=0ahUKEwjTpLXtqb_MAhVC0mMKHWLiBLcQ_AUIBigB)

Abstract

Early detection is the key to minimizing failure damage induced by configuration errors, especially those errors in configurations that control failure handling and fault tolerance. Since such configurations are not needed for initialization, many systems do not check their settings early (e.g., at startup time). Consequently, the errors become latent until their manifestations cause severe damage, such as breaking the failure handling. Such latent errors are likely to escape from sysadmins’ observation and testing, and be deployed to production at scale.

Our study shows that many of today’s mature, widely used software systems are subject to latent configuration errors (referred to as LC errors) in their critically important configurations—those related to the system’s reliability, availability, and serviceability. One root cause is that many (14.0%–93.2%) of these configurations do not have any special code for checking the correctness of their settings at the system’s initialization time.

To help software systems detect LC errors early, we present a tool named PCHECK that analyzes the source code and automatically generates configuration checking code (called checkers). The checkers emulate the late execution that uses configuration values, and detect LC errors if the error manifestations are captured during the emulated execution. Our results show that PCHECK can help systems detect 75+% of real-world LC errors at the initialization phase, including 37 new LC errors that have not been exposed before. Compared with existing detection tools, it can detect 31% more LC errors.

Getting There

Stage 3 Systems is near the Vancouver City Centre Canada Line Station, or the Chinatown/Stadium Sky Train station, and there is pay parking available. Enter into the lobby, if no one is there to take you up the elevator, hang on if it's before 6:10, otherwise use the comments to get a hold of us.