Measuring Success of Lean Six Sigma Efforts

When applying Lean Six Sigma for process improvements, a general principle is to ensure the process is stable. If it’s not, it needs to be brought under control before anything can be done to improve it.

Unfortunately, some processes are naturally unstable, and if nothing is done to change this, they will continue to produce outcomes that fail customer expectations. However, if improvements are pursued, it’s important to define and measure success.

In 2021, I had the opportunity to follow a local company’s team that was applying the Lean Six Sigma methodology to improve its contracting process. I wrote about this experience in “How Statistics Tell a Lean Six Sigma Story,” which detailed the problem they were solving — reducing contracting cycle time — and how they applied the define, measure, and analyze phases of Lean Six Sigma’s DMAIC (define, measure, analyze, improve and control) process.

This follow-up article looks at their approaches to the other two phases of the process — improve and control.

Interpreting the Control Chart

As the team collected more data about the contracting process, they performed various in-depth analyses and concluded that the root causes were bottlenecks and waste. They then reviewed these findings with other departments and generated improvement ideas that were then implemented.

At the end of 2022, the team compared contracts executed since the implementation with older contracts. They were excited to see that the average contracting cycle time (number of days between the beginning and end of the contracting process) had been reduced by 30 percent, with a p-value of less than 0.05. (A p-value is a statistical concept. A low p-value such as 0.05 provides high level of confidence in the findings from statistical analysis.)

However, a control chart generated with the cycle time data for all the contracts showed that there were still outliers among the contracts executed during the post-implementation phase (see chart below), which indicates that the process is still not entirely under control.

Each dot represents a contract and is arranged along the X-axis chronologically in the order it was executed. The “before” section covers the period before the improvements were implemented. The “after” section is related to the post-implementation period. The Y-axis shows the value of cycle time. The red squares are outliers.

This finding triggered two important questions:

• Does the existence of outliers negate the apparent improvement in contracting cycle time?
• If not, does it mean the team can start working on improving a process before it becomes stable, contrary to the general principle of process improvement?

Questions Answered, Insights Gained

The team consulted with Lean Six Sigma experts from within and outside of the company, who advised team members to look further into the process and answer questions.

One of the questions was whether the contracting process is uncontrollable by design. Due to two factors that the team had no control over but had impacted cycle time, they answered affirmatively. The first factor was a change in the statement of work (SOW); the second was a decision to contract with a new supplier; both were made by internal customers. These types of decisions can cause significant delay and result in outliers in the cycle time.

With the knowledge that their process was unstable by nature, team members looked at how to measure the success of process improvement, and considered using process capability to define it. Since the internal customer’s ultimate expectation is for every contract to be executed in 30 days, the team’s level of success could be measured by the percentage of contracts that are met within that time frame.

Using the same set of data, the team performed a capability analysis, which provided encouraging results. Before the team started Lean Six Sigma efforts, the out-of-spec percentage (those not meeting specifications) of the contracting process was 78 percent; it was now 45 percent (see chart below). The P-value was less than 0.05, which meant the team increased the capability of the contracting process — it is now capable of meeting the end user’s expectation 55 percent of the time, compared to 22 percent before.

The X-axis represents contracting cycle time (in days). UB means upper boundary; it represents the upper limit of the cycle time range expected by internal customer, which is 30 days. “% > UB” indicates the percentage of contracts that falls outside the range of internal customer’s expectation.

As the team was celebrating its successes and getting ready to continue its improvement journey, the members also obtained valuable insight into the application of the Lean Six Sigma methodology.

Overall, some processes are naturally uncontrollable, but this does not mean improvements are unattainable. Instead, it is key that a meaningful measurement of success is selected.

(Top image credit: Getty Images/Bakhtiar Zein)