Dr. Deming's Funnel Experiment
Source of main material: [1] - Henry R. Neave, "The Organization as a System: Edwards Deming's Principles for Building a Sustainable Business." / "The Deming Dimension. Henry R. Neave"; Per. from English - M.: Alpina Publisher, 2017. Scientific editors: Y. Adler, Y. Rubanik, V. Shper. You can purchase the book from the publisher Alpina Publisher .
Notes and illustrations: Scientific Director of the AQT Center Sergey P. Grigoryev
Free access to articles does not in any way diminish the value of the materials contained in them.
To simultaneously translate English-language videos into Russian, use the new Yandex Browser feature - real-time video translation.
The Funnel and Target experiment is an excellent demonstration that is important both for studying their critical importance in management and for understanding why common management practices produce unsatisfactory results.
“We ourselves will destroy everything with our own persistent efforts.”
Description of the experiment
As we have noted many times before, Deming tells us that the best intentions, the most fervent effort, and the hardest work will not produce quality.
Why not? Is there anything wrong with good intentions and hard work? Unfortunately, the answer is yes. We all know the saying that the road to hell is paved with good intentions. Deming’s version is: “We ourselves will destroy everything with our own persistent efforts.”
If our efforts and hard work are directed toward the wrong things, harmful things, or toward the right things but not in the way we should, we may end up in a much worse position than we were in the beginning. The harder a person struggles in quicksand, the faster he disappears into it.
Efforts and hard work are often aimed at correcting those things and phenomena that are useless or harmful. No one is saying that effort and hard work in themselves are bad, but knowledge is in-depth knowledge - are needed to ensure that this virtue is rewarded and brings positive results rather than disappointment.
The funnel-and-target experiment, which stemmed from a proposal made by Lloyd Nelson to Deming in 1986, is a fairly simple physical model of how even great improvement efforts can end badly.
The experiment can be carried out with very simple equipment:
- A funnel like the one found in any kitchen or garage.
- A tripod holder for a funnel, such as a table lamp, to which the funnel can be attached with wire.
- A small ball that can fit into the hole in the funnel. Have a few in stock (you won't have to crawl around and look on the floor under the furniture).
- A table or other horizontal surface covered with a soft, washable material, preferably ironed to prevent wrinkles.
- Colored markers (one color for each of the four rules).
- A ruler or any other device for measuring distances and angles, not necessarily with great accuracy.
The target is applied to the cloth and the funnel is placed over the target. The ball is thrown through the funnel, and the position where it lands on the table is marked with a marker. The holder and therefore the funnel can move according to a set of rules that we will formulate below. The ball is thrown into the funnel a second time, its stopping position is marked, and the funnel moves again. The process must be repeated several dozen times. What rules for moving the funnel can we consider? Deming suggests four.
Rule 1
The easiest thing is to not move the funnel regardless of where the ball stops.
Rice. Rule 1 of Edwards Deming's funnel and target experiment.
One hundred successive throws of the ball in accordance with Rule 1 give the result shown in Figure 1. It is not surprising that we obtain a scattering figure approximately in the shape of a circle centered on the target.
Rice. Rule 1 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws.
Rice. Rule 1 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws with the upper limit of acceptable results (UPL - upper limit of tolerance).
This is not exactly what we would like. Let's try to improve things.
"Let's do something. Don't just sit there. Something needs to be done. Move the funnel."
In accordance with rules 2 and 3, the funnel moves in such a way as to compensate for the discrepancy by which the ball has deviated from the target. Let's start describing the Rules in order of increasing complexity.
Rule 3
Rule 3 works as follows. Let's say the ball comes to rest six inches east of the center of the target. The funnel then moves west from the center of the target before the next throw. Or, if the ball stops four inches southwest of the center of the target, the funnel moves to aim at a point four inches northeast of the center of the target before the next throw.
Rice. Rule 3 of Edwards Deming's funnel and target experiment.
The obvious weakness of Rule 3 is that when determining the next position of the funnel, it does not take into account the position it currently occupies. The consequences of this are easy to trace if you conduct an appropriate experiment. The reader can try to draw this situation and figure out what behavior would be observed according to Rule 3 before looking at Figure 3.
Rule 3 gives a terrible result. Over time, the general tendency is for the ball to move further and further away from the center, oscillating in successive throws from one part of the pattern to another.
Rice. Rule 3 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws.
Rice. Rule 3 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws with the upper limit of acceptable results (UPL - upper limit of tolerance).
"Rule 3. Oscillate, back and forth, with gradually increasing amplitude until an 'explosion' occurs."
The reason for oscillation is that if the funnel is, say, three units east of the target, then the ball will likely end up somewhere in that area, as Rule 3 suggests; then move the funnel approximately three units west of the target on the next throw. After that she will return back to the east, etc.
Rule 2
Rule 2 demonstrates a more reasonable position for the funnel to move relative to its previous position, rather than in relation to the target (Rule 3). Therefore, returning to the previous illustration, suppose that the ball came to rest six inches east of the center of the target.
Rule 2 moves the funnel six inches west of its current position. And if in the next step the ball is four inches southwest of the center of the target, then the funnel moves four inches northwest of its current position.
Rice. Rule 2 of Edwards Deming's funnel and target experiment.
Now let’s turn with hope to the results of the “improved” Rule 2. But what a disappointment! Of course, things are not as outright bad as in the case of Rule 3. So, we are back to the situation giving almost the same circular shape of the scattering of results around the target for Rule 1. But the circle is now larger than it was, i.e. The spread has increased and the quality has deteriorated. In fact (although this cannot be calculated) any reasonable approach to measuring the areas of two circles shows that the area under Rule 2 is twice the area under Rule 1.
Rice. Rule 2 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws.
Rice. Rule 2 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws with the upper limit of acceptable results (UPL - upper limit of tolerance).
So, the great idea turned out to be worthless. What to do? It appears that one should forget about the goal and, in the interests of improving quality, concentrate on minimizing the variability between subsequent ball throws. In this way, we can at least improve uniformity and reproducibility, albeit by focusing on something other than the original target. There is a clear path to achieving this goal. This gives us rule 4.
Rule 4
Rule 4: At each step, place the funnel directly above the position where the ball just stopped.
Rice. Rule 4 of Edwards Deming's funnel and target experiment.
Well, one part of this description is true. Rule 4 actually minimizes the likely distance between the marks of the two subsequent ball throws. So over a short time frame, this rule does seem to make some sense. But be careful! What will happen in the future? Look for the answer in Figure 4. The behavior is almost as bad as in the case of rule 3: “The system explodes.”
Rice. Rule 4 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws.
Rice. Rule 4 of Edwards Deming's funnel and target experiment. The result of one hundred consecutive ball throws with the upper limit of acceptable results (UPL - upper limit of tolerance).
Note S. Grigoryev: I focus on a subtle and untouched moment in the description of the experiment with the funnel and the target. This is the time and effort spent preparing a new throw in Rules 2, 3 and 4. As with the dominant style of performance management (reactive management), much more time and effort is required than using Rule 1. The additional time and effort required results in to an even worse result than when using Rule 1 (non-interference in the operation of a stable system). And managers get bogged down in reactive turnover, not having time to make truly important improvements.
As the Rule 4 experiment continues, the ball tends to move further and further away from the target. This is not surprising, given that the target does not appear in our calculations of the crater's location. The only real difference from Rule 3 is that the position of the ball does not fluctuate from one side of the picture to the other, it continuously moves away from the center in some general direction.
“All this only leads to a worsening situation!”
Any form of systematic sample-to-sample compensation is a candidate for Rule 2. Calibration procedures are also often a good example of this type of situation. A standard sample is measured at the beginning of each day and the instrument is adjusted according to the observed error.
One of the participants in Deming's four-day seminar reported a similar example in which the sights of submarine torpedo tubes were adjusted in accordance with the value of the deviation from the target in the first shot.
Having established the essence of Rule 2, we can easily take the next step and consider how interference manifests itself in other, less formalized, but much more serious cases. Workers who are praised or criticized based on whether their performance is above or below average (when their performance actually corresponds to a statistically controlled situation) are exposed to Rule 2 or perhaps other rules, and the overall outcome their performance will therefore be more variable.
Deming viewed the immediate direct response to defects, complaints, errors, or incidents as an example of Rule 2 (or worse).
A few words of clarification. The reader is certainly right to believe that some kind of reaction to an incident, mistake or failure can sometimes be justified. Indeed, actions aimed at reassuring the consumer or compensating him and other victims may be necessary or at least desirable. The type of action that Deming is concerned about is the impact on a process or system due to the occurrence of a questionable isolated event.
So how does this fit with the theoretical premises of the funnel and target experiment? The answer is that such experiments are based on two fundamental assumptions. One of them is this: the funnel can actually be placed exactly above the target or above any other point. Second: the process that produces a deviation from the goal is in a statistically controlled state.
On the other hand, if the second assumption is incorrect, and the first is at least somehow true, Rule 2 may give a better result than Rule 1, i.e., it may be relatively useful to adjust based on the results of exploratory observations. So, in particular, if the average of the process changes within certain limits in an unpredictable way, Rule 2 allows you to track its features. Such tracking will not be completely hopeless if the average changes by a small amount between steps. An example of such a “walking” of the average as a source of special causes of variations is the aging of catalysts and fluctuations in the metal content in the ore.
Of course, the result in terms of dispersion will not be nearly as good as it would be if the process could be brought under control and Rule 1 applied. Consequently, the key question becomes whether the determining process is in a statistically controlled state or not. If yes, then Rule 2 is harmful; if not, it can be useful. Therefore, an important point in learning about response to incidents, errors, complaints, etc. is whether they will be part of the system? In other words, are they due to some exceptional special cause or are we simply dealing with relatively high or low values that are nevertheless within the control chart boundaries? In the first case, some action is really needed to identify the cause of this special case in order to try to prevent it from occurring again.
A practical illustration of Rule 3 is not so easy to find. The main reason for this is primarily because Rule 3 is just stupid. Based on the same idea as Rule 2, it does not take into account the position the instrument was aimed at before the last attempt. Moreover, as we have seen, the application of Rule 3 causes such unbridled behavior in the system that in practice such behavior will be quickly detected, and even if its reasons are not fully understood, a different strategy will be adopted. However, manifestations of Rule 3 are not entirely undetectable. Let us shoot a rifle, and suppose the bullet hits one inch above the target. It may not be so unreasonable to decide to aim one inch below the target, rather than one inch below the point at which you were previously aiming. Silly, but imaginable. A useful clue to recognizing the effects of Rule 3 may be the observed behavior of the system, that is, increasing instability with wild jumps in one direction or the other. Since such behavior is undesirable, it tends to occur only if the time interval over which the process occurs is quite long. Examples include the vacillation between left and right in politics or between freedom of the press and censorship.
Given that the harm caused by Rule 2 is limited (this in no way means that it can be neglected), and Rule 3 occurs quite rarely, in practice, Rule 4 can create the most serious problems. However, it is also insidious. In a very narrow sense, it does indeed reduce dispersion. If the two premises discussed above are true, then one cannot but agree that a possible way to minimize the average difference between randomly selected outcomes would be to aim the funnel at the point of the previous outcome. But we already saw the long-term effect of such local optimization in the picture of the results of Rule 4. A good way to understand what happens in the case of Rule 4 is to imagine the game of “broken telephone”, which is sometimes played at parties. One person whispers a sentence or two to another, who then whispers what he understands to a third, etc. By the time what has been said reaches the fifteenth or twentieth player, it is quite different from what it was at first.
Deming gives three examples of Rule 4 in action.
- The first is the operator who tries to achieve homogeneity by trying to make each item the same as the one before it.
- The second case, directly related to the first, is the practice of color matching, when each batch of material, or paint, or processed photographic film is selected as close as possible to the previous batch.
- The third example is the practice of training a worker to work. People involved in the work process train a new person. After three days he is already a “veteran” and is considered to be quite ready to teach other newcomers, who, in turn, in a few days teach the new generation. Why is this done? One reason is obvious: it's cheap.
“Managers are busy with cheap things, they ignore huge losses.”
What is excusable in principle (though not in practice) is that there is a general belief that whoever does a certain job knows it better than anyone else. This is true for some part of this work. But partial knowledge is wrong, wrong.
The idea of having an employee train others sounds great. But over time, the result will move further and further from the desired one. In this case, we actually apply Rule 4 and engage in a process that statisticians define as a "random walk" away from the target. Practice, when a worker teaches a worker, can give an even worse result if the learning process requires knowledge of teaching methods: without their knowledge, additional deviation from the goal will occur. Yet the worst effect of Rule 4 practices occurs at the top level of management:
"It's even worse when leaders work together without the principles of deep knowledge. Based on Rule 4, they wander off into the vast distance, trying their best."
In conclusion, let us mention some of the many examples given at the seminars by participants or by Deming himself. It is not always clear which rule is appropriate in a given case. What is clear, however, is that they all serve as examples of the fact that, no matter how reasonable some approaches may seem at first glance, in the longer term they nevertheless only make things worse. Such examples relate to holding workers responsible for defective products that they obviously produce; language teaching. The third obstacle was "search for examples" . All of them illustrate the actions of Rule 4. The most familiar example, of the newfangled ones, is benchmarking.
Once the implications of these ideas sink in, participants in the four-day workshops provide numerous examples of the wrong kind of influences, where efforts are directed more towards neutralizing the consequences than at their root causes, including blaming workers for things that have nothing to do with the causes (for example, altercations with waiters regarding the poor quality of food, blaming the hotel maid for lost towels). Other proposals initiated by isolated undesirable cases concerned changes in bank discount rates; changes in safety or security practices, cost forecasting and planning, including the requirement to spend budget balances at the end of the year under threat of budget cuts the following year; linking the salary level to the inflation rate; revising courses based on individual student feedback; quarterly redistribution of material costs and efforts in order to achieve the results required for the quarterly report; lower prices compared to the market average; planning an upcoming meeting based on the duration of the previous one; attracting consultants to overcome the manifestations of a certain problem, rather than finding why it appears; stopping the production line as soon as something goes out of tolerance; making a copy of a copy; setting the temperature in the thermostat; introduction of trade barriers, proliferation of nuclear technologies.
The funnel and target experiment is a great example of how a very simple tool can demonstrate deep and insightful ideas. In the funnel experiment, a natural question arises: what can be done to improve things? We already know the answer. Since the system in question is in a state of statistical control, real improvements can only be achieved through real (systemic) change. They cannot be obtained by influencing the outputs, i.e., the results of the system's operation: influencing the outputs is only suitable in the presence of special causes of variations. Influencing the results is exactly what Rules 2, 3 and 4 are aimed at in the funnel experiment, and all the emotional exclamations of the master in this experiment are also aimed at. Influencing a system to eliminate common causes of variation is usually a more difficult task than acting to eliminate special causes. Thus, in the funnel experiment, the funnel itself can be lowered or a softer cloth can be used to cover the table in order to dampen some of the movement of the ball after it falls. In an experiment with red beads Somehow the proportion of red beads in the box must be reduced - by introducing improvements in upstream stages of the manufacturing process or in the supply of raw materials, or both.
Don't let this simplicity fool you: the meaning it conveys is extremely important.
Dr. Edwards Deming on the Funnel and Target Experiment. Deming Funnel Experiment.
P.S.
Sergey P. Grigoryev: Conducting training seminars, demonstrating experiments that E. Deming demonstrated at his four-day seminars, I am faced with a gap between the knowledge acquired during the training period and the subsequent application of Edwards Deming’s theory of systems management in practice by management. I see one of the main reasons for this circumstance as the unpreparedness of many managers for a full-scale change in management style, and without this transformation is impossible.
The best reminder of the ideas demonstrated at the seminars are not certificates, but woven materials placed in a frame on the office wall with a target (target) and marks of where the ball stops when performing Rules 1, 2, 3 and 4.
In his article [14] "THE DEMING DIMENSION: Management for a Better Future", Henry Neave estimates that approximately a quarter of a million people attended Edwards Deming's famous four-day seminars between 1980 and 1993. . This article presents an excerpt from an interview with Edwards Deming to The Washington Post conducted in 1984, which we publish below:
"You have been very successful in attracting people to these seminars. Isn't that encouraging to you?"
"I don't know why that should be encouraging. I want to see what they're going to do. It'll take years."