A validation regarding effectiveness of Scenario Graph

Scenario Graph is a structured mind mapping methodology that aids design teams to generate potential scenarios for new products and services while visually organizing contextual information. Since its introduction in industry and academia, the Scenario Graph has helped design teams to capture new values and behaviors of potential customers during the problem formulation stage. At the same time, the Scenario Graph, sharing a common challenge with various design methods, has faced difficulty regarding validation of its effectiveness as a design method. This paper describes a validation framework for a method in problem formulation stages and an experiment, which compare ideation results of 87 people-43 people with the Scenario Graph method (as a test group) and 44 people with the Brainstorming (as a control group) - to solve an identical problem. While the results show no statistically significant difference in the number of ideas generated, the data reveals statistically significant differences in the quality of ideas. The test group, which used the Scenario Graph, yielded ideas that were more novel, feasible, and abstract than the control group, which used the Brainstorming, did. These metrics represent a way to measure the quality of ideas in the domain of engineering design. Our experiment confirms the hypotheses that the Scenario Graph is effective in improving the performance of idea generation sessions, which is consistent with qualitative evaluations. The lessons, gained from this experiment, provide an insight on how this method can be effectively used during the early stages of concept generation of a company's process for product and/or service development.