Usability Sciences Corp. is an independent, internationally recognized testing center that performs competitive usability studies. Since 1988, it has been at the helm of competitive testing as an objective third party to do research that clearly differentiates products based on their ease of use. With over seven years of experience in testing hundreds of products with thousands of real world users, it has established itself as an industry recognized testing standard and an integral part of most software manufacturerΓÇÖs development processes.
Micrografx, Inc., one of the leading vendors of diagramming software hired Usability Sciences to assess the usability of its new diagramming product Micrografx FlowCharter 7 against its primary competitor Visio 4.0. The goal was to determine the relative productivity, efficiency and overall satisfaction of users working on identical processes using two competitive products. The test was also designed to reflect the real world experience of flowcharting users, capturing the entire procedure of process charting from the creation of the initial diagram of a process through the changes and editing that accompany the modification of a process. The test was conducted by Usability Sciences at their central testing facility in Irving, TX.
In their feedback, testers considered Micrografx FlowCharter 7 and Visio 4.0 to be professional and flowcharting applications. However, as in all comparative situations, one product outperformed the other based on features and functionality coupled with user friendliness. In the words of Jeff Schueler, president of Usability Sciences, ΓÇ£It is rare to see a product outperform a competitor as significantly as FlowCharter did in this study, which focused on working with process flowcharts and multi-paged diagrams. Users were more productive, more satisfied and preferred Micrografx FlowCharter over Visio by a wide marginΓÇ¥.
Overall users were able to complete their tasks 43% faster in FlowCharter 7 as compared to Visio 4.0. In fact, FlowCharter was found to be over 100% faster than Visio in modifying Process Flowcharts. Users expressed their extreme satisfaction over features like multiple page support, automatic shape connection and automatic line routing which are available in FlowCharter only. In the end, user were more productive, more satisfied and were more inclined to use FlowCharter for their process diagrams.
Observations
The study found that, on average, users completed the four tasks 43% faster in FlowCharter than in Visio. Most of this time was gained modifying a chart. Indeed, FlowCharter 7 proved to be over twice as fast as Visio 4 in modifying charts and diagrams.
Users were asked to comment on features they liked most in FlowCharter during the test. The most popular features that allowed them to work more efficiently were :
Multiple Page Support: FlowCharterΓÇÖs ability to view more than one page while modifying and/or formatting a multi-paged process flowchart was highly praised. Visio users felt this to be a big plus as one of the users noted that, ΓÇ£..it is a highly frustrating experience [with Visio].ΓÇ¥
Automatic Shape Connection and Line Routing: Users liked the way they could move shapes around and have the lines automatically adjust and reconnect when intermittent shapes were deleted.
Micrografx FlowCharter 7 has been developed to suit user needs and to make a user more productive thus concentrating on the process rather than the way to illustrate it.
Survey Responses
After the test, users were asked to complete a product satisfaction survey. In this section they ranked each product on a scale of 1 to 7, where 1 is ΓÇ£very dissatisfiedΓÇ¥ and 7 is ΓÇ£very satisfiedΓÇ¥ across 14 different factors they would consider while evaluating a flowcharting program.
Other than in the area of ΓÇ£ease of use in adding text,ΓÇ¥ a function which both programs accomplish equally well, FlowCharter 7 beat Visio 4 in every category. FlowCharter was truly the application of choice with a mean satisfaction rating of 6.25 while Visio trailed at 4.68.
Interview Responses
All users were interviewed individually after working with both products. Almost all the users chose FlowCharter 7 as their choice of an ΓÇ£easier to use, easier to learn and more productiveΓÇ¥ diagramming/flowcharting product.
The interview responses collected by Usability Science clearly show that FlowCharter 7 was the most easy-to-use tool to create and modify charts and diagrams of any kind.
Test Methodology
The study was conducted in November 1996 at the Usability Sciences Lab in Irving, TX. Six participants were chosen and pre-qualified users of flowcharting applications for process diagrams with intermediate to advanced experience in Windows 95. They received up to ten minutes of training on both FlowCharter 7 and Visio 4.0 before the test. In addition, each user spent approximately 15 minutes creating sample drawings to become familiar with the product.
Five major tasks were identified by Usability Sciences and Micrografx for the testing:
ΓÇó Create a one-page process flowchart
ΓÇó Modify a multi-page process diagram
ΓÇó Format a multi-page process diagram
ΓÇó Create an organization chart
ΓÇó Create a circle/spoke diagram
During the test, the users were not assisted in any way to complete the diagrams. The time it took for each user to complete all tasks was recorded for each product. All users were observed and videotaped as they worked. Usability SciencesΓÇÖ data logging software, TestLogr, was used to record detailed electronic notes of all user activities and comments. At the end of the test, users were asked to complete a satisfaction survey and were interviewed on camera to gain additional insight into their perceptions of FlowCharter 7 and Visio 4.0.
