Carnegie Mellon University - Heinz School of Policy · Management · Information Technology

A Survey of the Incidence and Cost of Software Problems 

Your cooperation will guide in decision-making for users of software and for post-purchase quality checks. Your information will be kept absolutely confidential and anonymous. If you are working at a large organization please answer questions for your division.

1.   What prompts a decision to buy or build a new software product at your organization? Please check all that apply.
      A change in organizational needs (e.g., growth, new product lines) prompts a decision.
A favorable change in the price and quality of available software products prompts a decision.
A software vendor approaches us with a solution to one of our problems.
An internal IT employee approaches us with a solution to one of our problems.
2.   How long does the entire buy/build process for new software take, from the idea recognition stage to the actual usage stage? The length of the process varies from to .
3. What percentage of the software in use is of particular type:
Customized for our organization 
Commercial off the shelf
   Commercial off the shelf that underwent significant customization
4. Roughly how many business applications does your organization have (excluding software for utilities, operating systems, etc.) 
 Some examples of such applications are customer relationship management, data warehousing, enterprise resource planning, sales force automation, supply chain management, and others unique to your business.
5. What is the average number of patches/upgrades per month your organization implements?
6.   Here are two hypothetical software products that differ by cost (price/time to market), features, and quality certainty. 

Please choose one as if you are buying one for your organization.
    Software Product  Cost  Features  Quality Certainty
1  High  Sufficient  Uncertain
2  Low  Insufficient  Certain
7. All software has bugs, minor or major. Minor software problems may add up and have a cumulative effect on productivity. 
Some examples of minor problems are listed below: 
• Java errors requiring browser restart
• Other application dependability errors (not behaving as expected)
• Application warnings, crashes, or sluggishness requiring restarts
• OS restart/reboot
• e-mail connectivity problems
• e-mail attachment errors
• OS or application reinstalls to correct chronic errors
• Document or file corruption
• Document or file loss
• Virus activity (triggered)
• Untriggered virus detection and deletion
• Interaction with other applications causing failures
• Expired, missing, corrupt license files, or codes
In your opinion what is the average number of times per week a typical employee (or yourself) encounters a minor problem (for any software product)? 
When a typical employee (or you) experience a minor problem, how many people typically assist (co-workers, IT staff, others) 
How much time passes before resuming work? 
8. Now we turn to more significant problems, that effect everyone at the same time. Please check all that apply. Due to software problems, at one point or other, in our organization one or more of the following happened:  
We lost intangible property such as data and information

The loss of intangible property such as data and information probably cost at least ($) but no more than ($).
Relevant business activity got interrupted 
For how long?
No business got interrupted but because of the problem, our staff was kept busy 
For how long?
The problem was:
Not at all apparent to our clients
Some clients knew but no major impact
Inconvenienced some clients
Posed a serious problem for customer relations
 
We had to write off financial loss
Financial loss that we wrote off was at least ($) but no more than ($) .
We had to terminate the project
The terminated project cost at least ($) but no more than ($) .
Additional capital expenditure
Additional capital expenditure to solve the problem cost at least ($) but no more than ($) .
Problem impacted our brand equity
9. Your organization currently has how many full-time equivalent employees (FTEs)?
10. What percentage of your FTEs are working in IT? (%)
11. What is your annual IT budget?
12. Approximately how much was the total revenue of your organization last year? 
Less than $500,000
Between $500,000 and $1 million
Between $1 million+ and $5 million
Between $5 million+ and $15 million 
Between $15 million+ and $50 million 
Between $50 million+ and $100 million
More than $100 million
13. What best represents the industry in which your organization operates? (Please answer for your organization and not for the parent firm, if any.)
Agriculture, Forestry, and Fishing
Mining 
Construction 
Manufacturing 
Transportation and Public Utilities
Wholesale Trade
Retail Trade
Finance, Insurance, and Real Estate
Services 
Public Administration
Other (please specify) 
14. Your organization is:
Nonprofit 
Private 
Public 
Government 
15. Your position within the organization is