How We Measure Success - Part I

The current economic climate has made it increasingly important to the public to track the performance of programs receiving federal funds. People want to know how their tax money is being spent and what the impact is. Are the programs spending the money in the right way? Are they achieving the goals and outcomes they set out to achieve? Measuring performance is in the spotlight like never before, but it doesn’t make it any easier.

How is performance measured?

On the website, ExpectMore.gov, the Office of Management and Budget (OMB) explains how performance is measured for Federal programs and distinguishes between outcomes and outputs as follows:

• An outcome refers to the events or conditions of direct importance to the public/beneficiary that are external to the program. An outcome answers the question 'What is the program's goal or purpose?' For example, the goal of a job training program is to give someone the skills to find a job, as opposed to giving out a grant. An outcome measure may be the number and percent of people employed within six months of completing the job training program or how much their income increased.

• An output refers to the internal activities of a program, such as the products or services delivered. The output answers the question 'What does the program do to achieve its goal or purpose?' For example, a job training program may provide a class to teach someone the skills necessary to find a job. An output measure may be the number of people who complete a job training program.

How does Business Gateway measure performance?

The examples cited by OMB illustrate that while outcomes are far more indicative of success, outputs are much easier to measure. So how do we measure Business Gateway’s performance? First, we answer the question “what is the program’s goal or purpose?”

The mission of Business Gateway is to provide innovative information services to the small business community that save time and money, engage citizens to participate, and enable collaboration across all levels of government. Right off the bat, we have three goals to measure.

Measuring outcomes and outputs

The first and foremost goal – the cornerstone of the program – is to save the small business community time and money. That is one of the key, desired outcomes of the program.

Now we answer the question “what does the program do to achieve its goal or purpose?” The products and services we deliver to the small business community – the information services provided on business.gov – are key outputs of the program.

Since outputs are easier to measure, let’s start there. We track the usage of our information services on business.gov as output measures. Here are some of our key output measures for March 2009:

• Total number of visits = 332,262
• Total number of page views = 1,017,876
• Average page views per visit = 3
• Average visit length = 7 minutes

Monthly visits have nearly doubled in the first quarter of 2009, compared to only 171,050 in September 2008. The average visit length has increased by nearly a minute. All indicators are moving in the right direction. More and more people are visiting business.gov and spending more time once they get there. That is good, right? But is it enough? While these output measures show continuous improvements, what do they say about our key desired outcome of saving the small business community time and money?

This is the start of a series of articles on measuring performance. Stay tuned for Part II next week where we will explain how we use the American Customer Satisfaction Survey to gauge how satisfied our customers are with the information services provided on business.gov and how much time and money they believe visiting the site saved them.