Volume 5, Number 1 – March 2008

New Metrics for New Programs: How to Measure the Impact of Entrepreneurial Development Investments

One of the hardest parts of supporting new entrepreneurial development strategies is finding good measures of progress and impact.   Economic developers have traditionally measured their progress by counting jobs.   This approach makes sense when applied to traditional economic development efforts that emphasize business recruitment.   When a new plant opens in town, you can count the jobs pretty easily. 

It’s not quite so easy with supporting entrepreneurs.  These companies create jobs, but they do it slowly.   An entrepreneurial business might create hundreds of jobs, but they do it over years.  Meanwhile, the newly opened branch plant creates lots of new jobs at one point in time. 

So, if your region is introducing new entrepreneurial development programs, you might also think about new ways to measure their impact.  That’s one important lesson from a forthcoming evaluation of the Appalachian Regional Commission’s (ARC) Entrepreneurship Initiative. This program, which began in 1997, was the first Federally sponsored economic development program with an explicit focus on nurturing local entrepreneurs.

The program operated until 2005, and ARC continues to fund similar efforts through other ARC initiatives.    The Entrepreneurship Initiative ended doing pretty well when assessed using traditional job creation measures.   More specifically, over 12,000 jobs have been created or retained and over 1,700 new businesses were created, at a cost that suggests an efficient allocation of resources relative to other similar types of economic development programs.

However, these measures tell only part of the story of impacts. Entrepreneurship Initiative investments helped to train teachers and expose students to entrepreneurship concepts, almost 12,000 throughout the region. They supported incubators that served at least 475 emerging businesses and helped spark the development of sectors such as the sustainable wood products industry. These public investments have been instrumental in attracting almost $73 million in private capital injection to support entrepreneurship development in the region. Through its educational investments and the demonstration effect of ongoing projects, ARC also helped make entrepreneurship a legitimate and desirable economic development activity in local communities while raising awareness about the importance of the entrepreneurial and small business sector to the region’s economy.

ARC has not been able to tell this complete story in the past, because, like many Federal programs, ARC’s current performance metrics focus almost exclusively on tracking short-term job creation.   ARC’s experience is not unique, and other economic development agencies face similar dilemmas.  As we move ahead, program managers should heed two lessons from ARC’s experience:

Replace old, overused metrics with a comprehensive “entrepreneurship development metrics portfolio”.  Ã¢â‚¬Å“Job creation” is an overused metric.  It paints an incomplete picture of the outcomes of entrepreneurship development investments, and should be replaced by a more encompassing “entrepreneurship development metrics portfolio.” While job creation should be reported as an outcome of program investments, a much richer understanding of impacts can come through efforts to define and capture outcomes as measured by a broad set of performance metrics. In addition to jobs created/retained and new business starts, this system should include other outcome measures such as the following:

• Changes in business profitability following a capital investment
• Number of incubator tenants who graduate from a facility and stay in the region
• Number of business owners still in business after receiving counseling or technical assistance
• Changes in sales after receiving outside technical support
• Changes in attitudes following completion of training programs.

A “best in class” metrics system requires investment in a “best in class” evaluation system. Performance measurement should be viewed as an integral part of program development – from the perspective of funding agencies like ARC and project leaders.  Federal and state economic development agencies should support the creation of a performance measurement system for future investments by developing a participatory evaluation system in partnership with grantees. This measurement system would be developed by grantees and their customers, i.e., the entrepreneurs, with outside funding, and would be designed to provide project leaders with useful information that can be used to adapt programs to changing circumstances as well as to report to funders on project performance.

If program managers want to “tell a better story,” they will need to invest resources and energy into this process.  Effective program measurement will require that evaluation becomes a core part of program design and implementation, and not an afterthought.

OTHER RESOURCES

Much of this essay is based on a forthcoming report:  Ã¢â‚¬Å“Creating an Entrepreneurial Appalachian Region:  Findings and Lessons from an Evaluation of the Appalachian Regional Commission’s Entrepreneurship Initiative, 1997-2005.”  Prepared for the Appalachian Regional Commission by Deborah Markley, Erik R. Pages, Brian Dabson, Thomas Johnson, Sara Lawrence, Sara Yanosy, and Karen Dabson.  The full report and executive summary will soon be published and will be available on the EntreWorks Consulting website.

Other excellent recent reports on program evaluation in the fields of small business development and entrepreneurship include the following:

Pennsylvania Department of Community and Economic Development, “Measuring Up:  Enhanced Metrics for a New Economy,” Harrisburg:  Pennsylvania DCED, 2007.  Available at:  http://www.newpa.com/default.aspx?id=55

Urban Institute Evaluations of US Small Business Administration (SBA) programs.  The Urban Institute has recently completed a series of excellent evaluations of various credit and support programs operated by the SBA. The studies csan be found at http://www.urban.org.

What’s New at EntreWorks Consulting?

NEW ARTICLES:

We’ve got some new goodies that have been posted to the EntreWorks Library. To access the reports, click here.

“Fueling Your Business in North Carolina:  A Guide to Financing for Small Business.”  Prepared for the North Carolina Rural Entrepreneurship Development Systems Project.    While this guide contains a lot of North Carolina-specific materials, it will be helpful for anyone working with small business owners who are seeking capital to start or grow business. 

“Lessons in Self-Made Success,” co-authored by Erik Pages and Stuart Rosenfeld, appeared in December 2007/January 2008 edition of Community College Journal.

UPCOMING SPEAKING ENGAGEMENTS:

Erik R. Pages of EntreWorks Consulting will be on the road for lots of speaking engagements this Spring.  Here’s a sampling:

• April 3: Rural Development Conference, Cookeville, TN
• April 13: IEDC Federal Forum, Washington, DC
• April 29-30: Advantage West Development Group, Boone, NC
• May 7: Small Business Center Network, Chapel Hill, NC
• June 6: Professional Developers of Iowa, Spencer, IA

We hope to see you on the road!