How a ‘culture shift’ at SBA got program offices to embrace evaluations

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

Thanks to recent efforts like the DATA act, the federal government has massive amounts of data at its fingertips; the challenge now is figuring out what to do with it. Some agencies are linking data to performance management in order to make more informed, evidence-based decisions about spending on their programs.

That’s what the Small Business Administration (SBA) did about a year-and-a-half ago, when they created a central evaluation unit within the office of its Chief Financial Officer.

Advertisement

“We saw a lot of synergy between performance management and program evaluation — two different disciplines, yet interconnected,”  Tim Gribben, CFO and associate administrator for performance management at the SBA said at the May 1 Performance Improvement Officer/Chief Financial Officer Summit. “And it’s in the CFO’s office, because we’re also connecting it with financial management and what are we getting for the money? What is the true evidence from an independent standpoint?”

The first step was to create a learning agenda — a prioritized list of all the questions the various program offices were trying to answer.

“To me, [a learning agenda] is the key,” Gribben said. “We took it from our strategic plan on down to what are all the different things that we want to learn. And I think by doing that exercise, throughout the agency, it’s what got people thinking about taking it from an abstract concept to thinking ‘what are the things I want to know, what are the things I want to learn?’That, and being able to write them down in a series of different questions that we’re going to try to answer over the next four to five years. To me, that’s the key, that’s what changed the outlook at SBA as well.”

And changing the outlook was necessary, he said, because at first, many of the program offices viewed this new push for evaluations with suspicion and anxiety. They saw a new way to justify cutting their budgets; they didn’t see the benefits at first. For example, an objective program evaluation can help justify what a program office wants to do, and what new directions it may want to go.

Gribben said SBA started trying to change this culture by offering programs money for performing evaluations. That first year, he said they got more proposals than they could fund, and SBA was able to pick from the best. But even the programs that didn’t get picked began to see the benefit of evaluations, and set aside some of their own money to perform them. The following year, he said they got twice as many proposals.

“With SBA, how did we change the culture?” Gribben said. “We changed it by creating a community of practice, where we have the lead evaluation experts on a monthly basis meet with somebody from each of the program offices who had an interest in evidence and evaluation.”

But Peter Warren, associate director at the Office of Management and Budget, said this isn’t always the case at other agencies.

“There’s not really a very consistent baseline that is adhered to across the government,” he said. “I think that there’s a lot of areas — and certainly performance data is one of them — where more use of rigorous evidence metrics could be brought to bear.”

And that’s going to be a key part of evaluations moving forward, as agencies continue to collect more data and make it more available. The President’s Management Agenda identified “data, accountability and transparency” as one of its three “drivers of transformation,” and detailed how they would be used to improve both IT modernization projects and  employee engagement at low-performing agencies.

So having evidence that shows the value of programs and projects will be increasingly important to agencies, especially when it comes time to justify the budget.

“In this environment, if you can’t show that your programs are really producing, it’s harder and harder,” said Nancy Potok, chief statistician of the U.S. and chief of statistical policy at the Office of Management and Budget. “The purse strings are going to tighten up considerably. They already have been. And, I think if you’ve been paying attention to revenue projections from the federal government, you know it’s only going to get worse.”