|
Frequently Asked Questions
- What are outcomes and how do you measure them?
- What is the difference between outputs and outcomes?
- How do I establish outcomes for my project?
- What is outcome-based evaluation?
- What can outcome-based evaluation do for organizations?
- How much time and effort will it take?
- How complicated is outcome-based evaluation?
- Aren't some things difficult to measure?
- What is a logic model?
- Is a logic model required as part of my grant application,
and do I have to evaluate my project this way?
- How do I report outcome-based evaluation information?
- Where can I find NEA's outcomes?
- How many outcomes should my project have?
- How do I report to NEA about the results of my project?
- What is NEA going to do with my information? Will there
be a report that includes my project?
- Where can I get more information?
As you review the information below, you may also wish to refer to the
Glossary, which contains definitions of outcome-based
evaluation terms.
1. What
are "outcomes" and how do you measure them? TOP
Outcomes are the benefits that accrue to participants
of a project; they represent the impact that the project has on participants.
Typically, outcomes represent a change in behavior, skills, knowledge,
attitude, status, or life condition of participants that occurs as a result
of the project.
Outcome example: Students demonstrate an interest in theater.
Outcomes are measured by identifying characteristics or
conditions that tell whether change has occurred. These measures are called
“indicators” and they are concrete, objective phenomena that
indicate that a specific change has taken place. To be useful, indicators
must be countable and/or observable evidence of change.
Indicator example: The # and % of students who attend at
least two live theater performances during the life of the project.
2. What
is the difference between outputs and outcomes? TOP
Outputs are the direct products of a project.
They are measured in terms of quantity. Typically, outputs are the number
of visits or number of attendees involved in a project, or the number
of times a particular activity has been conducted. Outcomes are the benefits
that accrue to project participants; they represent the results or impact
of the project.
3. How
do I establish outcomes for my project? TOP
First, the ultimate purpose of a project must be carefully
thought out and described. Generally, the purpose of a project is not
simply the performance of various activities. There must be a reason for
providing the activities. Ask, "Why are we offering this project?
What do we want to accomplish?" It may be helpful to ask project
staff, project partners, and other stakeholders: "If we are truly
successful with this project, what would those results look like for the
people we have served?" When those questions can be answered, it
is possible to state the kind of changes or impact that will happen as
a result of the project. Those hoped-for changes will become the intended
outcomes of the project. When applying for an Arts Endowment grant, you
will be required to select one Arts Endowment outcome that your project
intends to help achieve.
4. What
is outcome-based evaluation? TOP
Outcome-based evaluation is a systematic way to determine
if a project has achieved its goals. This organized approach helps organizations
establish clear project benefits (outcomes), identify ways to measure
the project benefits (indicators), and clarify for whom the project's
benefits are designed.
5. What
can outcome-based evaluation do for organizations? TOP
Employing outcome-based evaluation and reporting on the
impact of the project can have many positive benefits for an organization:
First, it can help organizations tell their story in ways
that their stakeholders and the general public can understand and appreciate.
It helps organizations convey important information about what has been
the collective impact on project participants, while maintaining the ability
to convey the very powerful and personal stories that show how important
the project was to specific individuals.
Second, it can help arts organizations be better positioned
to request and receive funding because they can describe the benefits
and impact of the proposed project in very specific terms by identifying
what the project will do for participants. This is particularly important
given that more and more funders are expecting projects to identify what
they hope to achieve as a result of funding.
Third, when outcome-based evaluation becomes part of the
organization's management routine, their projects can be improved. Project
goals are well planned and established, these goals are regularly reviewed,
and stakeholders are informed about the impact of funded projects. In
turn, outcome-based evaluation will help an organization's project staff
better communicate the benefits they intend to deliver to project participants.
6. How
much time and effort will it take? TOP
Conducting outcome-based evaluation does take a commitment
of time and resources. Sometimes organizations anticipate extra time and
resources needed for outcome-based evaluation and explicitly reflect this
in a portion of their budget designated for evaluation. Another approach
is to focus on one or two very important outcomes and ways to measure
them as a way to begin this effort while minimizing cost. Sometimes organizations
will partner with colleges, universities or other institutions to help
implement a project and carry out project evaluation.
7. How
complicated is outcome-based evaluation? TOP
Outcome-based evaluation is NOT formal research. It is
a management tool that helps staff know whether their project is achieving
its intended results. It is not necessary for outcome-based evaluation
to involve the use of statistical analysis and scientific research designs.
It does not have to be complicated.
8. Aren't
some things difficult to measure? TOP
Some things will seem more difficult to measure than others,
and not all things projects accomplish need be measured. It is easier
to measure "hard" impact, such as knowledge, behavior, and skills than
it is to measure "soft" impact such as attitudes. That is why clarifying
the relationship between the outcome and the measurable and observable
"indicators" is important. For a very good example of how one project
measures a soft impact, see "Measuring Joy: Evaluation
at Baltimore Clayworks," or request a copy of the article published
by the National Arts Stabilization Journal by writing to info@artstabilization.org.
9. What
is a logic model? TOP
A logic model is a step-by-step approach for defining
and measuring outcomes. It is your project's evaluation plan that shows
how you will measure outcomes, what information you need to collect, when
you will get the information and what constitutes the targets for the
outcomes.
10. Is
a logic model required as part of my grant application, and do I have
to evaluate my project this way? TOP
A logic model is NOT required as part of your project.
It is a tool we are sharing with applicants and grantees that can be used
to help structure project evaluations. Applicants, however, must address
directly one of the NEA-defined outcomes and state how they plan to measure
it.
Our intent is to provide as much information as logistically
possible for applicants and grantees to help them organize their project
in outcome terms and to show through examples and other tools how outcomes
can be measured.
11. How
do I report outcome-based evaluation information? TOP
It is often helpful to consider what your project's stakeholders
want to know about the results of your project when developing reports
from outcome-based evaluation data. The organization's board, its community
and funders may want similar information, but this does not mean that
one report will satisfy everyone. In general, consider the following as
desirable information for reports: Inputs (what we used); Activities and
Services (what we did); Outputs (what we produced); Outcomes (what impact
we achieved).
12. Where
can I find NEA's outcomes? TOP
Each funding category includes a section, “Intended Outcomes,”
that lists the NEA-identified outcomes for that category. The outcomes
for all categories are listed on the application form (Basic Information,
Part 2).
13. How
many outcomes should my project have? TOP
Applicants must select one NEA-defined outcome for their
project. The selected outcome, its indicators, and related information
must be presented in the application. Applicants also may elect to identify
in their applications additional outcomes for their projects, along with
their indicators and related information; this is not an application requirement.
14. How
do I report to NEA about the results of my project? TOP
In your grant award package you will receive information
about final reporting requirements. You will be expected to address planned
and achieved outcomes in your final report to the NEA.
15. What
is NEA going to do with my information? Will there be a report that includes
my project? TOP
NEA will use information from projects in a collective
manner. The information sent to us regarding project outcomes will be
aggregated and reflected in reports to Congress and others that summarize
the impact of funded projects during the previous year. In cases where
stories and facts can augment the data, special comments, remarks and
anecdotal information may be part of NEA's reporting.
16. Where
can I get more information? TOP
Refer to "Resources"
on the NEA Web site for additional information concerning outcomes and
outcome-based evaluation.
To clarify the meaning of terms look at the Glossary.
|
