After offering communications training opportunities to grantees for five years, the Hewlett Foundation engaged Williams Group, an independent communications design firm, to assess the effectiveness and accomplishments of the programs.

Purpose

  • Better understand the value of communications training programs for grantees
  • Continue to improve how training programs are designed
  • Track program participants and measure how they’ve used what they’ve learned
  • Help other funders learn about approaches to help their grantees improve their communications strategies

Scope

The resulting nonprofit communications training study involved more than 200 individuals representing nonprofit organizations that currently receive or have recently received grant dollars from the Hewlett Foundation. The study included:

  • 181 survey responses from individuals who took part in Hewlett Foundation-sponsored communications training
  • 38 survey responses from individuals who did not receive the training but who broadly represent Hewlett Foundation grantee organizations
  • 40 interviews of past training participants broadly representing grantees who completed surveys; included review of communications planning documentation and materials, if available
  • 10 case studies for greater understanding of individual experiences and results; subjectively selected based on interviews and materials submitted
  • 1 observation of a live three-day training session in 2010

Considering the Data

All training participants over the course of the past five years were invited to complete an online survey. A total of 219 surveys were analyzed as part of this study. This number includes 181 survey responses out of a pool of 332 individuals who took part in all types of training experiences offered through the Hewlett Foundation, as well as 38 survey responses out of a pool of 85 grantee orgaizations that did not receive Hewlett Foundation-sponsored communications training.

More than 50% of all training participants responded to the survey. Results are presented at a confidence level of 95% (within an interval of 5) but are also subject to a non-response bias (i.e., individuals who did not respond to this survey may have different answers than those who did respond). Multiple follow-up methods were used to encourage all training participants to respond to this survey. As reported answers on communications capabilities and knowledge align with pre- and post-training answers to similar questions, it may be inferred that survey results are representative. The small sample of non-trained grantees surveyed is for comparison purposes only (without statistical inference).

Initial analysis showed that participants in specialized training programs, such as training tailored to a particular group or issue of interest or intensive and lengthier training offered to executives only, varied from those receiving the more common training experience. For this reason, numerical data (e.g., charts and tables) presented in this study represent only the survey responses of participants in the “core,” or most common training experience (104 survey responses) and omit responses by those who experienced a specialized training (77 survey responses). Findings from these distinctive experiences are noted where deemed significant.

Additionally, study findings should be interpreted with care and consideration of these three factors:

  • Training varied over time, most significantly in the areas of curriculum, delivery method and duration, and participant selection.
  • Comparative evaluation was limited. Individuals were engaged for this study as many as five years following their participation in training. Efforts to understand their progress over time were limited to comparing survey self-ratings on communications capability and knowledge given at the time of training and more recently, and assessments made during the course of phone interviews and review of present-day communications materials. Organizations that did not participate in training were also asked to complete a survey with similar questions to gauge their communications capability and knowledge, though the size and differences in experience of this comparison group prevent it from constituting a true “control” group.
  • Participation in this study was voluntary. However, due to the power dynamics inherent in the funder-grantee relationship, grantees may have felt obliged to participate in a study offered by their funder and to respond favorably to survey questions in order to maintain good rapport with the funder. To counteract this possibility, grantees were assured that their responses would remain anonymous and—except for case study summaries—would not be individually identified to Foundation staff.