Approach
The measurement and evaluation recommendations comply with the Association for the Measurement and Evaluation of Communication Association’s (AMEC) Barcelona Principles and make use of its integrated evaluation framework.
The Barcelona Principles say that your communications objectives must be directly linked to your organisational objectives.
The primary organisational objective is to improve the energy efficiency of homes.
The primary communication objective is to ensure residents are supportive of this and even welcome it.
However, each situation is different so the specific organisational objectives and communications objectives will vary for different social landlords and depending on the type of energy efficiency work being done.
Methodology
| Activity | Output | Outtake | Outcome | Impact |
| A count of the activity and channels used. | A count of what went out. | The extent to which the audience was aware, understood and remembered it. | The degree which the campaign has actually changed audience opinion, behaviour or attitude. | Quantifiable impact on the organisation’s objectives. |
Example metrics
Activities
- Number of letters sent
- Number of emails sent
- Number of text messages sent
- Number of videos produced
- Number of social media posts
- Number of newsletter articles
- Number of events organised
Outputs
- Number of event attendees
- Number of residents spoken to face-to-face
- Video views
- Email opens
- Visits to web content
- Reach of social media posts
Outtakes
- Survey residents on understanding and opinions on key messages
- Monitor questions (via all channels) to identify knowledge gaps
- Clicks on email links
- Use of QR code
- Social engagement – comments, likes, questions etc
- Qualitative evaluation of face-to-face interaction to identify attitudes and knowledge gaps
- Qualitative evaluation by observation of interaction at events to identify attitudes and knowledge gaps
Outcomes
- Residents agreeing to energy efficiency work in their homes.
- Resident ambassadors recruited
- Testimonials and case studies secured
Impact
- Number of homes completed
- Resident satisfaction scores
- Complaints data
Using the metrics in a measurement framework
As every situation is different don’t just take this list of metrics and use them for your communications. Instead use them as a source of ideas and inspiration for the metrics that are most relevant to your campaign and organisation.
| Activity | Output | Outtake | Outcome |
| Letters to residents | Responses to letters – visits to unique URL, use of QR code, replies etc. | Complying actions to requests in the letter e.g. home visit appointments arranged. | |
| Face to face contact | Number of contacts | Qualitative (opinion-based) scoring on receptiveness to changes. | Complying actions to request.
Follow-up survey to measure opinion. |
| Roadshow | Number of residents attending. | Fun quiz about energy efficiency with small prize draw. | Residents signing up to take part.
Residents volunteering to be ambassadors. |
| Video | Number of views (note that depending where it is hosted/shown, this might include lots of people who aren’t residents). |
Some of the energy efficiency improvements are small changes that residents can make themselves. The organisational impact will be if they do them.
This impact could be measured by residents self-reporting improvements, or tenant liaison officers or other workers observing changes during visits for other purposes.
This section was developed by Stuart Bruce. Here he shares his top tips for evaluating your energy efficiency communications: