Ep. 103 The Art of Capturing Impact

Sunshine and Stories Episode 103: The Art of Capturing Impact

Episode Summary: 

What happens when one Kate asks another Kate what she thinks about how we capture the impact of what happens during our summer library programs? A whole lot of reflecting on why capturing impact might be meaningful to us, what playful ways might be out there to collect the numbers and stories we can use to measure impact in ways that are relevant to our unique outcomes, and all the bits and bobs we might need to remember to plan for before the summer begins. 

00:00 – Introduction
00:58 – Defining success and capturing impact
02:21 – Collecting numbers and stories
14:00 – Logistics to consider
16:58 – We captured, now what?
22:12 – Book and program highlights
24:09 – Closing  

References:
Collaborative Summer Library Program
5 Steps of Outcome-Based Planning & Evaluation for Public Libraries by Melissa Gross, Cindy Mediavilla, and Virgina A. Walter 
5 Steps of Outcome-Based Planning & Evaluation for Youth Services by Melissa Gross, Cindy Mediavilla, and Virgina A. Walter
Research Institute for Public Libraries (RIPL) 12 Months to Better Library Data
Public Library Association Project Outcome
Your Brain on Art: How the Arts Transform Us by Susan Magsamen & Ivy Ross
The Power of Art: The Human History of Art: From Babylon to New York City by Caroline Campbell

Next Drop-in Chat:
Join us on September 10, 2024 at 11:00 am to noon Mountain Time for our next online Summer Drop-in Chat.

No need to register. Come when you can, stay for as long as you can. See you soon! 

If you work in a library and gave us a listen, please let us know what you think by completing a short feedback survey for this episode. 

Podcast theme music by Transistor.fm. Learn how to start a podcast here. 

This project is made possible in part by a grant from the US Institute of Museum and Library Service. 

Transcript

Sunshine and Stories podcast transcript for Episode 103

[00:00] [music]  

Kate Compton: You're listening to Sunshine and Stories, a Kate-cast from the Colorado State Library's summer design studio with your hosts,   

Kate Brunner: Kate Brunner  

Kate Compton: And Kate Compton.  

Kate Brunner: Tune in every other week as we explore one summer reading topic. From colorful program ideas to community wide engagement to avoiding burnout, we have got you covered.  

Kate Compton: If you have other questions or want to dive into any topic further, check out Libraries Learn.org for our next live summer drop in chat date happening Tuesdays starting this fall.  

[00:42] 

Kate Compton: Did you know when The Beatles came to play Red Rocks on August 26th, 1964, an estimated 10,000 fans were waiting for them at the then Stapleton Airport? That is quite an impact. 

[00:58]  

Kate Compton: OK. In our last episode, we talked about finding the why behind your summer and kind of figuring out your outcomes and want to ask you; how will you know at the end of summer whether you've been successful? 

Kate Brunner: That's a great question and I think you have to answer that question of what you want your summer program to actually do in your library, in your community and the impact you want to have. So, I think that is actually where you have to start like we were kind of talking about in the previous episode, thinking about what success looks like and maybe putting those into some sort of one sentence statement of the outcome you want to have. Or several one sentence statements, if you want to have more than one; if you want to achieve more than one outcome or have more than one particular impact in your summer program. But then, I think the best process to go through or very meaningful process to go through is to look at those and really think about; what would I need to know? What information do I need to have to be able to tell if I've been successful with that particular outcome?  

And so, in outcomes-based planning and evaluation, it's the outcome that drives what you are actually going to spend time. You know it's super valuable. Your time is so valuable during the summer and it's one of our most precious resources during the summer. And so we want to be collecting information that will be the information we need to be able to evaluate whether or not we're being successful. 

[02:21]  

Kate Compton: OK, so you've figured out your why. You've figured out what you need to ask and what you need to know in order to know if you've been successful. Then how do you go about getting that information? 

Kate Brunner: For me, that's the fun part where you get to kind of play with the different tools and you get to think about; What tools could I use to get the information that I want to get? And so I think that information can be numbers and it can also be stories. Those are kind of the two buckets maybe that I think about when I'm thinking about what information do I need to know whether or not something I'm doing is successful? Whether or not I'm achieving the outcomes I wanted to achieve with a particular project? So, thinking how do I get the numbers I want? How do I capture what's happening with the numbers that I want and then how do I capture the stories I want to capture?  

In one episode we talk about trackers. How are people tracking participation in the program, whether that's number of books, read minutes, read a bingo card or using a digital tracker of some kind. Those tools give you some of that number data that you might be looking for. If you feel that that number data is going to be information that's going to be helpful in determining whether or not you've had the impact you want to have, then analytics data that comes from a digital tool as well. I think that you can look at, you know, how many people have created Beanstack accounts or Read Squared accounts. You can look at things like that too. So I think there's the opportunity to capture some of that number data that we're looking for. But I think there's also a lot of other ways we can think creatively about that.  

And I think surveys are one of the things that come to mind really quickly. And there can be some pros and cons of using a survey. If you are using a survey, you are actually asking folks for feedback versus just sort of making assumptions about what they think or feel about your program. So that can really give you some of that story information that is super helpful in telling the story of the impact of your program later overall. 

I think also though designing and conducting an effective survey takes a lot of time and takes effort. So that can be something that can make it seem like another thing that we have to put together and manage for the summer. And survey fatigue is really real. I think I realized I've been getting a lot of e-mail feedback survey requests lately, like pretty much any goods or service vendor I have any contact with personally. And I realize I'm observing my experience of survey fatigue, you know? And if you're asking people to fill out a survey all the time, people are going to start not wanting to participate in your survey because they're just going to be tired of filling out surveys. So I think that’s something to keep in mind there too. 

Kate Compton: I'm hearing you say there's a lot of fatigue around surveys. You have to have a lot of time and resources to design good interviews or surveys. Is there any other ways that we might get some of this more qualitative feedback? 

Kate Brunner: I think a lot of folks have some really creative, playful, maybe a little like, you know, sneaky ways. Like it doesn't look like a survey. It's not a piece of paper we're asking people to fill out. Or it's not a QR code we're asking them to use to access a digital survey that's sort of like the traditional survey format. I almost think of those as kind of covert surveying strategies. 

I think there are so many creative ideas I've seen people out there using and I'm always learning new ones as people come up with them and share them. I really like the talkback board model which actually came out of Anythink to begin with. So it came out of a Colorado library. There are two different types of talkback boards you can use. One is called an open talkback board and one is called a closed talk back board. Those are really where you're putting prompts on a big piece of paper on the wall. Or you can do them in a digital format too. And you may be leaving sticky notes, pens or whatnot if you're asking an open-ended question. That's what that open talkback board is. So you're asking people for more than yes/no, 5, good/bad. You're asking people to answer that question in a sentence. So if people can write their answers on a sticky note and leave it on your talkback board, then you can look at those responses. Then the closed talkback board is more like those short answer kind of things and it's usually something like a series of prompts or like a rating one to five or whatever. Not prompts but sentences like “I learned something today”, “I enjoyed today's program”. Thinking about what is meaningful for you to know again, like which sentences you might want to put there. And then people can put sticky dots on all the ones that apply as they're headed out the door. Or coming into the library, depending on what you're asking and where you're putting it. Or you can ask people to vote yes/no. Or use a Likert scale of one to five or, you know, anything like that, which might be a little bit more concrete, a set of options. And so that's kind of what we talk about when we talk about a closed talkback board. 

But I've also seen people set out bins and or jars and have people vote for different things or different options with puff balls. They get to put a puffball in the jar depending on their answer. Or building lego towers, I think that was one that we did at a summer program workshop that Beth Christ did one year. So yeah, you know, add your lego to the tower, depending on which option you're voting for. Those are kind of interactive display pieces, I think.  

You can also kind of conduct those surveys, like short one-question kind of surveys, via your social media posts. I've seen people do that as well. So just really kind of thinking about how to be playful. Instead of handing to a caregiver with a child who probably needs a snack and a nap on the way out the door from a story time program a paper survey and asking them if they'll fill that out, you could put some sticky dots out and ask them to answer a question on a talkback board instead. And that might be much more manageable for them. 

Kate Compton: How about other tools or options that people have for collecting both kind of stories or numbers? 

Kate Brunner: I think there's a lot of different ways that people are out there doing this which are really, really creative. And any of that number data that you want to grab, you know we kind of talked about trackers and what not, but also program attendance, events held, number of giveaways, like maybe number of books distributed, circulation data to see what the effect of your summer program. If there's more people coming into the library, is that affecting your circulation data? Because maybe that's something you want to know about or an impact you want to have.  Number of take and makes that have gone out the door, number of hits on your website. I think you can think about a lot of different ways to connect or collect different numbers. Just choosing which of those numbers are meaningful for y’all, depending on your community. You are the expert there. So kind of picking out which of those things you want to track.  

And then as far as collecting that story data, I think there's so many different ways to do this, so many different options that people talk about. And I think, you know, we think about certain of these story capturing tools as all formal or they're referred to in kind of academic language but they can be really casual kind of and very easy to implement. And so you know we talk about interviews per se and that sounds very formal, but maybe it's just one question that you're asking people that week as they come to programs or as you talk to them at the desk. Or as you just see them in the library or see them in an outreach event or whatnot. Maybe it's just one question everybody is asking folks. You do have to think about how you're going to collect all that. You know, so as people tell you things, as those people answer that one question, how are you hanging on to those stories? 

I think also we hear in evaluation, the term focus groups, and again that's another one that sounds really formal. But I think if you think a little bit more broadly about it, it could be something like going to a Friends meeting and having conversation with the folks who are attending the Friends meeting that day about what they're thinking about the summer program or what their experiences have been so far if they've engaged it. So you know, there's the opportunity to think about where are you going any way that you could then ask for some time to have a conversation and ask a few questions about summer and their experiences of summer to get those stories there.  

And I would say I think the last category maybe is again another one that sounds really formal. It’s this idea of artifact analysis and is kind of the big fancy term for it. But what that means really is just looking at the things that people are making or the things that you all are making or generating or capturing during the summer. So that could be things that people make in a program in particular and sort of capturing the story of the creativity that is emerging in your community. And then it could be photos of events that you've taken. You know, kind of looking at those photos and wow, people were so excited and engaged. You can totally see that in this picture of this event. And then also I don't know, like when you use bingo cards or passports, I think there is the option to maybe in addition to counting them, counting how many you're seeing come back, you can also look at which squares on the bingo card were the most popular or which were the least popular. Maybe that gives you some information about how to adjust your bingo card next year. Some new ideas to put on it, or ideas that seem super popular that maybe you want to retain for next year. 

Kate Compton: Are there any other considerations that folks need to take into account?  We kind of have the idea around choosing the right tool for what your impact is, but is there anything else when you're choosing what you're doing, you should be thinking about? 

Kate Brunner: Maybe I think it's an opportunity to think about accessibility for two reasons. One, because it is a really great thing to think about. And two, because you don't want to pick a tool and put a lot of effort into using it and then end up with a really low response rate or really low engagement with that particular tool. So I think thinking about are you making that tool accessible and as available as it can be. Making it easy for folks to engage with that particular tool so that people can have the opportunity to share their stories with you. So they're participating in some way that leads to those numbers that you're wanting to capture and being able to put some faith in those numbers that they're meaningful. Are you offering a digital version and an analog version of something? So if you do want to use sort of a traditional paper survey, which there's nothing wrong with using that, are you offering that survey in maybe two different formats? Or in a format that appeals to the portion of your community you're really trying to connect with through that tool? 

Also you can think about the languages that you know, language groups that are in your community and are you able to use a tool that is accessible to folks no matter what language they speak? Or are you able to offer that tool in multiple languages if that's possible? 

[13:54]  

Kate Compton: I love that and I think we also have kind of the consideration of when do you use these tools? Is it always just at the end? Where are you going to use them and who's using them too? What are your thoughts around any of that? 

Kate Brunner: Yeah, I think that really depends on how you're defining success. You know, for your particular community and the information that you need to be able to say you were successful. So maybe that looks like how many people signed up and how many people finished. And so that's kind of your pre-measure and your post-measure. So you're measuring something at the beginning of the summer and at the end of the summer. And then you're comparing those two pieces of information and I'm looking for what that tells me. There may be opportunities there if it seems like the most appropriate strategy for to you for you to take.  

And then I think there's also sort of checking in and taking the pulse throughout the program. An example of that might be putting up some sort of interactive display or doing some sort of one-question casual interview periodically throughout the program. What is manageable for you and what is appropriate for you? Like maybe that's halfway through the program, maybe that's at the end of every program, who knows? It's really going to depend on what you want to try.  

And then also there’s the possibility of looking to collect some of these numbers or collect some of these stories at individual programs or individual events. So that's a great place to think about using that talkback tool. I think the talkback board tool, it really appeals to me to use that at programs and events. I've had a lot of success with that. I've enjoyed using those. This has been a really fun way to capture data; really quickly capture stories or numbers really quickly at a particular program.  

I think you asked about who might be collecting that information. Which I think is a really, really great question to ask because if it's more than one person, then you all want to be on the same page about how you're collecting that data and then where you're putting it. So, if you are keeping track of sign ups and then completions, are you keeping that in the spreadsheet? Are you keeping that in some other tool that everybody who's collecting information has access to? 

If you're thinking about using the one-question strategy, are you doing that in a talkback board or an interactive display? And who's setting those up then? And keeping an eye on those, making sure they have enough sticky dots or enough sticky notes? Or if you're doing it through conversations, then how are you recording that information after you ask somebody that question? Are we all asking people the same question? And then where are we putting those answers that we collect? So just some logistics I think to think about when you're choosing a tool. 

[16:58]  

Kate Compton: So, we've got a lot of information here, Kate. Thinking about what you want to figure out, how you're going to figure that out, who's going to be figuring it out. What other last ideas or thoughts do you have about capturing your impact for the summer? 

Kate Brunner: I think it's great to pull it all together then you know you've got the numbers that you wanted to collect. You've got the stories that you wanted to collect. And so now you can figure out how you want to make those numbers and stories work for you. 

So that does take a little time, building in some time to sort of review what you've collected and then reflect on how your program's gone. And then decide how you're going to use what you've learned in the future. And how you're going to share that information. You know, share that overarching story of your summer program and the impact of your summer program back out with your community somehow.  

I think there's some great ways to do that. If there are numbers you want to highlight or stories you want to highlight, do you have a library newsletter that you could put them in? Or does your library have social media or website where you could put them? Really any storytelling platform you have, you know. We're using a podcast, right? So, to tell stories and to share ideas that way, and to share out the potential impacts of summer and the things we think about and all the creative ideas that are out there. If you're going to partner meetings or friends’ meetings there is an opportunity to ask for a few minutes for this year’s highlights. Like, hey, I'd really like to share highlights from our summer program and celebrate our success, however you defined that success.  

Whether or not that’s sharing numbers, or sharing stories or both, or maybe even creating some sort of infographic or poster or display at the end of the summer and sharing that at a final event if you do one. I think those are all opportunities to sort of collect everything together you've collected; some individual stories and you've collected these numbers. You can share them back out with your community and invite everyone to celebrate what you've put together, what everyone's participated in throughout the summer. And I think that one of the things about sharing your story of the of the summer is that it really gives everybody an opportunity to celebrate. But I think some folks have found that the more they share back out with their community about the impact of their summer program and about what they've learned from the community by engaging the community in that storytelling piece, it has a tendency, it seems, and we've talked, we're talking a lot about data collection and tools for capturing numbers and stories. Here's some you know, now we're going to, like sort of sort of share an anecdote more than anything else.  

But I think folks who have been sharing back out their stories are sometimes finding that the community members are more likely to be willing to continue to give you feedback in the future. Because by sharing those stories out, you're actually really showing the community that you are listening to them, that you're hearing their feedback, that you welcome their feedback, that you welcome their participation in co-designing what is being offered by the library. I think we all like being heard. When you know you're being heard, you're more likely to continue to offer your thoughts and contribute to that conversation. 

Kate Compton: Speaking of sharing stories, I'm excited to hear everyone's thoughts around how they're capturing impact for the summer at our next drop-in session. So check out Libraries Learn.org. Come share your tricks, tips and stories about how you're going to capture your impact this summer.  

Kate Brunner: If you're interested in learning more about this idea of outcomes-based planning and evaluation and other creative tools you can use or how you can match up a tool to the success that you want to have however you're defining success, I think there are some great options out there. For those of us who like reading a book, picking up a book and kind of digging into a topic, there is a really great book called Outcome-Based Planning and Evaluation by Melissa Gross, Cindy Mediavilla, and Virginia Walter. They've written both one that is general outcomes-based planning and evaluation and then I think they also have one that's specifically Youth Services targeted. So those two books are out there as a possible resource.  

And then the Research Institute for Public Libraries has an ongoing webinar series that's running through 2025 and I believe it's called 12 Months to Better Library Data. We will include the link to their website and to the webinar series in our show notes. They started in March of 2024. There are recordings of the earlier sessions and materials available. So depending on when you're listening to this, there may be some live sessions left, but the recordings and materials are there and you can look at the different topics that they dig into and maybe pick and choose the webinars that are that seem most interesting to what you're thinking about when it comes to capturing impact.  

And then I would also want to highlight the Public Library Association's Project Outcome, which is a free tool that any library can use. They actually have some pre-made, pre-designed surveys. So, if you're needing something that you can grab and go for a tool, Project Outcome might be a great resource to look at as well. 

[22:12]  

Kate Brunner: Before we go, let's chat about this episode's book recommendations, which are all about the impact of art on us and our world today. There are two titles for your adult readers this coming summer that I'd like to share. 

The first title is Your Brain on Art: How the Arts Transform Us by Susan Magsamen and Ivy Ross. This book is about neuroaesthetics. Susan is the director of a lab at the Center for Applied Neuroaesthetics at Johns Hopkins University School of Medicine, and Ivy is a director of design at Google. This book is actually a fascinating look at the impact of engaging in artistic activities on our brains and our bodies. It's a fascinating read and you might want to check it out for your adult readers this summer. 

The other book I'm going to mention is The Power of Art: The Human History of Art from Babylon to New York City. This book is by Caroline Campbell. She is a curator and the director of the National Gallery of Ireland and what she takes a look at is not necessarily specific works of art, but the human stories behind art, creativity and innovation in various cities around the world at different points in history. And then the impact that those moments had on the art that was created there. 

Our programming highlight for this episode celebrates the way arts are already alive and well in Colorado libraries. And this week, we're going to talk about the Songwriting Shindig that Gunnison County hosted at the Old Rock Community Library in Crested Butte. They hosted this program in August of 2024, where families got the opportunity to explore a variety of instruments in person and then together, everyone helped compose some brand-new music. So, everyone got to try their hand at being a musical artist. This program is such a beautiful way to build community at the same time you're exploring the arts. 

[24:09] 

Kate Brunner: That's our Kate cast for today, y’all but we hope you'll join the upcoming community drop in discussion.  

Kate Compton: We're hosting these discussions all year long. All Colorado library professionals involved in summer programming are welcome. No need to register, stay for the full hour or just pop in to say hi to colleagues across the state. Details for the next drop-in session can be found in the show notes for this episode and on Libraries Learn.org the Colorado State Library’s continuing education calendar and archive.  

Kate Brunner: We'd like to say thank you to all the Colorado library professionals who shared book recommendations, program successes and other artful ideas with us as we were designing this year's podcast. This project is also made possible by a grant from the US Institute for Museum and Library Services.  

Kate Compton: Thanks for joining us and we'll catch you next time.  

[24:57] [music]