From theory to action: What new data tell us about tracking digital equity milestones

In this episode of the Connected Nation Podcast, we explore a report addressing pivotal questions surrounding the digital equity landscape in the United States.

As the nation invests billions of tax dollars to bridge the connectivity gap, our guests, the authors of "Developing Digital Equity Theory of Change with Tech Goes Home," provide valuable insights. Join us as we delve into the report and seek answers to the crucial question: "How do we measure progress?"

Recommended Links:
DERC Website: https://dercenter.org/
TGH Website: https://www.techgoeshome.org/
Sangha's LinkedIn: https://www.linkedin.com/in/sangha-kang-le/
Colin's LinkedIn: https://www.linkedin.com/in/colinrhinesmith/

Transcript

Copy Transcript Copied

Jessica Denson (00:07):

This is Connected Nation, an award-winning podcast focused on all things broadband from closing the digital divide to improving your internet speeds. We talk technology topics that impact all of us, our families, and our neighborhoods. During our discussions on this podcast, you've heard the terms digital equity and digital inclusion, and we've talked about how critical it is to ensure everyone can access technology that can improve their lives. But as organizations across the US work on this issue and billions of tax dollars are put towards digital equity programs, it's important to ask how do we measure progress? Today I'm talking with the co-authors of a new report that seeks to answer that and other important questions within this space. I'm Jessica Desen, and this is Connected Nation. I'm Jessica Desen, and today my guest are Dr. Colin Rsmith, founder and director of the Digital Equity Research Center at the Metropolitan New York Library Council and Zaa Kle, an advocacy research specialist at Tech Goes Home. The pair helped co-author a recent report titled Developing Digital Equity Theory of Change With Tech Goes Home. Welcome. Dr. Colin Rhinesmith (01:17):

Hello for Jessica Denson (01:17):

Having us. Yeah, thank you. I really appreciate you both joining us. This is such an important topic. Often when we talk about digital equity and digital inclusion, it can be very confusing how to measure that, so I'm excited to dive into this report. Before we do that though, I like to give our audience an idea of your background and your current role so they know who we're talking with today. So let's begin with you, Dr. Ryan Smith. Share some of your background in the digital equity space and what it is that interests you about this issue. Dr. Colin Rhinesmith (01:50):

Yeah, thank you so much and thank you so much for having us today. So one of the things that really drew me to this space, one of the reasons why I got involved is because I actually started working in a community technology center, like a public library or other community center many, many years ago before I became an academic and a researcher. So I worked as a community media and technology manager at Cambridge Community Television in Massachusetts. And that was really my introduction to these issues, how the digital divide impacts people who cannot afford access to technology, but also folks who need the skills as well to be successful in their everyday life. So it was really through this experience that I became aware of broader initiatives like we're talking about today, related to how do we measure the impact of these efforts, like the work that I did back then.
(02:51)
Building on that, just briefly, I'll just tell you that I went from that experience to becoming an academic 14 years ago, the last time around when the National Telecommunications and Information Administration launched the broadband technology opportunities program, and that was under the Obama administration. It's sort of the internet for all that we talk about today, but the last time around and back then I was thinking, wow, we're never going to see this again in our lifetime. And that was 14 years ago, and here we are again today with Internet for All and a 65 billion investment. Back then, I think it was about 7 billion. So that was really my introduction to this space in my interest. And so I went really again, just sort of from being a practitioner to a researcher and looking at these issues of measurement. Jessica Denson (03:45):

And just to give an idea of your background, you went to Emerson College, you were a research fellow for Benton, which is really well-known research group. Why do you love that so much? The research side of things? Dr. Colin Rhinesmith (04:00):

Yeah, absolutely. I mean, so I think one of the things that I love about research, to answer your question is that I think the opportunity to sort of elevate stories, stories, everyday stories of people who are struggling either that don't have access to the internet, these stories need to be told people's everyday experiences with technology need to be told. And as a researcher, it's really our mission at the Digital Equity Research Center where I work now to elevate these stories, not only of people who are impacted by the digital divide, but the organizations also like Tecos Home that are working every day to support people's access to the internet and to be successful in their lives. Jessica Denson (04:47):

And you said you work there, but you actually founded the Digital Equity Research Center, didn't you? Dr. Colin Rhinesmith (04:52):

I did, yes. Thank you. Yeah, so it's fairly recent. Just really briefly, I can tell you that I was very fortunate. I left formerly, I was a professor at the School of Library and Information Science at Simmons University in Boston for a number of years. And really when Internet for All was sort of ramping up and it looked like the whole country was going to be focused on this issue, I had the opportunity to move into research full-time, starting the center at the Metropolitan New York Library Council. They invited me to start the center there and was really fortunate. So it's been just over, I guess about a year and a half. Oh, coming up to two years now on the center being around. So it's still new and exciting and it's, it's been great to work there. Jessica Denson (05:45):

You mentioned the Obama era investment in this issue. I wasn't working in this space at the time, but I've heard a lot from colleagues and peers and others that have worked in this space that part of the reason it didn't work is because there wasn't a one place stopping place where everybody was doing it the same way and measuring it the same way. Is that kind of why you were drawn to do this again then because you want some answers and want things done better? Dr. Colin Rhinesmith (06:16):

Yeah, Jessica Denson (06:17):

I'm not be putting you on the spot there, sorry. Dr. Colin Rhinesmith (06:19):

No, well, I mean, so a yes and no to your characterization of the program. I wouldn't say that it didn't work. What I would say is that I agree with you and how you characterize it, that it was siloed, it was a siloed approach to the issues and that didn't work. So you're absolutely right. It was infrastructure. On the one hand, it was sort of more of the digital equity thing separately, I think yes, today with Internet for All, I think the NTIA and what we learned from BO is that you need a coordinated effort and it has to all be working together in order to be successful. And just lastly, on that point, I'd say one of the huge successes though is what grew out of it is that communities became organized to address the issue and that's what launched the National Digital Inclusion Alliance and also launched a better way to address this problem. So I tend to look at the bright side, and so I think we learned a lot from the Btop days that I have made this new effort I think will be much more successful. Jessica Denson (07:25):

Yeah, that's a fair point. It wasn't necessarily that it wasn't successful, it's just that here we are again 14 years later working on this issue still and it's such a massive issue. There's no one way to do it. But yeah, I agree. A coordinated effort is really what I was gunning for there. I do want to, you work with the Library council, the New York Library Council before I get to Sangha. Libraries really have a critical role in this work in both urban and rural settings, don't you think? Yes. Is that something that the Library council really recognized? Dr. Colin Rhinesmith (08:01):

Yeah, absolutely. Just so folks know, I mean the Metropolitan New York Library Council is actually one of New York's nine regional library councils, and this is very similar states across the country have a similar sort of setup where we have a state library agency, but under that we have a number of local, more regional library organizations that are there to provide support, particularly to those smaller libraries where there might be just one or two people working there very under-resourced, maybe in rural areas or otherwise. So these are really critical organizations that really help to provide additional supports and services like access to technology support and smaller communities, but also professional development opportunities as well. So yes, and as part of all of that addressing the digital divide has been always been because it's about providing access to information is a huge role for Metro and many other regional library councils across the country. Jessica Denson (08:59):

And a lot of people use the library if they don't have access at home just to even apply for a job or get government services. There's so many important things. I've also heard from a lot of librarians when we celebrated our 20th anniversary, I met with some librarians and their thing was teaching people even basic digital skills was an issue. Absolutely, absolutely. Such an important role, SGA Conley. Let's turn the discussion now to your background. You also have done some pretty impressive work in the research and advocacy arena. Tell us a little bit about your path to tech goes home. Sangha Kang-Le (09:36):

Sure. So my path was a little bit more winding to Tech goes home into the world of digital equity, but one thing I want to mention is that I went to Simmons College, Simmons University while Colin was there and unfortunately wasn't able to meet him at the time, which is a bummer for small Jessica Denson (09:54):

World. Yeah, I know. Sangha Kang-Le (09:57):

But I started off my career in state politics. So I went through a fellowship program that was for young Asian American women who were interested in pursuing a career in politics, which was a really wonderful supportive way to get introduced to that world. So I developed a great relationship with the state senator who I worked with at the time, and moved from there into the world of political campaigns. So worked on some state political campaigns as well as the presidential. At the same time, I think I was developing a real interest in research and particularly in research that could be used to inform policy. So I worked as a research assistant on a couple projects, one of which allowed me to do focus group research on the topic of gender equity and work-life balance in China and in South Korea, which was really exciting. And then I returned back to the Asian American Women's Political Initiative, that fellowship program to work as a policy and research advisor.
(11:03)
Actually, one of the critical issues that was big at the time that I was working in the State House and continued to be a real way that Asian Americans were engaging with the political process moving forward was the issue of data disaggregation, which is a jargony term, but essentially has to do with the way that the state collects and represents data on Asian Americans in all racial categories. And I think that was a pretty formative experience for me and informed the way I engaged with policy and with public data in the rest of my career. And then I also worked doing some research and policy consulting for a women of color-led economic development fund before finding my way to Tech Goes home. Jessica Denson (11:50):

So you've done a lot of research and advocacy it sounds like, for marginalized populations and even immigrant populations. You said that it really changed your approach to advocacy or how things were gathered when it comes to data and shared with data. Explain that a little bit more. I would love to hear what was learned there and what's something that maybe we can all take away from that. Sangha Kang-Le (12:18):

Absolutely. Yeah, so I've been very fortunate from the beginning again that state senator I worked for in my very first job represented some of the most racially diverse neighborhoods in Boston. And one of her big efforts or how she spent a lot of her time that was surprisingly difficult, even for the time, was just getting those people's stories heard and elevated. I think folks who are marginalized, as many of us know who are in this field, are often the least likely to make their voices heard through civic engagement through voting, and there can therefore be, their stories can go unrepresented in the policymaking process. And that also extends to public data and research. So the data disaggregation issue I was speaking of essentially the most detailed category that the state collects data on for Asian Americans is just the umbrella category of Asian American. And when you collect data in that way, Asian Americans appear to have high rates of college educational attainment, high median incomes, good health outcomes, et cetera. But if you disaggregate, so collect data on individual categories, Cambodian, Vietnamese, Chinese, Korean, some of those Asian American groups have some of the highest rates of, or excuse me, lowest rates of educational attainment, specific health outcomes, et cetera. And those stories, it's impossible to do policymaking on those issues if they're completely invisible. Jessica Denson (14:02):

So basically people were being grouped into this one massive group and it's actually a cross section of people that come from all kinds of different backgrounds, all kinds of different places. And so they have different points of view and different experiences, is that what you're saying? Sangha Kang-Le (14:16):

Yeah, exactly. If you kind of group folks together in this specific context, you miss some of the nuances of specific communities and their needs. Jessica Denson (14:26):

Sounds like important research, I have to say for both of you guys looking at your backgrounds, I'm like, oh gosh, hopefully I can stand toe to toe and do strong interview for this. So you're doing great. You're doing great. Oh good. So now that we've got a little bit of your background, Sangha, can you tell us a little bit about what Tech Goes Home is and what you do with them? Sangha Kang-Le (14:51):

Absolutely. So Tech Goes Home is a digital equity nonprofit. It's based, it has its roots in the Boston area but serves Massachusetts and beyond. We have kind of a three-pronged model to the digital to addressing the digital divide. So the first is skills and skills training. So folks need to be able to make use of digital tools and of the internet. So to achieve this, we work with community partners including libraries, community health centers, public schools, et cetera, organizations that already have a trusted touch in their communities and their neighborhoods. And we train the staff at those organizations to deliver digital skills courses. So through those trusted partners, we put on digital skills courses across the state and every learner who graduates, that's free of cost for a learner, and every learner who graduates a Tech Goes Home course earns a free laptop and a year of internet access that is paid for by Tech goes home.
(15:55)
And recently we've really been dedicating efforts to try to transition learners onto the affordable connectivity program benefit. So that's kind of the first prong. The second of course is devices and the third is internet access. And then alongside that, we've been operating courses for over 20 years. We have over 130 active partners, but we know that the Digital divide is an issue that's rooted in other systemic inequalities. So that's where our advocacy work comes in and that's where I sit. So we try to work with and influence policy makers, city officials, and other stakeholders to really advocate for the communities that we serve and bring digital equity to the forefront. Jessica Denson (16:47):

Just as a note for our audience, the Affordable Connectivity Program is a program that provides a discount on internet for low-income families and households, individuals. It is actually about to run out of money and there's a big fight or I dunno if fight is the right word, but there is a discussion happening at Congress now to extend that and preserve it, making it permanent. It'll be interesting to see what happens with that. On the website it says that within Tech Goes Home Mission is a focus on addressing digital exclusion. Define what digital exclusion means in this context. Would you sga? Sangha Kang-Le (17:27):

Absolutely. So I mean, first I think it's important to define digital inclusion. So the National Digital Inclusion Alliance, NDIA has a definition that tech goes home really likes, which is digital equity is a condition in which all individuals and communities have the capacity needed for full participation in our society, democracy and economy. And they say digital equity is necessary for civic and cultural participation, employment learning, and access to essential services. So I think that there are a lot of myths around digital equity and access. A lot of folks have a sense that most if not all people have access to the internet or have computers. And it's surprising when we say that I think it's like almost 17% of folks in Massachusetts lack a computer. Almost 18% lack internet. So digital exclusion, I think to Tech Goes Home is the overlapping systems and barriers that are preventing folks from having that access.
(18:34)
And I think it's said a lot that internet is no longer a nice to have, it's a need to have. What that means really on the ground is that if you think about all of the things that you need the internet to achieve, so someone without either internet access a device to use the internet or the skills to navigate that device isn't able to attend online school. They're not able to apply for jobs for most employers or search for jobs, look at job portals, they might have difficulty managing their government benefits. They can't use telehealth. So digital exclusion also contributes to exclusion and marginalization across many other categories and touching other impact areas. Jessica Denson (19:18):

The pandemic definitely changed people's point of view on that. I would have conversations with journalists saying, isn't this a luxury? No, it's something everybody needs. The pandemic changed that point of view. I think people understand that we need to have it. But let's dive into this recent report. The title is Developing Digital Equity Theory of Change With Tech Goes Home. So Dr. Ryan Smith, we're going to hand it over to you now for a moment. What was the catalyst or purpose for doing this study? Dr. Colin Rhinesmith (19:49):

Yeah, so there's kind of a fun background I think behind this report and this work is that actually several months before I had an opportunity to start working with Sangha and the rest of the team at Tecos Home, while I was still at Simmons University, I worked with Tecos Home actually in a class that I was teaching. And I had known Dan Noyce, who's the executive director of Tecos Home for a few years. And before a semester, I think it was a fall or spring semester at Simmons University, I was teaching a course that was what we might call the service learning course where students get an opportunity to work in and with community organizations to help address or a challenge or a problem that a local organization in the Boston area was working on. But usually the way those courses work is students get divided up and they go work with different organizations and then they come back together and talk about it.
(20:48)
And it's a really important part of the learning process, I think particularly for our library and information science students who many of whom are going to go out and work with communities and work in libraries themselves. So it sort of sets students up for success by giving 'em them that opportunity before they go out into the world after a program. So I had approached Dan before one semester and I said, Hey Dan, I know that you've worked with students before, but what if you had my entire class to work with and you had every student in my class to work on a problem with Tecos Home? What would that look like? What would that be? And that really was the sort of catalyst or starting point for the work that we did together because what my students did with Tecos Home was begin to look at the data that Tecos Home were gathering and to begin to look at and to develop essentially what this work at the end of.
(21:44)
It ended into a report at the end of the semester that students created talking about all the ways that Tecos Home does data gathering and analysis and helped Tecos home to think about how do they show the outcomes and impact of their work. So it was really exciting that we actually had some previous work that we could look at, but what was different this time and how we got started is Tecos Home approached me for this project and said, could we develop a theory of change and an evaluation framework building on the work that my students had done? So it was kind of the next step of that work with my students. What was different though also about this, and I'm sure Sangha can talk about this as well, because essentially Sangha hired me to do it, so Sangha's the boss. So if we have any questions, we can go back to Sangha and check this out.
(22:39)
We'll come to her. Yeah, absolutely. But what I would also add, and what's exciting about it is we decided together that we would really become co-researchers. That it wouldn't just be me dropping in as an outside researcher to do this work, to put Tecos home under a microscope, which is often the way that research is done. You bring experts in, experts who know how to solve problems and to solve communities problems. But what's the approach that we took? We took a participatory action research approach. And what that means is that researchers like myself approach SGA who is an expert in her own right to say, well, you're an expert. How can we work together as researchers to both solve the problem that we're looking at? So we use participatory action research to develop a theory of change, which is essentially what's otherwise known as a logic model is a piece of that.
(23:44)
It's a way that an organization can understand the outcomes and impacts of its work by working backwards to say, well, what are the activities that are necessary that lead to certain what are called outputs or numbers of, in the case of digital literacy and digital equity programs, so say the number of devices that are distributed or the number of people that come to classes. Those are typically things that are counted towards looking at things like changes in people's lives, more outcomes that we can help understand how are we doing in our work. And so that was really the sort of basis for and starting point for our work is to say, how can we set Tecos home up with even more success with a theory of change that's comprehensive, that represents all the work that they do, but is done in a way that's inclusive and has staff like Sangha and others who are on staff to be co-researchers in the design process. And that's important because obviously you want the expertise and knowledge from people who are closest to the problems that would be Sangha and her staff who work with community members every day. They know a lot more than I do about the needs of the communities that work with. So that was really the difference in how we went about this approach and the starting point for the work. Jessica Denson (25:06):

So essentially a theory of change when you talk about digital equity theory of change is if we do this, this is what should happen. Absolutely. That's our theory. Okay. I understand you said it much Dr. Colin Rhinesmith (25:18):

Better than I did. Much more succinctly. Thank you. Jessica Denson (25:21):

I've been reading through this for a while, so I've been learning, I've been learning Sangha if you would, I know he mentioned that they took some of the data that Tech Goes Home gathered. What were some of the key groups or areas that you focused on when you gathered data for this research project? Sangha Kang-Le (25:39):

Sure. I appreciate Colin referring to my expertise, but it was definitely the expertise of the team as a whole, including folks from the advocacy team department as well as our program facing folks and those who work directly with our community partners. So we wanted to make sure that we engaged the entire Tech Goes home team in the development of this theory of change. And then at the same time, the expertise we were really hoping to capture and to elevate is the lived experience of our learners and of our community instructors who deliver those courses. And Tech Goes Home believes, and this is mentioned as an element of our recommendations at the end of the report in the importance of compensating learners and instructors for their time when we engaged them in research and information gathering of this sort.
(26:31)
And we wanted to do it very intentionally. So that's where we really wanted to lean on the expertise and the knowledge of Colin and his team, the Digital Equity Research Center. It was really only through this researcher practitioner collaboration that we could develop or me devote all of the necessary time, energy, resources to engaging our community in a really thorough way. So a short answer to your question is we wanted to make sure we included the voices of the team. We also wanted to make sure we included the voices of our community instructors and of our learners, the folks who go through and graduate our programs. And then the final category that we engaged was other digital equity organizations. So we did a series of interviews, qualitative interviews with leaders, executive directors, CEOs of digital equity organizations and nonprofits across the country. Jessica Denson (27:33):

And I like that you mentioned different groups because you really did break down your recommendations in two categories, if you will, the Tech Goes Home specific category and additional insights for other digital equity organizations that they could also use. Staying with You Sangha, if you could talk a bit about what you learned for Tech Goes Home specifically, and then we could pass it to Dr. Ryan Smith for some broader findings that aren't specific to Tech Goes Home. Sangha Kang-Le (28:06):

Absolutely. So this collaboration with DERC was hugely valuable, I think to Tech Goes Home. So Colin mentioned some of the context and what our goals had been in pursuing this and setting out on this project. I think some other context that is helpful to understand just how valuable this was to TGH is the context of government funding. So we're in a moment right now where there's already been a significant amount of federal funding allocated through the Infrastructure and investment jobs Act. I think that's right. IJ, Jessica Denson (28:43):

Yes, IJA. Right Sangha Kang-Le (28:47):

Significant portion of funding set aside for states to use towards broadband infrastructure and digital equity. And then there's even more funding coming down the pipe. And in all of these situations, data collection and evaluation are huge components of creating a digital equity plan, evaluating which organizations and entities should receive the funding and in evaluating the success of each of these initiatives in actually closing the digital divide and making a difference in folks' lives. So we knew as an organization that we would need to really buff up and be very intentional about our data collection and evaluation methods if we were going to be competitive for this funding, and also help lend our expertise and our perspective to state planning entities as they were figuring out how to distribute and use these funds. So I think some of the really valuable findings that came out of the study, first of all, having a logic model in general is huge in my role.
(29:58)
I talk a lot with external stakeholders, whether it's a policymaker, an internet service provider, whoever and I might understand exactly, if we do X, then Y will happen, as you say. But to have something that's based in research and that's validated and that incorporates or represents almost all of the components of our program and our outputs and the changes we're creating that we think are important goes a long way in making our case as we're talking to folks who may be unfamiliar with Tech goes home and may be unfamiliar with digital equity as a concept. So as part of this moment where we're getting unprecedented amounts of federal funding and there's more of a spotlight on digital equity coming out of the pandemic Tech goes home is expanding I think at a rate that we've never seen before to other municipalities and cities. And to be able to take with us a really solid shared understanding of our logic model and the way we create change, I think helps us to ensure that the core of tech goes home is protected and the quality of our programs are maintained as we're reaching more and more people. Jessica Denson (31:24):

And for you, Dr. Ryan Smith, what would you say on the larger scale for other digital inclusion organizations or some of the additional insights that you had? Dr. Colin Rhinesmith (31:35):

Yeah, absolutely. And I appreciated how SGA also connected this, while this was a project for Tecos Home because of this moment that we're in, there's a lot of the findings from the report we feel can be helpful in exactly the ways that SGA described. So really it's within that context that some of the additional insights from our work with other organizations are things that are actually not new issues. But I think some of the things that are not new, we'll talk about it in a minute, but I just wanted to underscore the fact that this is very, very challenging work showing that X, now, I don't know exactly how you said it, but how X leads to YZ type of thing. It's very hard. Showing broader outcomes and impacts of work is hard and I'll talk about Y, but that can be frustrating for people.
(32:29)
So we wanted to really describe why. So the first point that was the real takeaway that we learned from talking with people was that measuring broader outcomes is challenging. It's really hard to show one thing. We do know a lot of any, I would say pretty much every organization that's working to address the digital divide knows that the impact of their work because they're working closely with their community. So someone will come in, they might need help with a resume. This happens in libraries and other organizations. And so a librarian might help them to fill out their resume, complete their resume online and submit it for a job. And that person might come back to the library and say, Hey, thank you so much for your help. I got a job. And that's direct outcome and result from that work. What gets challenging is when you collect that data on a larger scale with a lot more people, it gets challenging to show that straight line, that direct line that say that librarian and that experience on that one day was the only reason, say that someone got a job.
(33:49)
And it's tricky because our lives are complex and so there are lots of things that people rely on for support to be successful in their lives. I know I do, and all of us do probably on this call. So for someone who is low income and relies on a public library, and you might have those everyday stories, but it does get challenging to measure these broader outcomes. And the other thing too is because broader outcomes are often measured over time, you might say, because a change in someone's life, you might need a year, you might need an entire year or two years or five years to say, the result of my education led to all these things. And a lot of other things can happen during that time. So measuring broader outcomes is challenging, and we heard that from all the organizations that we talked about.
(34:40)
And so we provide some specific examples and stories in the report about how people talked about that. So it's important for us to realize, for states to realize as they measure outcomes of these investments and for the federal government and that we all develop a shared language and way of talking about what success looks like in this work. So that was one area we can go deeper in that, but I'll just lemme go to these other two points real quickly that are a little more straightforward. So the second one is that the software actually, the ways that we evaluate outcomes and impacts often relies on sophisticated software that you can kind of collect data, you can measure it and analyze it over time. This is expensive. It costs a lot of money to sort of purchase these tools that a smaller nonprofit organization that's often the organizations that are working closely with communities to collect the data maybe for states.
(35:31)
And the federal government can't often afford this type of software because it's very sophisticated programs like Salesforce and other technologies. Even if you have the money to purchase them, they often need to be configured in a way that they're relevant to the local context. So program evaluation software is expensive. That's number two. Number three, people really do need to work closely with everyone involved in these programs to really get a comprehensive picture to know what worked. So it's not simply as easy as sort of like I said before, outputs and a logic model are about we can count things, right? The number of classes that somebody went to, the number of devices that were distributed, this is great, but policymakers in Washington are most likely going to ask the National Telecommunications and Information Administration a few years from now when they say what is the result of all the investment?
(36:35)
They want to know beyond the numbers, right? Yes, in aggregate big numbers, it's great, but what was the impact on people's lives measuring that is challenging? And so part of that process, I think what we're we talk about in the report one way forward is we say that we should partner with those who are working closely with communities to gather data in a way that show the impact in people's lives. So the stories, like I said before, we need more partnerships in data collection. So evaluating programs with community partners closely like Tech Goes Home, they work closely with a number of different community partners that offer their programs and services. It's work, it's extra work and may need additional funding to support that effort. Jessica Denson (37:24):

And it's extra work, like you said, for these organizations that are sometimes very small, they're community organizations, they deal with a lot of volunteers. So there are some challenges there. What about some actionable steps? I know that we talked about the findings, but the report, what was really great about the report were there were some actionable things that groups could do. So Dr. Ryan Smith, since we just talked about the overarching point of view with that, with digital equity organizations, could you continue with that on the actionable steps and then we'll hand it off to you Sangha for what Tech goes home can really do as well? Dr. Colin Rhinesmith (38:07):

Sure. Yeah. So this is a message for digital equity organizations, but I think is also important. We do have other recommendations too for state, for state and federal policy makers as well. But for other digital equity organizations that will be, as I said, will play a major role in measuring the success of these investments is it's important for time, money, and effort to be allocated inside of organization in order to capture these insights and expertise of community members and their everyday experiences. There's a focus on equity with this work, and that is what equity looks like when we actually feature and take the time to highlight people's expertise in the community like SGA was saying earlier, but that takes extra time and potentially money and efforts to do that. The second point is that we also want to make sure that people are engaged when we are engaging community members to provide their experiences in their native languages.
(39:14)
I was just involved with just wrapped up working on New Hampshire's state digital equity plan and in some parts of the state there are communities where there's over 60 plus languages spoken in New Hampshire, which I think a lot of people might think is predominantly white rural state. We have a lot of diversity in New Hampshire as well. And so making sure that people have an opportunity to respond when they're asked to fill out a survey or participate in any kind of data gathering needs to be available in their native languages. And then also another, on that same note, something that I really learned from working closely with Sangha and Tecos Home because they again work closely with their communities that it's important to work with funders to balance reporting requirements, say from a grant with sensitivity to the needs of community members. Privacy issues are a whole topic we could probably have a separate conversation on, but not everyone feels comfortable sharing their personal information in a survey that we might as a researcher as researchers, Sangha and I might ask sort of demographic information, what's the annual household income, race, gender, other sort of personal information that helps us understand how our programs are working that can be hard to gather that information.
(40:44)
So that's when there are what we say in the report for other digital equity organizations in the broader field that we really need to balance these sensitivities and really work closely. The best way to get to that is by working with communities themselves to really understand how they feel comfortable sharing their personal information. Jessica Denson (41:05):

Asanga, I think it's interesting that you touched on that Dr. Rsmith touched on the native languages thing since you made a point of that with your earlier research and your background, the importance of acknowledging difference groups. So let's touch on what Tech Goes Home really found. And also Dr. Ryan Smith mentioned what lawmakers should really take away from this or what they can learn from it. So if you could also expand upon that, I would really appreciate it. Sangha Kang-Le (41:36):

Absolutely. I would love to just add just one point to what Colin said, which I completely with particularly on the points of engaging participants in their native languages and in having a sensitivity to privacy concerns exactly as you said. I think that's something that Tech Goes Home is a huge takeaway for us through the course of this project. And I think it's going to be really useful for anyone who's engaging in data collection on digital equity programs. So for us, only about 40% of our learners speak English as a primary language. And so Tecos Home has an existing kind of evaluation infrastructure where we do an annual follow-up survey to ask folks about their experience with the program and how they're using their newfound digital skills and tools. And what we found is that to really actually capture the experiences of that, those 60% of our graduates who speak English as a second language just distributing the survey over email or online isn't going to cut it.
(42:45)
So what we've had to do and what this report is a huge takeaway for us is that we have to strengthen this arm of our programming is do really direct outreach work in native languages for evaluation specifically. So for us, that looks like literally hiring someone over the summer to do, to work full-time on calling folks over the phone in their native language and ask them to complete the survey in that format. So just to really drive home the point that I think many organizations, I would hope we've heard this said, acknowledge the value and see the value in engaging their participants and the people with lived experiences when they're evaluating these programs. But it does take a lot of resources and a lot of time, and that's something that we really want to be clear as a finding in the report. And then to your question about the actionable steps for Tech Goes Home and for policymakers, for Tech Goes Home, I think one thing we found is that we need to gather specific data on the outcomes that working with us and our programs create for those partner organizations that we're working with.
(43:59)
So not specifically for the learners or for the instructors, but actually for those organizations. So TechCo Home staff have a strong kind of sense over our years of work that partnering with TechCo Home actually strengthens our partner organization's ability to achieve their own missions. One example, we've talked a fair bit about libraries today. Something we hear all the time is that librarians spend a lot of their time doing tech support for the folks that come in. And if we can train them, equip them to deliver a digital skills course, that's then time that librarians can actually devote to their work and is freed up for them to establish programs and provide other supports for the community. But this is something that we need to specifically engage and specifically gather data on moving forward. And then another piece is about collecting data on advocacy outcomes. So I mentioned that advocacy sits right alongside our three-pronged approach as pretty close to the core of what tech goes home does.
(45:13)
But advocacy outcomes are famously hard to measure. And members of the advocacy team at Tech Goes Home. We're kind of the key partners in this research project and we have a strong understanding of the work that we do. But I think that we identified that although we know that addressing the digital divide requires tackling some of those structural issues, that's not something that we have strong data to validate. And so a big action item for us moving forward is to really articulate and make those connections, database connections for external stakeholders. And then finally, I think I mentioned the value of a researcher practitioner collaboration. I think we want to continue that work as we collect some of the data that I just mentioned, whether it's community level impacts or specific outcomes created for partner organizations or advocacy outcomes. I think we found it a really valuable process to combine the expertise of a research team and of a practitioner. Jessica Denson (46:28):

So really for you, this is just the tip of the iceberg. A lot more needs to be done. Sangha Kang-Le (46:34):

Yeah, I mean unfortunately or fortunately. So this often is the result of a research project. It tells you all of the things that you need to keep doing and that you don't have the data on yet. But that's what we love about it. Jessica Denson (46:49):

Alright. I find you both so fascinating, but I know I need to let you go eventually. So let's talk about the future. I'd love to hear for each of you what you have next for your organizations. I know Dr. Ryan Smith with the Digital Equity Research Center. What's next for that organization in your mind? Dr. Colin Rhinesmith (47:11):

Yeah, thank you so much. Yeah, I mean, actually picking right up on what SSGA was saying, I mean, I think that we're going to know so much more over the next four years in particular with this amazing national coordinated investment where states actually control over how this work happens because they know what works best in their state and the different regions within their state. So to the point of needing more research, there's a lot more work to do just in keeping up and tracking sort of how these investments will work. And also the structure of, for example, one of the other things that I've been looking at in my work is looking at the role of coalitions, digital inclusion and digital equity coalitions since the pandemic and in response to the pandemic communities came together to say, how do we make sure kids have devices and internet access when schools are closed? And lots of different people came together in coalitions to do this work. The NTIA has realized that coalitions are going to play a huge role in important role in the success of this program. So that's one area in particular that I'll be focused on. I'm actually writing a book right now on that. It's called Digital Equity Ecosystems. It'll be out next year in 2025. And that will really focus on the role of coalitions and making all this work successful. Jessica Denson (48:41):

Make sure you contact me, we'll get you on the podcast to talk about that when it comes Dr. Colin Rhinesmith (48:44):

Out. Fantastic, thank you. Jessica Denson (48:47):

And for you, sga, what do you see ahead for 2024 for Tech Goes Home? Sangha Kang-Le (48:53):

Yeah, so 2024 is an exciting year for Tech Goes Home. We've been engaged in a strategic planning process over the last year, so this is the first year we'll be implementing it. As I mentioned previously, tech goes home. A big element of this coming year is going to be expansion again. Our roots and our focus has been in Greater Boston, but there are folks across the state and the country that we think could really benefit from these kind of digital skills supports and courses. And so we've been fortunate to receive state funding that's going to allow us to expand to 13 additional municipalities and cities across the state. So we're really interested in the process of learning and iterating and adopting our programs to fit the needs of those communities. And then at the same time, as I mentioned, a big focus for the advocacy team at Tecos Home is going to be the distribution and planning of these buckets of federal funding that are coming.
(49:53)
I think we feel they have a huge potential to make a real dent in the digital divide if they are allocated and evaluated and planned with the voices of community members at the center. So yeah, I'd love to touch also on the recommendations that we laid out in the report for policymakers specifically. I think one thing we'll be doing in 2024 is bringing those recommendations to those policymakers doors as we're expanding and making new connections. So we really believe that the success lot of funding is going towards broadband infrastructure programs, building up the internet infrastructure that's necessary to get an internet connection in the first place, which is of course hugely important. But one of our findings from this report is that the success of those programs relies on funding for digital equity, specifically digital equity being all three elements of tech goes home programs, not just the internet, but also support for getting internet equipped devices.
(50:57)
And that hands-on digital skills training. And we're also really going to advocate, I think that government entities that are allocating this funding set aside, some of it that organizations like Tech Goes Home, but specifically like our community partners who are already at capacity doing really wonderful work in their neighborhoods and their communities. Setting aside funding that those organizations can use to engage a research team like DERC and to conduct program evaluation in the holistic, linguistically culturally responsive ways that we've laid out in the report. And also to provide technical assistance on program evaluation for those organizations. So I think 2024 is a really exciting year for the digital equity world and for Tech Goes Home. Jessica Denson (51:53):

I love that idea about having some research money set aside for those resources. So this report, I'll be including a link of the report and descriptions of this podcast. Before I let you go, I'd like you each to answer one last question. Dr. Ryan Smith will let you start and we'll let SGA have the final word of the day. But what is the one thing that you hope other organizations or leadership take away from this report? Dr. Colin Rhinesmith (52:24):

Yeah, well, thank you so much and thanks for inviting us to be here. While a lot of what we were talking about today is fairly complicated and technical and digital equity itself often tends to focus on technology. At the end of the day, digital equity in this report and what we're talking about, it's all about people. It's about your neighbors, it's about people in your community, and it's really about how we can make our community stronger and more connected both technically and socially. And that's a positive thing, not only for our quality of life, for individuals, for our communities, but also for our society, for our democracy. I think we need this work because it's important to make sure we're all successful in our lives. Jessica Denson (53:10):

And for you, Sangha. Sangha Kang-Le (53:12):

Yeah. And again, thank you so much for having us. It's been a pleasure, but I'm right there with Colin. I think many organizations kind of see, especially community organizations, might see research and evaluation as a burden. But hopefully one of the takeaways from this report is that if you have the right resources and engage really thoughtfully with the research process, asking community members and participants about their experiences can teach you things about your program that you didn't know and really strengthen your understanding of the core pieces that you're already operating under. And so I think the point I would really want to drive home is just reiterating that for digital equity initiatives to succeed and for the funding that's coming down from the federal government to have a real impact, we have to engage community members in the planning and implementation of all of those programs and dollars. And it can be a really fulfilling, exciting, and informative process. And I hope that the recommendations will help and guide those programs and make that a possibility for stakeholders of all kinds. Jessica Denson (54:38):

Well, that's a wonderful place to leave it for today. I'd love to get an update further down the line on how things are going from each of you. Thank you both for taking your time with me today and explaining this report to us. Dr. Colin Rhinesmith (54:50):

Thank you so much for having us. Sangha Kang-Le (54:52):

Thank you so much Jessica Denson (55:02):

Again. I've been talking with Sangha Kong LE and an advocacy research specialist at Tech Goes Home. And Dr. Colin Rsmith, founder and director of Digital Equity Research Center at the Metropolitan New York Library Council, the Para helped co-author a recent report titled Developing Digital Equity Theory of Change With Tech Goes Home. I'll include a link to the report in the description of this podcast. On the next episode of Connected Nation, we ask, what does it really take to connect rural America? You'll hear from one company that's providing critical guidance and tools to help ISPs navigate that space and from two companies working in the field to connect homes about the challenges they face. I'm Jessica Denson. Thanks for listening. If you like our show and want to know more about us, head to connect to nation.org or look for the latest episodes of our podcast on iTunes iHeartRadio, Google Podcast, Pandora, or Spotify.