Using Data to Advance Digital Skills: A State Playbook

Develop guiding questions: The creation of guiding questions can help state leaders refine a scope for understanding digital skill needs in their states, including their top priorities, populations, or programs and services for review. Such guiding questions can be used to inform data collection as well as help leaders understand the types of stakeholders that need to be engaged throughout the planning process. Sample questions for states include:

• What information does our state need to inform our broader vision for developing digital skills among state residents?
• Who are the people in “covered populations” as defined under Digital Equity Act Sec. 60302(8) (see Glossary), and how do we best measure digital skill assets and needs within each of those populations?
• Where are the greatest gaps in digital skills programs including, but not limited to, foundational digital literacy training?
• What are the skills workers need to meet the needs of the local labor market?
• How can businesses play a role in helping to analyze data on digital skills needs in the workforce?

Determine what to measure: Once guiding questions are developed, the next step is to define key performance indicators that can help to answer the guiding questions, including:

• Digital literacy: Define what it means to be digitally literate by applying existing definitions (see Glossary) for “workforce preparation” including digital skills and reviewing existing digital skill competencies frameworks. Digital skill definitions range from focusing on the discrete skills needed to navigate a computer or online environment to skills that are harder to measure, such as confidence and capacity to become lifelong learners of new technologies.
• Digital skills in demand: This data can be gathered by tapping into states’ existing systems and processes for determining local employers’ workforce needs, such as state labor market information agency data; local workforce boards’ employer relationships; local chamber of commerce member surveys; and/or industry sector partnerships that bring together multiple employers along with an education or training provider.

Digital skills baseline data: Data on the existing digital skill levels of residents, particularly disaggregated by the “covered populations,” is difficult to obtain at the state and local level. States should consider reviewing available national data (see Commonly Used Datasets Matrix) and using those numbers as a starting point to apply to state residents. To more accurately “downscale” the national data, states should compare proportional differences in key demographic characteristics —  such as educational attainment, poverty, or limited English proficiency —  between the state’s population and that of the nation, using sources such as the Decennial Census and American Community Survey. Finally, states can look at their own administrative data (see Digital Skills Beyond the Workforce) to help them round out the picture of state residents’ digital skill abilities and needs.

Inventory existing datasets: States have a variety of data sources to inform their digital skills work, including education data, employment data, labor market information, public benefits data, and workforce program data. View the Commonly Used Datasets Matrix to learn more about how other datasets can be used to measure digital literacy and skill demands, and identify which covered populations are in most need of support.

• Education data includes individual student records that schools and colleges, from preschool to postsecondary, keep to track the progress and development of students, including completion and types of degree or credential earned. This information can sometimes be made available in aggregate, non-identifiable form to other state agencies and stakeholders, such as a state higher education coordinating board.
• Employment data is collected by state agencies, most often on a quarterly basis, to inform the state and the public about the employment status and earnings of residents in the state or region. Again, aggregate data is often made available to stakeholders through the state agency in charge of overseeing the Unemployment Insurance system or the state’s Labor Market Information agency.
• Labor market information is the analysis of job markets by local areas, regions, or states based on real-time job postings of businesses and industries. Each state has its own labor market information agency, and private companies (i.e. Burning Glass or LinkedIn) also help states and cities make these real-time analyses of labor market data.
• Public benefits data refers to information on individuals or families who receive public assistance, including unemployment insurance benefits, Temporary Assistance for Needy Families (TANF), or Supplemental Nutrition Assistance Program (SNAP). The data collected include the time and amount of assistance received and may also include data about education or workforce training programs in which they have participated.
• Workforce program data includes records from state or federally funded workforce development programs that have reporting requirements, as well as programs sponsored through different industry associations, career technical education (CTE), apprenticeships, and adult education services. This includes programs funded through the federal Workforce Innovation and Opportunity Act (WIOA).

Identify what additional data needs to be collected: Once states have conducted an initial review of available data, it is important to identify gaps in the data and determine which gaps most need to be filled to develop effective Digital Equity Plans. States should distinguish between crucial data that is essential to have, and optional data that is helpful but not essential. Being as rigorous and specific as possible in documenting data gaps will help states make wise decisions in allocating their resources to fill these gaps.   For example, a state may identify a lack of data on rural residents’ digital skill needs and decide to prioritize data collection in rural regions by “over-sampling” survey respondents or partnering with additional rural organizations to gather data. As states prepare Digital Equity Plans, documenting gaps in data can provide valuable justification for implementation funds to support additional data collection efforts.

Develop a data collection plan: States should determine method(s) of data collection, who will be responsible for carrying out the plan, and what resources are available to implement. Most importantly, all data included in the data collection plan should directly contribute to answering one or more of the guiding questions discussed above. Including documentation of the link between each dataset and the question(s) it helps to answer can keep data collection efforts efficient and targeted, particularly with less structured qualitative methods. States should distinguish between analysis of existing datasets (which is cheaper but offers less precision) and new, original data collection (more expensive, but can be precisely targeted to the specific data a state needs). Data collection strategies include:

• Analyzing and modifying existing data sources (listed above)
• Collecting new quantitative data via surveys and/or direct assessments
• Collecting new qualitative data via interviews and focus groups (which can add texture and context and validate findings from quantitative research)  

Leverage partners for data collection and analysis: Partnerships are crucial for enabling education and workforce systems to help individuals develop digital literacy and occupational skills. States can consider engaging a number of state and local partners based on what data needs to be collected. For example, states interested in gathering more data about the changing demands of skills needs from employers may wish to leverage partners at the state and local workforce development boards, institutes of higher education, and state departments of labor. Alternatively, states interested in learning about the gaps in foundational digital literacy may wish to leverage partnerships with adult basic education organizations and state and local libraries.

States should consider which of the following types of partners can best help them accomplish their goals:

• Labor Market Information agencies, typically housed at state departments of labor
• State workforce agencies operating reemployment programs (WIOA Title I), typically housed at state departments of labor
• State and local Adult Basic Education programs (WIOA Title II)
• State library associations and local public libraries
• State and local workforce development boards
• Institutions of higher education and state higher education coordinating boards
• Governor appointed state infrastructure advisors/coordinators
• Governors’ broadband leaders, including heads of State Broadband Offices 
• Other divisions of government services (see Digital Skills Beyond the Workforce)

States may also consider partnering with organizations across the country that collectively comprise the “digital inclusion ecosystem,” such as the National Digital Inclusion Alliance and the Digital US Coalition led by World Education.

Establish baseline data: After layering data sets to better understand the digital skills of state residents and how these findings may differ from those identified in national data, states can develop baseline data against which to measure progress.

When developing baseline data, states should consider the following:

• Identify ‘covered populations’ in states and communities, such as those listed in the Digital Equity Act (see Glossary)
• Map existing assets
  • – Identify systems (e.g., public benefits, education, workforce development) and organizations (e.g., community-based organizations) currently providing digital equity services (including access, adoption, skill building, navigation) to covered populations
  • – Identify existing and pending digital equity initiatives (e.g., digital navigation, digital skill building, digital access) serving ‘covered populations’ which can help to collect or illuminate data
• Establish measurable objectives in digital literacy to collect within each covered population, as required by the Digital Equity Act

Turn data into stories: Data collected via the above-outlined process can help inform a state’s Digital Equity Plan, but can also form part of a larger aspirational vision for digital equity. States should capitalize on the rich array of data they are collecting and connect it to larger, existing stories, such as the story of a state’s goals for postsecondary attainment, a state’s journey toward closing racial equity gaps, or a state’s vision for connecting small and rural communities. For example, states may consider strategies to visualize gaps in digital equity and skill development using software, such as ArcGIS mapping, to highlight achievements or locations of policy focus.

Make data available for others’ storytelling: States should make every effort to provide aggregated, anonymized data in ways that nonprofit advocates and other stakeholders can use. By providing raw data via website or Application Programming Interface (API), states can make it easy for stakeholders to piggyback on the state’s prior work and use it to support shared goals and visions.

State Digital Equity Scorecard

The State Digital Equity Scorecard, created by the National Digital Inclusion Alliance (with support from Microsoft and the National Skills Coalition), offers states a public score based on the extent to which states are addressing digital equity across six key indicators:

• data on digital skills needs
• plan to close digital skill gaps
• online digital skills training
• incumbent worker training funds
• technology apprenticeships
• state broadband plans

States can use the interactive map to explore the methodology behind these factors, including links to original national and state datasets on state level job openings, openings requiring digital skills, and more.

Incorporate findings into state plans: While states are putting in extensive effort to gather and analyze digital skills data, it is imperative to close the feedback loop and share that final data with workforce development, adult education, and higher education partners. States can leverage existing collaborative planning processes to ensure that findings are incorporated into regularly scheduled efforts such as WIOA plans, Perkins Career & Technical Education Act plans, etc.

Distribute findings to local recipients and service providers: States can leverage local partnerships to disseminate findings to local recipients of workforce and education funds, and to chambers of commerce, industry associations, and other business stakeholders. States may also consider partnering with municipalities to distribute local-level data and develop recurring feedback loops to inform state and regional goals.

Hawai‘i Digital Literacy and Readiness Study: Case Study

State of Hawai‘i Department of Labor & Industrial Relations Workforce Development

In 2021, the Hawai‘i Department of Labor and Industrial Relations (DLIR) sought to gather more information about the state of digital literacy and readiness among Hawai’i residents between ages 18 to 65 across the state. After approving definitions including “digital literacy” and “digital readiness,” the Workforce Development Council (WDC) launched an initiative to develop a statewide benchmark to quantify the digital readiness and literacy levels among residents of the State of Hawai‘i.

The objectives of the Digital Literacy and Readiness Study (DLRS) included:

• Establishing a baseline measurement of the digital literacy and readiness of Hawaii‘s working age population;
• Measuring the present rate of digital readiness statewide, disaggregated by demographics, education, occupation, industry, and geography;
• Determining the proportion of adults who are not digitally literate; and
• Identifying the largest population segments of adults who are not digitally ready.

The state measured literacy and readiness by demographics, education, occupation, industry, and geography and partnered with a private partner, Omnitrak, to collect individual-level survey data via landline and cell computer-assisted telephone interviewing and an online survey. As digital readiness has expanded beyond its initial hardware-oriented definition, self-efficacy, attitudes, and behaviors have become important dimensions in preparing residents for the digital age. This study analyzed underlying similarities and differences based on respondents’ self-assessment on seven self-reported aspects of readiness. Using cluster analysis, WDC identified five distinct groups of residents who fall along a continuum of digital readiness from relatively more ready to relatively unprepared based on geography, education, and other demographic features. 

Survey results found that Hawai‘i shows relatively lower rates of digital readiness compared to a national benchmark digital readiness survey conducted by the Pew Charitable Trust in 2015. With results from the study, the state proposed several recommendations, including a new Framework for Hawai’i Digital Literacy and Readiness. In partnership with employers, this Framework will help establish a baseline measurement of the current digital transformation stage of Hawai’i businesses and understand digital skills needed by the employers in key industries affected by digitization.

North Carolina’s Broadband Infrastructure Office of the Department of Information Technology, in partnership with the National Telecommunications and Information Administration, developed the  Digital Inclusion Plan Template and Guide to support North Carolina communities as they develop digital inclusion plans and digital equity strategies. This document includes both a template with the specific categories a community should consider including in their plan as well as a guide with probing questions to assist communities in the process of building that plan.

Oklahoma In an effort to gauge the role digital skills play in state occupational earnings and anticipated demand for those skills in the future, the Economic Research & Analysis Division of the Oklahoma Employment Security Commission recently conducted a study for the NGA Workforce Innovation state team. The Division used the digital skills scoring methodology from a 2017 Brookings Institution study and combined it with state data on job openings and wages produced by the division’s Bureau of Labor Statistics Occupational Employment & Wage Program and the Department of Labor Workforce Information Grant Projections. The study found a modest positive relationship between higher digital scores and average wage levels – offering the state new projections about how digital skills are related to occupational demand and economic outcomes, and new evidence about the importance of Oklahoma’s work to bridge the digital skills gap. State leaders plan to leverage this data and the NGA WIN grant to better prepare and inform the state’s strategic policy and programs to ensure a thriving, digitally skilled workforce.