ATD Blog

Assessing Digital Literacy—A Framework for Developing a General Measure

By and

Mon Mar 02 2015

Assessing Digital Literacy—A Framework for Developing a General Measure

Content

In January 2015, ATD offered the webinar, Helping Learners Learn in the Digital Era , during which participants asked a number of thought-provoking questions. In addition to inquiring about how they, as learning professionals, could increase their own digital literacy, they wanted to know how to:

In January 2015, ATD offered the webinar, Helping Learners Learn in the Digital Era, during which participants asked a number of thought-provoking questions. In addition to inquiring about how they, as learning professionals, could increase their own digital literacy, they wanted to know how to:

Content
make a strong business case and sell leadership on the importance of a digitally literate workforce
make a strong business case and sell leadership on the importance of a digitally literate workforce
Content
create institutional support for digital literacy initiatives
create institutional support for digital literacy initiatives
Content
motivate individuals to become more digitally literate.
motivate individuals to become more digitally literate.

Content

Underscoring these queries were questions about assessing digital literacy, including: “Are there particular assessments to get a sense of skill set for a diverse organization—from production line to R&D engineers to sales and marketing?” or “Does everyone need the same skill set?”

Content

In addition, one participant asked about ATD’s potential role in developing some kind of tool for assessing digital literacy.

Content

Although it’s not practical to create a single measure to assess the digital literacy of workers at all levels and in all functional areas, it is possible to create a general measure to assess basic competencies. Here’s our take on developing a framework for creating this sort of measure.

Content

Defining Digital Literacy

Defining Digital Literacy

Content

Before designing a way to measure digital literacy, it’s important to define it.

Content

Both the American Library Association and Wikipedia provide solid working definitions of digital literacy. Here is Wikipedia’s definition: “A digitally literate person will possess a range of digital skills, knowledge of the basic principles of computing devices, skills in using computer networks, an ability to engage in online communities and social networks while adhering to behavioral protocols, be able to find, capture and evaluate information, an understanding of the societal issues raised by digital technologies (such as big data), and possess critical thinking skills.”

Content

Although definitions like this are focused on digital literacy in a global sense, the core concepts can easily be applied to the context of work.

Content

Assessing Digital Literacy—Four Components and Three Focal Areas

Assessing Digital Literacy—Four Components and Three Focal Areas

Content

In general, workplace digital literacy is comprised of four hierarchical components. The first three focus on basic knowledge and understanding, as well as organizational and individual applications. The fourth component focuses on related skills and the ability to leverage digital technology effectively.

Content
Digital era concepts. Focuses primarily on job-related communication and collaboration, such as platforms, channels, content creation and curation, crowdsourcing, cloud computing, and cybersecurity.
Digital era concepts. Focuses primarily on job-related communication and collaboration, such as platforms, channels, content creation and curation, crowdsourcing, cloud computing, and cybersecurity.
Content
Digital tools and systems. Digital tools include the obvious—email, chatting/instant messaging, the Microsoft Office suite of products (and equivalents), as well as tools like photo and video editors. Systems include software applications developed for specific purposes, such as accounting, business intelligence, and learning management.
Digital tools and systems. Digital tools include the obvious—email, chatting/instant messaging, the Microsoft Office suite of products (and equivalents), as well as tools like photo and video editors. Systems include software applications developed for specific purposes, such as accounting, business intelligence, and learning management.
Content
Social technology features, platforms, and tools. A sampling of social technology features include blogs, customized aggregators, dashboards and portals, discussion forums/threads, media sharing features, user-generated profiles, and wikis. Platforms and tools include obvious public networks like LinkedIn, Twitter, and YouTube, but also tools like Disqus, ShareThis, and privately oriented offerings such as Yammer, Jive, and Interact Intranet.
Social technology features, platforms, and tools. A sampling of social technology features include blogs, customized aggregators, dashboards and portals, discussion forums/threads, media sharing features, user-generated profiles, and wikis. Platforms and tools include obvious public networks like LinkedIn, Twitter, and YouTube, but also tools like Disqus, ShareThis, and privately oriented offerings such as Yammer, Jive, and Interact Intranet.
Content
Digital engagement skills and tactics. This component focuses on the skills required to use social and digital technologies efficiently, as well as the necessary judgment to use them effectively. Examples include knowing the right channel to use for a given communication, using email productively, creating and engaging productively in discussion threads and forums, curating and validating content, contributing to a wiki, and understanding HTML basics.
Digital engagement skills and tactics. This component focuses on the skills required to use social and digital technologies efficiently, as well as the necessary judgment to use them effectively. Examples include knowing the right channel to use for a given communication, using email productively, creating and engaging productively in discussion threads and forums, curating and validating content, contributing to a wiki, and understanding HTML basics.

Content

In considering how to develop a general measure for assessing workplace digital literacy, there are three focal areas of primary concern for employers.

Content
Communication and collaboration. The ability to communicate and collaborate with others using digital technology is critical in enabling an organization to function both efficiently and effectively. This includes internal communication with colleagues and external communication with clients and others.
Communication and collaboration. The ability to communicate and collaborate with others using digital technology is critical in enabling an organization to function both efficiently and effectively. This includes internal communication with colleagues and external communication with clients and others.
Content
Cybersecurity . Individuals have typically been found to be the weakest link in protecting an organization in cyberspace. Understanding the risks and engaging in the right behaviors creates a strong first line of defense against hackers, viruses, and other digital threats.
Cybersecurity. Individuals have typically been found to be the weakest link in protecting an organization in cyberspace. Understanding the risks and engaging in the right behaviors creates a strong first line of defense against hackers, viruses, and other digital threats.
Content
The law and ethics . Workers now have added responsibilities with respect to things like protecting an organization’s brand, intellectual property, and trade secrets; maintaining proper levels of confidentiality; and ensuring the privacy of clients, fellow employees, and other stakeholders. These responsibilities apply not only to their digital engagements while on the job, but also can extend to their non-work activities.
The law and ethics. Workers now have added responsibilities with respect to things like protecting an organization’s brand, intellectual property, and trade secrets; maintaining proper levels of confidentiality; and ensuring the privacy of clients, fellow employees, and other stakeholders. These responsibilities apply not only to their digital engagements while on the job, but also can extend to their non-work activities.

Content

Building a Measure to Assess Digital Literacy

Building a Measure to Assess Digital Literacy

Content

Any tool for assessing digital literacy should include questions that address the relevant components in the context of each of the three focal areas. Here are some examples:

Content

Communication and collaboration:

Communication and collaboration:

Content
Digital Era concepts : What is an enterprise social network? What is crowdsourcing?
Digital Era concepts: What is an enterprise social network? What is crowdsourcing?
Content
Digital tools and systems : What is the main difference between texting and chatting/instant messaging?
Digital tools and systems: What is the main difference between texting and chatting/instant messaging?
Content
Social technology features, platforms, and tools : What are the basic elements of a user profile?
Social technology features, platforms, and tools: What are the basic elements of a user profile?
Content
Digital engagement skills and tactics : What is the best digital means for creating and conducting that conversation? What are the basic steps for adding a hyperlink to text or an image in a Microsoft Word, PowerPoint, or Excel file? Is it okay to express strong negative emotions via digital channels?
Digital engagement skills and tactics: What is the best digital means for creating and conducting that conversation? What are the basic steps for adding a hyperlink to text or an image in a Microsoft Word, PowerPoint, or Excel file? Is it okay to express strong negative emotions via digital channels?

Content

Cybersecurity:

Cybersecurity:

Content
Digital Era concepts : What is phishing? What is hacking? How is malware different from a virus?
Digital Era concepts: What is phishing? What is hacking? How is malware different from a virus?
Content
Digital tools and systems : What is the primary way an individual can expose an organization to malware or a virus?
Digital tools and systems: What is the primary way an individual can expose an organization to malware or a virus?
Content
Digital engagement skills and tactics : If you suspect your computer has been infected by a virus or malware, what’s the first thing you should do? What’s a good password to use on your mobile device? If you access work systems remotely, should you make sure you’re on a secure channel first?
Digital engagement skills and tactics: If you suspect your computer has been infected by a virus or malware, what’s the first thing you should do? What’s a good password to use on your mobile device? If you access work systems remotely, should you make sure you’re on a secure channel first?

Content

The law and ethics:

The law and ethics:

Content
Digital tools and systems : How can trade secrets be leaked via email?
Digital tools and systems: How can trade secrets be leaked via email?
Content
Digital engagement skills and tactics : If you’re discussing the organization or one of its brands on a public social network (for example, LinkedIn), should you disclose your working relationship? Is it appropriate to discuss clients on your personal social networks (for example, Facebook), even if you don’t name names? If you suspect a confidentiality leak, what is the first step you should take to report the loss or compromised information?
Digital engagement skills and tactics: If you’re discussing the organization or one of its brands on a public social network (for example, LinkedIn), should you disclose your working relationship? Is it appropriate to discuss clients on your personal social networks (for example, Facebook), even if you don’t name names? If you suspect a confidentiality leak, what is the first step you should take to report the loss or compromised information?

Content

Additional design considerations:

Additional design considerations:

Content
Questions can be true/false, multiple-choice, and open-ended.
Questions can be true/false, multiple-choice, and open-ended.
Content
The assessment can include specific skill exercises, such as sharing a link to a web page or starting a discussion thread.
The assessment can include specific skill exercises, such as sharing a link to a web page or starting a discussion thread.
Content
To the extent possible, questions should be contextually independent, although different versions of the same question may need to be created to adapt to different browsers (Internet Explorer or Chrome) and operating environments (Windows or Mac). Some answers may need to be tweaked to reflect an organization’s unique policies, procedures, and practices.
To the extent possible, questions should be contextually independent, although different versions of the same question may need to be created to adapt to different browsers (Internet Explorer or Chrome) and operating environments (Windows or Mac). Some answers may need to be tweaked to reflect an organization’s unique policies, procedures, and practices.
Content
The assessment should probably be weighted toward skills and tactics, including assessments of judgment, etiquette, and ethics. In other words, it should address the question: What’s the right thing to do in a particular situation?
The assessment should probably be weighted toward skills and tactics, including assessments of judgment, etiquette, and ethics. In other words, it should address the question: What’s the right thing to do in a particular situation?
Content
To motivate people and make the test more palatable, the assessment could take the form of a digital scavenger hunt, with gamification elements to reward people for correct answers and accomplishments.
To motivate people and make the test more palatable, the assessment could take the form of a digital scavenger hunt, with gamification elements to reward people for correct answers and accomplishments.
Content
Questions should have clear right/wrong answers to enable the creation of strong cutoffs between different competency levels, as well as to establish baseline measures and standards.
Questions should have clear right/wrong answers to enable the creation of strong cutoffs between different competency levels, as well as to establish baseline measures and standards.

Content

Your Thoughts?

Your Thoughts?

Content

Creating a tool for assessing digital literacy is certainly possible, but it will not be easy. There are plenty of nuances associated with the measurement components, and you will need to address lots of details. Not to mention, you will need to test the assessment to ensure the tool is both valid and reliable.

Content

As you start to develop your assessment, be sure to ask:

Content

Does this first cut at creating a general digital literacy assessment make sense to you?

Content
What other components should be included?
What other components should be included?
Content
What other focal areas should be considered?
What other focal areas should be considered?
Content
How would you tweak the design?
How would you tweak the design?

Content

We welcome your feedback and ideas.

You've Reached ATD Member-only Content

Become an ATD member to continue

Already a member?Sign In