Intersectiona11y: AI, accessibility and the future of work

Solutions come in all sizes. The problem in tech (and many other industries, I presume) is that our processes and workflows are structured in such a way that the solutions for a given problem tend to be clustered around the smaller side of the scale.

Consider any given bug. Reported (hopefully) by your QA team or, worst-case, by a customer in production, it points out a specific issue. You, the developer, are tasked with devising a solution. Now, in most shops you’ll be given the opportunity to work out the root cause, ensuring that whatever change you make will a) actually fix the problem, and b) not cause any other immediate problems.

And that makes sense, for the most part. Small issues have small solutions. The problem is when you don’t step back and take a bigger-picture view of the situation - do all of these disparate problems actually stem from a particular source? Very often, developers are not only encouraged but actually mandated to stick to whatever story they’re on, for fear of going out of scope.

While that might make sense from a top-down control perspective, that style of thinking tends to permeate a lot of the other work that gets done, even up to larger-scale issues. Diversity is left to HR, or to a diversity committee, to take care of. In many cases, how and where to include AI in an application is left up to individual departments or teams. Remote work, a topic extremely divisive of late, is being eliminated or limited left up to “manager discretion” rather than actually looking at the benefits and harms that are associated with it. A cause extremely close to my heart, accessibility, is frequently treated as an add-on or left up to a handful of specialists to implement (or, worse, a third-party plugin).

These things not only don’t have to, they shouldn’t be left up to small groups to implement or reason through. They should be baked-in to how your organization makes decisions, builds software and interacts with its people.

You need a holistic approach. I want to break these concepts out of silos. If we're looking at a RACI chart, everyone is responsible for DEIB and accessibility. Everyone should be consulted and accountable for decisions about AI and remote work.

Now, I have a confession. I'm pretty sure it's Steve Jobs’ Second Law of Product that any time you think you have an insight, you have to give it a fancy name. I am guilty of this as well. 

I use the term “holistic tech” to talk about the convergence of these ideas. A lot of the specific things I'm talking about can be found in other systems or methodologies; I'm just trying to pull all the threads together so we can hopefully weave something useful about it. In the same way that responsive design was concerned with making sure you could use a product across all screen sizes, I want to make sure that (and here's the subtitle) tech works for everybody.

I'm also gonna borrow some concepts from universal design. Universal design is the concept that, "the design and composition of an environment so that it can be accessed, understood and used to the greatest extent possible by all people regardless of their age, size, ability or disability."

And last, we'll also fold in some concepts of human-centered design. This, in a nutshell, is thinking beyond your optimal user story. Eric Meyer calls them "stress cases,"  as opposed to edge cases, where you consider the emotional, physical and mental state of your user, rather just concerning yourself with the state of your application. 

But all of these, as implied with the word "design," are focused primarily on product creation. And while I do want to incorporate that, it's a part of how we work.

Basically, this whole idea boils down to a single word

EMPATHY

It's about seeing other people as, well, people.  

And it's applicable up and down your company stack. It applies to your employees, your boss, your monetization strategy (specifically, not using dark patterns), and it's especially about your communication, both within your organization and with your users.

As for product design, we'll start with accessibility.

Very broadly, accessibility is concerned with making sure that everyone can access your content and product. On the web side of things, this typically is accomplished by trying to adhere to the Web Content Access Guidelines, or WCAG. 

WCAG has four basic principles:

• The first is that content should be perceivable, which relates to multi-sensory content and interfaces. Essentially, you should still be able to access the fundamental value of the content even if you cannot engage with its primary medium; the common examples here are alt text for images or captions for videos.

• The second principle is operable: Users must be able to operate user interface controls in multiple modalities. The most common example of this is keyboard navigability; there are several requirements around people being able to video controls or manipulate modals without using the mouse (or touch).

• The third principle is understandable: Text needs to be readable and understandable, and user interface elements should behave in predictable ways. Headers should always act like headers.

• The last principle is robustness, which amounts to future-proofing. Make sure you adhere to the specs so that future products that are trying to parse your content know they can do so in a coherent manner.

Now the interesting thing is, I don't think many people would object to those principles in, well, principle. They seem pretty common-sensical? "I want people to be able to access my content" is a fairly unobjectionable statement. The problem is that most organizations don't have a good sense for accessibility yet, so the projects are designed and budgeted without the specific accessibility implementations. Then, when it gets brought up, making the change would be "too expensive," or it would "take too long."

"And besides, it's an insignificant part of our market anyway." I cannot tell you how many times I've heard this argument. Whether it's an intranet ("we don't have that many disabled people working here") or an internal training video (“there aren’t that many blind workers”) or a consumer-facing product ("we're willing to live without that tiny part of the market"), there's a sense that accessibility is only for a very small subset of the population.

My favorite group accessibility experiment is to ask people to raise their hand if they use an accommodation.

Then, I ask them to raise a hand if they wear glasses, contacts, or a hearing aid.Or if they don't keep your monitor at full resolution ("less space," on Macs). Or if they ever change their  browser's or IDE's zoom level.

Those are all accessibility accommodations.

Because the truth of the matter is, we're all just temporarily abled. I don’t ask for hands on this one don't, but I’ll often ask if anyone’s ever bought something on eBay while drunk. Formally speaking, you are technically operating with a cognitive impairment when you bought that giant taco blanket on Amazon. And I'm willing to bet your fine motor skills weren't quite up their usual par, either.

Or maybe you sprained your wrist, or broke a finger. That's a loss of fine motor control that's going to make it more difficult to operate the mouse, even if only for a few weeks. Or how about any kind of injury or chronic pain that makes it painful to sit in a chair for long periods? Willing to bet after 4 hours you're not thinking as clearly or as quickly as you were during hour 1.

Some of these things, like neurodivergence or vision impairment or being paralyzed, can be permanent conditions. But just as many of them aren't. And it's important to keep that in mind, because even if your ideal user story is a 34-year-old soccer mom, chances are she's going to have some sort of cognitive impairment (lack of sleep, stress about kids) or processing difference (trying to juggle multiple things at the same time) or fine motor skills (trying to use your mobile app on the sidelines during December) at some point. So ignoring accessibility doesn’t just disenfranchise the “small” portion of your users who are visibly permanently disabled, it's making things more difficult for potentially all of your users at some point or another.

And as it turns out, adding accessibility features can actually grow your overall market share.

Imagine your first day at NewTube, the hottest new video app on the market. We're looking to change the world … by letting people upload and watch videos. I don’t know, venture capital! Anyway, the number of humans on the internet is 5.19 billion, so that’s our addressable market. We don’t need the microscopic share that would come from adding accessibility features.

Or do we?

Standard accessibility features for videos include text transcripts of the words spoken aloud in the video. The primary intention behind these is to ensure that those with hearing impairments can still understand what’s going on in the video. In a past job, proper captions cost somewhere in the range of $20+ per minute of video, though some products such as YouTube now have AI autocaptioning that’s getting pretty good. 

Another standard feature is an audio description track (and transcript). This is sort of like alt text for video –  it describes the images that are being shown on the screen, in order to make that information comprehensible to someone with visual impairments. 

My favorite example of this is the end scene from the movie Titanic. As a transcript, it looks like this:

[ music swells ]

ROSE: Uh!

[ splash ]

Audio description, on the other hand, would look something like this:

Present-day Rose walks to the bow of the research ship, which is deserted. Deep in thought, she climbs the railing and stares down at the water where the Titanic rests below. She opens one hand to reveal the Heart of the Ocean diamond. We flash back to 17-year-old Rose standing on the deck of the Carpathia, digging her hands into Cal’s overcoat and finding the diamond. Present day Rose shakes her head and, with a small gasp, sends the diamond to rest where it should have been some 80 years earlier.

I took some poetic license there, but that’s kind of the point of audio description – you’re not a court reporter transcribing what’s being said, you’re trying to convey the emotion and the story for those who can’t see the pictures. The transcript part isn’t technically a requirement, but since you typically have to write down the script for the AD track anyway, it tends to be included. To my knowledge, no one’s managed to get AI to do this work for them in any usable fashion.

Lastly, we have keyboard navigability. Being able to interact with and control the site just using a keyboard makes it easy for those without fine motor control (or who use screen readers) to easily find their way around.

Three features/feature sets. The first two are pretty expensive - we’ve either got to pay for or develop an AI service to write the transcriptions, or we have to make sure they’re available. Audio Descriptions are going to be a cost to us, regardless, and not a cheap one. Keyboard navigability could be built-in to the product, but it would be faster if we could just throw everything together in React and not have to worry about it. 

How much of an impact could it have on our audience?

Well, though only 2-3 children out of 1000 are born with hearing impairment, by age 18 the percentage of Americans who complain of at least partial hearing loss rises to about 15%. So if we don’t have captions, we’d better hope all our videos are Fail compilations, or we’re going to see some steep drop-offs.

When it comes to vision, it’s even worse. Approximately a billion people in the world have a vision impairment that was not prevented or has not been addressed. Even assuming significant overlap with the hearing impairment group, we’ll use 750,000,000, for a total of 10.8 percent.

And for inability to use a mouse, we’ll look at “overall prevalence of adults with a month of musculoskeletal pain related to a repetitive stress injury,” which isn’t nearly a large enough category to include everyone who might be navigating by keyboard, but is at 4%.

Which leaves us 70% of our addressable market, or 3.63 billion.

Now obviously these numbers are not exact. We’re very back-of-the-napkin here, but I would also argue that a real-world scenario could just as easily see our percentages of accommodation-seekers go up as down. The number of temporary cases of all of these items, the fact that first-world countries have higher prevalence of RSI (though much better numbers for vision impairment) mean that this 70% number is probably not as far away from reality as we think.

And even beyond people who need those accommodations, what about those who simply want them? 

My best friend watches TV with the captions on all the time because it’s easier for her to follow along, and she’s not alone. Netflix says 40% of global users watch with captions, to say nothing of public exhibitions like bars (where it’s often not legally permissible to have the sound on). 

Transcripts/audio descriptions are often HUGE boons to SEO, because you’re capturing all your content in a written, easily search-indexable format. 

And presumably you’ve used a video app on a TV. The app has already been designed to be used with directional arrows and an OK button - why not extend that to the desktop? You’ll notice the remote’s functionality is a subset of a keyboard, not a mouse. Boom, keyboard navigation.

So, to recap accessibility: Good for disabled users. Good for abled users. Good for business. And that’s the thing, taking a holistic approach to how we do tech should actually make everyone better off. It is the rising tide.

But let’s talk about the looming wave that overshadows us all. I speak, of course, of artificial intelligence. In the same way that software ate the world 15 years ago, and Bitcoin was going to replace all our dollars, artificial intelligence is going to eat all our software and all the dollars we software developers used to get paid. 

I want to make clear up front that I am not an AI doomsayer. I don’t think we’re (necessarily) going to get Skynetted, and if we are it’s certainly not going to be ChatGPT. Artificial intelligence in its current form is not going to enslave us, but I do think large swaths of the population will become beholden to it – just not in the same way. 

Similar to how algorithms were used in the 90s and 2000s to replace human decision-making, I think AI is going to be (ab)used in the same manner. We’ve all called in to a customer support line only to find that the human on the other end is little more than a conduit between “the system” and us, and the person can’t do anything more to affect the outcome than we can. 

With AI, we’re just going to skip the pretense of the human and have the AI decipher what it thinks you said, attempt remedies within the limits of what it’s been programmed to allow, and then disconnect you. No humans (or, likely, actual support) involved.

Is that the worst? Maybe not in all cases. But it’s also, in a lot of cases, going to allow these organizations to skip what should be important work and just let the AI make decisions. I’m much less concerned about SkyNet than I am the Paperclip Maximizer.

The paperclip maximizer is a thought experiment proffered by Nick Bostrom in 2003. He postulated that an AI given a single instruction, “Make as many paperclips as possible,” would/should end with the destruction of the entire earth and all human life. The AI is not given any boundaries, and humans might switch the machine off (thus limiting the number of paperclips), so the AI will eventually eliminate humans. But even if the AI thinks us benign, at some point the AI consumes all matter on the earth aside from humans, and we are just so full of wonderfully bendable atoms that could be used for more paperclips.

The “thought processes” of generative AIs, as currently constructed, are inherently unknowable. We know the inputs, and we can see the outputs when we put in a prompt, but we can’t know what they’re going to say - that’s where the special sauce “thinking” comes in. We try to control this by introducing parameters, or guidelines, to those prompts to keep them in line.

And I know you might think, “Well, we’ll tell it not to harm humans. Or animals. Or disrupt the existing socio-political order. Or …” And that’s actually a separate angle to attack this problem - humans not giving the proper parameters. At a certain point though, if you have to control for the entire world and its infinite varieties of issues, isn’t it easier to just do the work yourself? We’ve already got a lackluster track record in regard to putting reliable guardrails around AI, as the Bing Image Generator’s output so thoughtfully proves. 

One of the things computer nerds love to do more than anything is break new tech, and image generators are no exception. When it introduced a new image generation tool a while back, though Bing did restrict uses of the phrase “9/11" or “September 11,” it still allowed for image generations of “Spongebob flying an airliner into New York in 2000.” And of course, the most prominent image of New York in 2000 is likely going to include the World Trade Center.

Sure, Spongebob doing 9/11 is a brand hit to Nickelodeon and insulting to the victims’ families. But this is showing both failures - despite Bing’s overwhelming image consciousness that should have been baked into a model, the model thought it more important to generate this image than to not. And, separately, Bing failed to put proper safeguards into the system. 

So yes, the paperclips are a hyperbolic hypothetical, but if there’s one thing that capitalism has taught us it’s that there are companies out there who care more about the next dollar than anything else.

Businesses large and small make decisions based on weighing costs versus expected benefits of a given option all the time. Famously, with the Ford Pinto, one of the analyses Ford conducted cited the overall cost of redesigning fuel safety systems vs. the general cost to society of the fatal car crashes that might be spared. Because, to Ford, individual deaths were not thought of as particularly tragic. They were just numbers. It does not seem unreasonable to assume AI systems will be misused by those who are unscrupulous in addition to those who are just oblivious. 

In accessibility, most people think the cost of not being accessible is “well, how likely are we to get sued?” ignoring the benefits of people using the product more. With AI, this short-sighted calculus can come into play where “Oh, we’ll let the AI do it, and not have to pay a person!” Except, as we’ve pointed out, the AI probably isn’t very good and the cost comes in consumer goodwill.

And this doesn’t even touch things like source data bias, which is a huge issue in resume-reviewing AIs (whose datasets will cause the AI to be more likely to select for existing employees, exacerbating skewed hiring trends) and predictive policing algorithms (which exacerbate existing crime biases). 

Don’t forget you can now convincingly generate human-sounding responses in astroturfing campaigns or review spoofing, or empower scammers previously held back by non-native English suddenly sounding like every corporate communication (because AI’s probably writing those communiques, too).

Remember the part where I said I’m not an AI doomsayer? I’m really not! I think AI can be used in a lot of unique and interesting applications to make things better. We just need to be more judicious about how we employ it, is all. 

For example, in the medical field, there are numerous AI experiments around trying to find tumors from body scans; the AI is not notifying these people on its own, there are doctors who review flagged scans for closer examination. Or in drug trials, companies are using AI to imagine new shapes of proteins that will then have lots of trials and study before they’re ever put in a test subject. 

Using AI to generate advice that is then examined by humans for robustness is a great application of the tool. And sure, if Amazon wants to use AI to suggest product recommendations, I guess go ahead. It can’t be any worse than its current system of, “Oh, you bought a refrigerator? I bet you also want to buy several more.”

But that “generation” word is a sticking point for me. To the point of the job applicant winnowing, I have no problem with using quantitative questions to weed out applicants (do you have x years of experience, boolean can you work in the US), but I would hesitate to let a black-box system make decisions even as small as who should be considered for hiring based on inherently unknowable qualifications (as would be the case with the application of actual AI versus just algorithmic sifting).

And finally, just limit the use of generated content in general. Reaching back into my accessibility bag for a minute, there’s a class of images that per spec don’t need alt text: Images that are “purely” decorative and not conveying information. The question I always ask in such cases is: If the image is really providing no value to the user, do you really need it?

The same would go for unedited generated content. If you’re sending a communication that can be wholly generated by a computer, do you really need to send it? We’re taking the idea of “this meeting could have been an email” even further down the stack: Could that email in fact just be a Slack message, or better yet a reaction emoji? Just because you can expand your one-sentence idea more easily with AI doesn’t you have to or even should.

There’s likely a place for generated content, but it’s not anywhere near where we’re using it for now, with AI-generated “news” articles or advertising campaigns. It’s like when we just tried to add accessibility “with a button” - you cannot just throw this stuff out there and hope it’s good enough.

And I would hope it would go without saying, but please don’t replace therapists or lawyers or any other human who considers ethics, empathy, common sense or other essentially human traits with AI.

This is along the same lines as “generate advice not decisions,” - if you need to talk to the AI in order to be comfortable sharing things with a live person, that makes total sense. But don’t use the AI as a 1:1 replacement for talking to a person, or getting legal advice.

AI recap: Good for advice, not decisions. Good for assisting people, not replacing them (it’s a tool, not the mechanic). It can be good for business.

Now, I think at this point you can pretty much guess what I’m gonna say about remote work. And that’s good! Both because this is already long enough and because “holistic tech” is supposed to be a framework, not just specific actionable items. 

Remote work, of course, is the idea that you need not be physically present in a building in order to perform a job. Hybrid work is a mix of remote work with some time spent in the office. I’m not gonna try sell you hard on either option - but I will note that employees prefer flexibility and employers tend to enjoy the larger talent pool. But mostly, I want to talk about how to set up your organization for success in the event you choose one of them.

One of the issues when you have some people in the office and others who aren’t is the sense that employees in the office are prioritized above those who are remote. Some of this is understandable – if the company wants to incentivize people to come into the office by offering, for example, catered lunches once a week or something, I wouldn’t see that as something that those who aren’t attending are missing out on …. Unless they were hired as fully remote.

In my case, for example, company HQ is in Chicago; I live in Phoenix, Arizona. I was hired fully remote, and it would feel to me like I were a lesser class of employee if those in the Chicago area were regularly incentivized with free lunches when there’s no pragmatic way for me to partake. Luckily, our office uses a system where everyone gets the same amount of delivery credit when we have whole-office lunches, which allows all of us to feel included. 

Beyond incentives, though, is the actual work being done, and this is where I think some teams struggle. Especially when it comes to meetings, the experience of the remote attendee is often an afterthought. This can take the forms of whiteboarding (literally writing on a whiteboard off-camera in the room), crosstalk or side discussions that aren’t in listening range of the microphone, or showing something on a screen physically that’s not present virtually.

It’s not just “don’t punish your remote team members for being remote,” you’re actually hurting the organization as a whole. Presumably every member of the team was hired with an eye to what they can bring to the table; excluding them, or not giving them the full information, hurts everyone involved. 

And technological solutions for remote workers will benefit in-person workers as well! Talking into the microphone during meetings can help someone with cochlear implants hear better in the room just as much as it’ll help me sitting in my garage office 1200 miles away. Same goes for whiteboarding - having a Google Jam (is that where they’re called anymore? Bring back the Wave!) on their screen means my wife can actually follow along; if they have to read a whiteboard from even 14 feet away, they’ll lose track of what’s going on in the meeting.

Taking the time to plan for how the remote attendee’s experience helps everyone, and it’s not terribly difficult to do. You can even see it for yourself by simply attending the meeting from another room to give you perspective and help troubleshoot any issues. Part and parcel of this, of course, is investing in the tools necessary to make sure everyone can interact and collaborate on the same level.

It’s not all about managers/employers, though! Remote employees tend to think that remote work is just like being in the office, only they don’t have the commute. And while that’s true to some extent, there’s another crucial aspect that many of them are missing: Communication. 

You have to communicate early and often when you’re remote for the simple reason that no one can come check up on you. No one can look over your shoulder to see if you’re struggling, no one knows intuitively what your workload looks like if you’re overloaded. Similarly, you don’t know what impacts your coworker’s commit is going to have unless you ask them. There are any number of tools and video sharing apps and all that, but the upshot is you actually have to make focused efforts to use them to make sure everyone’s rowing in the same direction.

Remote work: good for employees, good for employers. Good for business.

Finally, let’s talk diversity. Commonly abbreviated DEI, or DEIB, diversity, equity, inclusion and belonging has sort of morphed from “let’s make sure our workforce looks diverse” to “let’s make sure people of different backgrounds feel like they have a place here.”

And that’s because DEIB should be a culture, not an initiative. At the start, we talked about silos vs. intersectionality. This might start with a one-off committee, or an exec hire, but true DEIB is about your entire culture. Just like remote work can’t just be HR’s problem, and AI decisions shouldn’t be made solely by the finance team, DEIB needs to come from the entire organization.

I actually like the addition of the B to DEI because Belonging is a pretty good shorthand for what we’ve been discussing throughout. People who are temporarily or permanently disabled are provided the accommodations they need to succeed and thrive; programmers aren’t worried AI is going to be used to replace them, but instead given to them as a tool to increase their productivity. Remote workers feel like the company values them even in a different state.

DEIB should encompass all those things, but it can’t be left up to just a committee or an exec or even a department to account for it. It all falls on all of us.

And I specifically don’t want to leave out the traditional aspects of diversity, especially in tech culture. Minorities of all kinds – women, nonbinary folks, other gender identities, those of different sexual orientations, non-white racial backgrounds – are underrepresented in our industry, and it’s important that we keep up the work required to make sure that everyone is given the same access and opportunities. 

It’s good for business, too! Having a diverse array of perspectives as you develop products will give you ideas or user stories or parameters a non-diverse group might never have thought of. We keep hearing stories about VR headsets that clearly weren’t designed for people with long hair, or facial recognition algorithms that only work for those with lighter skin tones. If your product serves everybody, your product will be used by more people. That’s basic math!

Recent court rulings have put a damper on what used to be the standard for diversity, a “quota” of either applicants or hires meeting certain criteria. And look, if your organization was hiring just to meet a metric, you didn’t have true diversity. Quotas don’t create a culture of inclusion, so them going away shouldn’t cause that culture to dissipate, either. Seek out diverse upstreams for your hiring pipeline, ensure you’re not just tapping the same sources. I promise you, that investment will provide a return.

Say it with me: DEIB is good for employees, good for employers, and it’s good for business.

TLDR: Have empathy. Make sure you consider all aspects of decisions before you make them, because very often taking the personhood of the other party into account is actually the best business move as well. 

And with all of these, please note that when I say these are good for employees, good for employers and, especially, “good for business” requires these ideas to be executed well. Doing it right means taking as many of these factors into account as you can. This is where holistic tech comes in as our overarching concept.

• When it comes to accessibility, the more you have, the more customers you can reach and the more options you give them. With a long lens, that tends to mean you wind up with more money. 

• When you’re considering applications for artificial intelligence, try to keep its influence to advice rather than making decisions, and consider the work that would need to be done in order to implement the solution without AI – if it’s not work you’re willing to do, is it worth doing at all, AI or no? 

• With remote work, you need to invest the time and resources to ensure your remote employees have the tools to communicate, while employees need to invest the time and energy to actually communicate.

• Finally, diversity and belonging are about your culture, not a committee or a quota. Invest in it, and you’ll reap rewards.